Back to Draft concepts

Game META theory

A draft about populations of strategies, game graphs, simplex dynamics, and categorical operations.

Editorial note: this text began as original work. In retrospect, it appears to independently rediscover part of the theory now recognizable as the replicator dynamics of Taylor and Jonker, or at least a closely related zero-sum evolutionary-game formalism. The rendering below preserves the exploratory status of the source text, including open questions and uncertain passages marked with (?).

Original date: December 2021.

Original PDF: Teoria del Meta.pdf

Contents

  • 1. Introduction
  • 1.1 Motivation
  • 1.2 Passage to the continuous model
  • 1.3 Formalization
  • 2. General characteristics
  • 2.1 Global existence and uniqueness of the solution
  • 2.2 Conservation of players
  • 2.3 Subgames at the boundary
  • 2.4 Nontrivial stationary points
  • 2.5 Mirror games
  • 3. Small games
  • 3.1 Conversion
  • 3.2 Rock-paper-scissors
  • 3.3 Chain of two conversions
  • 4. Category of Games
  • 4.1 Quotients
  • 4.2 Coproducts
  • 4.3 Products
  • 5. To-do list

1. Introduction

1.1 Motivation

Think of the following situation: many players play a game for which several strategies exist, and the probability that a given strategy prevails over another is determined. Draws are not possible. Players challenge one another randomly in pairs, and a defeated player begins to use the strategy against which they lost. Our goal is to model this problem and investigate questions about the evolution of the number of players adopting each strategy.

A game can be represented by a weighted directed graph. The vertices represent strategies, and on directed edges we indicate the probability that the incoming strategy wins over the outgoing one. For each pair of vertices we indicate only the edge leaving the winning strategy; we do not indicate the probability when it is 1, and we do not put any edge when the probability is 0.5. The probability that a strategy has of winning against itself should be 0.5, since for every pair of challengers using the same strategy there is one winner and one loser; therefore we do not indicate edges entering and leaving the same vertex. We use the letters x, y, z, and so on, denoting the name of a strategy, also to indicate the proportion of players using it.

A sensible idea for creating a discrete model is to make the system evolve by randomly choosing a pair of players who challenge each other. To obtain a continuous model more easily, we instead adopt a discrete model in which all players are paired as challengers at the same time.

Consider the example game reported in the original draft, with strategies x, y, z. Assuming a large even number N of players, it is reasonable (?) to think that, by pairing them randomly as challengers, the probability that in a challenge the strategies used are two distinct strategies s_i and s_j is well approximated by 2s_i s_j.

Consider the challenges involving the pair of strategies xy: these will be 2xyN in number, of which 0.7 · 2xyN are won by the player using x, and 0.3 · 2xyN by the player using y. Therefore there will be an increase of (0.7 - 0.3) · 2xyN players using x, that is, a relative increase of (0.7 - 0.3)2xy. Similarly, for the pair xz, there will be a relative loss of (0 - 1)2xz players using x. Challenges of type xx do not modify the number of players using x.

In general, in a game with set of strategies S, the players using x ∈ S undergo a relative change of

2 · (∑s∈S (2pxs - 1)s) · x,

where pxs is the probability that x wins against s.

Notice that the number 2p - 1 is equal to 1 for p = 1, to 0 for p = 0.5, and to -1 for p = 0.

1.2 Passage to the Continuous Model

Consider a game with set of strategies S. We now want to think of the change in the number of players adopting each strategy as a continuous problem, in which the relative change

2 · (∑s∈S (2pxs - 1)s) · x

indicates the derivative of x, and s∈S s = 1. From now on, we ignore the factor 2 in front of the first summation; the solution will have the same orbits, traveled at half speed.

Let pij be the probability of winning against s_j, and let s = (s_n) be the vector of players adopting each strategy. Consider the matrix A = (aij), with aij = 2pij - 1. Notice that A is a skew-symmetric matrix, with real entries between -1 and 1. Then (As)i = ∑j(2pij - 1)s_j. The evolution of the players therefore follows the differential system

s′ = (As) ⊙ s,

where indicates componentwise multiplication.

We are therefore interested in studying differential systems of the type s′ = (As) ⊙ s, with A a skew-symmetric matrix with real entries between -1 and 1, and initial conditions s_i ≥ 0, ∑ s_i = 1.

This type of system is a special case of generalized Lotka-Volterra systems.

1.3 Formalization

Definition 1.3.1. A game is a finite weighted directed graph in which every pair of vertices is joined by exactly two edges with opposite orientations, with weights p and 1 - p, where p ∈ [0,1], and p = 0.5 if the vertices coincide. The vertices of the graph are also called strategies, and the weight of the edge leaving vertex i and entering j is called the probability of victory of i against j, denoted pij.

Note 1.3.1. Usually, we will draw a game while ignoring edges with weight p ≤ 0.5. We will also call directed edges arrows, and we may say invert an arrow to indicate the operation of swapping the weights in the pair of edges joining two strategies.

Definition 1.3.2. Given a game X with ordered set of strategies S, we call the matrix associated with X the matrix A_X = (aij), aij = 2pij - 1, i,j ∈ S.

Observation 1.3.1. The matrix associated with a game is skew-symmetric.

Proof.

aij = 2pij - 1
= 2(1 - pji) - 1
= 2 - 2pji - 1
= -(2pji - 1)
= -aji.

Observation 1.3.2. The matrix associated with a game has entries between -1 and 1.

Observation 1.3.3. Replacing the nonzero entries of the matrix associated with a game by 1 gives the adjacency matrix of the game, if the arrows with weight 0.5 were removed.

Definition 1.3.3. Given a game X with ordered set of strategies S, and a vector of unknown functions s = (s_i(t)), i ∈ S, we call the system associated with X the system of differential equations

s′ = (A_X s) ⊙ s,

where the symbol indicates componentwise multiplication. Initial conditions for the system are called standard if they are of the type s_i(0) = α_i, 0 ≤ α_i ≤ 1, i∈S α_i = 1. The function x_i(t) solving the system with standard initial conditions indicates the players using strategy i at time t.

2. General Characteristics

2.1 Global Existence and Uniqueness of the Solution

Observe that the system associated with a game is autonomous; therefore, to satisfy the hypotheses of the Picard-Lindelöf theorem and prove local existence and uniqueness of the solution, it is enough to verify local Lipschitzness with respect to the unknown functions.

Notice that the equations are homogeneous polynomials of degree two in the unknown functions, hence differentiable and in particular locally Lipschitz.

We will later prove that the solution is always contained in a simplex in the configuration space, first for standard initial conditions and then in general; by the principle of escape from compact sets, global existence of the solution follows. Alternatively, using the results of the next paragraph, it is a simple observation to verify that the derivatives are bounded.

2.2 Conservation of Players

Theorem 2.2.1. Given a game X with strategies S, the number of players remains constant in time; that is, for the associated system, s∈S s(t) is constant.

Proof.

t ∑ s_i = ∑ ∂s_i
= ∑rows (As ⊙ s)_i
= sTAs = 0,

where the last equality follows because A is skew-symmetric.

This agrees with our intuition, since we are modeling the change of strategy by a fixed number of players.

In the next paragraph we will prove that if one starts with a positive number of players for every strategy, no strategy will ever reach a negative number of players.

From this, remembering that the standard n-simplex is defined as the convex combination of the vectors of the canonical basis in Rn, and represents the region of positive coordinates summing to 1, we obtain:

Observation 2.2.1. The state of a system associated with a game with n strategies in configuration space remains inside the standard n-simplex.

Observation 2.2.2. The coordinates in phase space can be interpreted as barycentric coordinates, so that the motion in the standard simplex can be seen inside a simplex described by the game.

Rescaling initial conditions so that their sum is different from 1, because of the homogeneity of the equations, also rescales the derivatives. Therefore analogous orbits appear, traveled at different speed on a larger simplex onto which they are projected (?).

2.3 Subgames at the Boundary

Definition 2.3.1. Given a game X, a subgame is a game obtained by deleting from X some strategies, and the arrows entering and leaving them.

Suppose we have initial conditions in which there are strategies adopted by no player. Intuition would tell us that no player will ever convert to such strategies, since no one can lose against them, nor can they give players to others; therefore the presence of non-adopted strategies is entirely irrelevant to the movement of players, and so one can pass to the subgame in which they are wholly absent.

In our formalism, this idea corresponds to the following result.

Theorem 2.3.1. Given a system associated with a game with standard initial conditions, if s_i = 0, then s_i(t) = 0 for every t. Moreover, the other strategies evolve as in the subgame in which strategy s_i is absent.

Proof. If s_i = 0, because of the componentwise product ⊙s, also ts_i = 0, and therefore s_i(t) = 0 for every t. Consequently, the i-th summand of every component of Ps, representing the contribution of s_i to s′, will be zero for every t. Ignoring the row and the zero summands, one obtains the system associated with the subgame with s_i removed.

Observation 2.3.1. If multiple initial conditions are zero, one passes to the subgame in which all the corresponding strategies have been removed.

Observation 2.3.2. The portion of the standard simplex where some coordinates are imposed to be zero is the subsimplex given by the convex combination of the canonical vectors associated with the nonzero coordinates. Therefore the system associated with a subgame describes the motion in the corresponding subsimplex generated by the nonzero strategies.

Observation 2.3.3. The evolution of the system is tied only to how it evolves on the one-dimensional edges of the standard simplex, since each of them corresponds to the weight of the arrows in a pair of strategies; knowing the weights for every pair reconstructs the whole game. We will later prove that different weights between two strategies give rise to different behavior on the corresponding edge.

Observation 2.3.4. The system of a game with a single strategy has constant solutions. Therefore, subgames with a single strategy identify stationary points of the system associated with a game, namely the axes in configuration space. With standard initial conditions, this tells us that the vertices of the standard simplex are stationary points of the system.

We finally prove the following.

Theorem 2.3.2. The number of players per strategy in a system associated with a game with standard initial conditions remains positive for every t > 0.

Proof. A positive component of the solution, in order to become negative, must be zero for some t. If it became zero, the system would begin to behave like a subgame in which only the number of players of the other strategies varies: indeed, one would have s′_i = 0, and so it would remain 0. One iterates the reasoning by descending to subcomplexes of lower and lower dimension, that is, by annihilating more and more components of the solution, until all but one are zero. But in that case we would be at a vertex, which is a stationary point.

We can actually improve the result by proving that, starting from the interior of a simplex, the boundaries are never reached. We verify the following.

Lemma 2.3.1. Reversing time in a solution of the system associated with a game X gives a solution of the system associated with the game obtained by reversing all arrows of X.

Proof. Reversing time is equivalent to changing the sign of the derivatives, which is equivalent to changing the sign of the associated skew-symmetric matrix, or equivalently to transposing it; hence it is equivalent to reversing the arrows of the game.

It follows that, given a system with standard initial conditions, every subsimplex is invariant both forward and backward in time. As a consequence, the motion in configuration space with standard initial conditions never reaches the boundaries of the standard subsimplex in which it starts.

2.4 Nontrivial Stationary Points

We are interested in finding stationary points with positive coordinates summing to 1, since these are possible standard initial conditions of the system. We ask in general who the stationary points of the system s′ = Ps ⊙ s are.

Definition 2.4.1. We call trivial the stationary points of the system associated with a game that have at least one zero coordinate.

Consider a configuration s with all entries nonzero, that is, with s lying in the interior of the standard simplex. It will be a stationary point of the system only if it is one for the linear system s′ = Ps: indeed, componentwise multiplication by nonzero numbers cannot otherwise annihilate any nonzero entry. Therefore, an element with positive entries in the kernel of the matrix associated with a game corresponds to a nontrivial stationary point.

Considering instead s with at least one zero entry, the problem reduces to looking for the stationary points of the corresponding subgame.

2.5 Mirror Games

One might ask how a system behaves with initial conditions that are not all positive. The following helps us.

Observation 2.5.1. Changing the sign of the number of players adopting a strategy, thus allowing negative values, is equivalent to changing the sign of the entries in the corresponding row and column of the skew-symmetric matrix of the system; this is equivalent to reversing all arrows entering and leaving that strategy while keeping positive the number of players adopting it.

Consequently, the evolution of the system with not-all-positive initial conditions is analogous to the evolution of the system associated with another game obtained by reversing the appropriate arrows, with nonnegative initial conditions.

The observation suggests an action of (Z/2)^n on the set of games with n ordered strategies, where the i-th canonical generator reverses all arrows leaving or entering the i-th strategy.

Definition 2.5.1. We call mirror games the games belonging to the same orbit of this action.

An orbit of this action contains at most 2^n - 1 different games, since two elements that differ by (1,1,...,1) act in the same way.

For this reason, finding an element of the kernel of a skew-symmetric matrix with nonzero entries means finding a nontrivial stationary point in a mirror game of the one to which the matrix is associated.

As a consequence of the observations made, the behavior of a global system is determined by its behavior, and by the behavior of systems of mirror games, in standard conditions.

3. Small Games

3.1 Conversion

The simplest nontrivial game is composed of two strategies, one of which wins against the other with probability 0.5 < p < 1.

For this game, calling α = 2p - 1 > 0, the associated system is

x′ = αxy
y′ = -αxy.

Using y = 1 - x, one obtains x′(t) = αx(1 - x), whose solution is a logistic curve:

x(t) = 1 / (1 + (1/x0 - 1)e-αt).

The case x(0) = 0 must be treated separately; one obtains x(t) = 0. The function y is found as 1 - x.

The growth of x satisfies the intuition coming from the probabilistic interpretation: it is slow for small x, since this means that few players of y are challenged and converted; it is maximal when there are as many players of x as of y, so that the probability of an encounter between the two strategies is maximal; and it becomes small again when players of y begin to be scarce.

The only stationary points are therefore the vertices of the standard simplex, corresponding to subgames with a single strategy. One is asymptotically stable, the other is unstable.

3.2 Rock-Paper-Scissors

Rock-paper-scissors is the game with three strategies that defeat each other cyclically.

The associated differential system is

[x′, y′, z′]T
= [[0, 1, -1], [-1, 0, 1], [1, -1, 0]] · [x, y, z]T ⊙ [x, y, z]T.

This admits a stationary point internal to the standard simplex, namely (1/3, 1/3, 1/3), which belongs to the kernel of the skew-symmetric matrix. This corresponds to the configuration in which each strategy has the same number of players; from the symmetry of the arrows it is intuitive that one cannot move in any particular direction. The subsimplices correspond to conversions, whose stationary points are the vertices of the standard simplex, which are unstable for the overall system.

We now see how some symmetries can be exploited to prove that the orbits internal to the standard simplex are closed. We use the fact that the behavior of the system behaves well (?) with respect to cyclic permutations of the strategies, and that under time reversal one still has rock-paper-scissors, with the victory relations reversed. Put the system in standard conditions with y > x = z and y ≠ 1, from which x′ > 0, y′ < 0, z′ < 0. The state of the system evolves until y = x in finite time; indeed x′ = x(y - z), but x grows, and y - z cannot fall below the initial value of x - z without having x = y. The same trajectory repeats by cyclically permuting the coordinates, that is, rotating the simplex by one third of a turn.

We now ask what happens with the same starting conditions but with time reversed: the system behaves as if the arrows were reversed, that is, as in a rock-paper-scissors game with x and z exchanged; therefore the same behavior appears, and this lets one conclude that the orbits are closed. The stationary point at the center is therefore stable, but not asymptotically stable.

As a variant of rock-paper-scissors, one can assign to the arrows a probability of victory 0.5 < p_i < 1. The skew-symmetric matrix associated with the system is of the type

[[0, a, -b], [-a, 0, c], [b, -c, 0]],

with a,b,c > 0 under the given hypotheses on p_i. These games admit the nontrivial stationary point

(1 / √(a² + b² + c²)) · [c, b, a]T.

Numerical tests still suggest closed orbits (?).

3.3 Chain of Two Conversions

A chain of two conversions is the game represented in the original draft. Writing y as 1 - x - z, the associated system becomes

x′ = a x(1 - x - z)
z′ = -b z(1 - x - z).

We can find the trajectories as

dz/dx = -(b/a)(z/x).

Separating variables, one easily obtains the first integral E = (b/a)log x + log z, assuming x ≠ 0 and z ≠ 0; from this it follows that the function xb/a · z also remains constant.

This suggests that it is impossible for all players to tend to move toward x, but that one converges to a state in which y = 0 and xb/a · z = 1, depending on the initial conditions. (?)

4. Category of Games

We want to construct a suitable category of games, and investigate the existence and properties of products and coproducts. In order to find a suitable definition for morphisms, it is useful to ask what a quotient of a game should be.

4.1 Quotients

Imagine having a game with strategies S, and a subset V ⊆ S of strategies with the property that, as v ∈ V varies, psv remains invariant for every s ∈ S \ V. Intuitively, this means that from the point of view of the players of a strategy s ∈ S \ V, the strategies in V are indistinguishable, and for the evolution of S it does not matter how the players are distributed inside the strategies in V, but only how many there are in total. This suggests the possibility of simplifying the game in order to study more easily the evolution of the strategies in S \ V, by treating the set V as a single strategy. We call this operation quotient. With the general idea that the right maps between objects are those whose images are quotients, we give the following.

Definition 4.1.1. A map between two games X and Y is a function between their sets of strategies

f : S_X -> S_Y

such that, for every {x_1,...,x_k} = f-1(y) with y ∈ S_Y, one has

pxx1 = pxx2 = ... = pxxk = pf(x)y

for every x ∈ S_X \ f-1{y}. If f is surjective, we call it a quotient map and say that Y is a quotient of X.

From the point of view of the system, calling Q = {x_1,...,x_k} means that in all rows associated with the derivative of a strategy not in Q, one can factor out the coefficient common to {x_1,...,x_k}, and delete the rows of the strategies in Q by replacing them with the one associated with the derivative of x_1 + x_2 + ... + x_k, obtaining the system of the quotient game. (?)

Let us see an example of applying quotients.

Example 4.1.1. Rigged rock-paper-scissors. Call the game represented in the original draft rigged rock-paper-scissors. The associated system is complicated at first sight, but we can simplify it by quotienting the strategies x and y into a single strategy x + y.

This is simply a conversion, which we have already solved. Therefore, although it is difficult to understand how x, y, and z evolve, we can simply study how y and x + z evolve.

Definition 4.1.2. We call GAME the category whose objects are games, and whose morphisms are maps of games.

Observation 4.1.1. Games composed of a single strategy are isomorphic, and constitute the final object of the category. The initial object would want to be the empty game, but it is sensible to ask that there be at least one strategy.

4.2 Coproducts

It turns out that coproducts always exist, and are trivially the disjoint unions of games. Let us therefore see how games with disconnected components behave.

Let X be a disconnected game. Enumerate the strategies so that the strategies of each connected component are placed next to each other. The matrix associated with the game, like its adjacency matrix, is block diagonal, one block for each component. Consequently, in the associated system the equations split apart, and one obtains systems independent of one another, one for each component.

This represents the fact that players of strategies in different connected components do not exchange players, and therefore one can think about the problem by separating it for each connected component. The possibility of drawing between players of strategies in different connected components, however, slows the evolution of players inside a component. This effect is tied to the fact that the players of a subcomponent may not sum to 1, and we have already observed that this translates into slower traversal of orbits analogous to those that would occur if they did sum to 1.

Observation 4.2.1. A connected component can always be quotiented. It is evident (?) that a point is stationary only if it is stationary when restricted to a given connected component, and that the converse also holds; from this it follows (?) that the stationary points are the convex combinations of the stationary points of disconnected components.

It is simple to see that the disjoint union of games satisfies the universal property of the coproduct. (?)

4.3 Products

To construct the product of games, we will try to invert the quotient. While quotienting collapses several strategies to one, we want an operation that replaces a strategy with an entire game, and adds the missing arrows. We will call this operation explosion product. This may fail to be commutative, but in the case in which it is, it will coincide with the product (?).

Definition 4.3.1. Given games X and Y, define the explosion of X on Y, denoted X ⋉ Y, as the game obtained by the following procedure: let S be the set of strategies of Y; consider a copy of X for every strategy of Y, and connect each strategy of copy i with an outgoing edge of weight pij to each strategy of copy j, for every i,j ∈ S.

More formally, we are obtaining a game whose strategies are pairs (x,y), x ∈ S_X, y ∈ S_Y, with

p(x1,y0)(x2,y0) = px1x2,
p(x1,y1)(x2,y2) = p(x0,y1)(x0,y2) = py1y2.

We can similarly (?) define the explosion of X only on a restricted set of strategies of Y, including a single one. Explosion on a single strategy inverts the quotient operation. (?).

We say that the explosion of X on Y commutes if it is isomorphic to the explosion of Y on X.

Note 4.3.1. For convenience, we may represent the explosion X ⋉ Y by drawing a single arrow between the copies of X.

Observe that quotienting by each copy of X gives Y.

An important case in which explosion is commutative is when the two games being multiplied coincide, that is, in the case of squares.

A stationary point of X ⋉ Y must first be a stationary point of the quotient Y, and in addition must be stationary when restricted to each copy of X. In this way there are no flows of players either among different copies of X or inside the copies, so the condition is also sufficient. To obtain a nontrivial stationary point in an explosion, therefore, one first distributes the players among the copies of X so as to have no flows among them while giving players to each, and then distributes these inside each copy so that they constitute a nontrivial stationary point for it.

In the cases in which the explosion X ⋉ Y is commutative, one would have X and Y as quotients, and plausibly the explosion is the product (?).

5. To-Do List

  • Investigate whether games with arrows of weight 1 behave like games with the same oriented arrows but smaller probabilities, and what happens when the weights are changed continuously.
  • Investigate elements of the kernel of skew-symmetric matrices with entries different from 0; discuss spectral theory and the Pfaffian.
  • Investigate combinatorial questions such as how many classes of mirror games exist.
  • Look for useful information in the literature on generalized Lotka-Volterra equations.
  • Improve the chapter on the category of categories, and look for a link with homotopy theory.
  • Look for conserved quantities or Lagrangians and prove the presence of closed orbits in some games.
  • Investigate the relation between the behavior of the system on the boundary of a simplex and inside the simplex.
  • Linearize the system of some games in order to study stationary points.
  • Investigate the relation between the orbits of a system and of its associated linear system.
  • Investigate the relation with Markov chains.
  • Study further small games, looking for further analytic solutions, or introducing numerical analysis.
  • Prove how compositions of arrows are tied to higher derivatives, and give a probabilistic interpretation.
  • See the continuous case as a probabilistic limit of the discrete case.
  • Find real situations well modeled by the theory.
  • Structure the article well, add references.
  • Solve or explain better the questions accompanied by (?).