Editorial note: this text began as original work. In retrospect, it
appears to independently rediscover part of the theory now
recognizable as the replicator dynamics of Taylor and Jonker, or at
least a closely related zero-sum evolutionary-game formalism. The
rendering below preserves the exploratory status of the source text,
including open questions and uncertain passages marked with
(?).
Original date: December 2021.
Original PDF: Teoria del Meta.pdf
Contents
- 1. Introduction
- 1.1 Motivation
- 1.2 Passage to the continuous model
- 1.3 Formalization
- 2. General characteristics
- 2.1 Global existence and uniqueness of the solution
- 2.2 Conservation of players
- 2.3 Subgames at the boundary
- 2.4 Nontrivial stationary points
- 2.5 Mirror games
- 3. Small games
- 3.1 Conversion
- 3.2 Rock-paper-scissors
- 3.3 Chain of two conversions
- 4. Category of Games
- 4.1 Quotients
- 4.2 Coproducts
- 4.3 Products
- 5. To-do list
1. Introduction
1.1 Motivation
Think of the following situation: many players play a game for which several strategies exist, and the probability that a given strategy prevails over another is determined. Draws are not possible. Players challenge one another randomly in pairs, and a defeated player begins to use the strategy against which they lost. Our goal is to model this problem and investigate questions about the evolution of the number of players adopting each strategy.
A game can be represented by a weighted directed graph. The
vertices represent strategies, and on directed edges we indicate the
probability that the incoming strategy wins over the outgoing one.
For each pair of vertices we indicate only the edge leaving the
winning strategy; we do not indicate the probability when it is
1, and we do not put any edge when the probability is
0.5. The probability that a strategy has of winning
against itself should be 0.5, since for every pair of
challengers using the same strategy there is one winner and one
loser; therefore we do not indicate edges entering and leaving the
same vertex. We use the letters x, y,
z, and so on, denoting the name of a strategy, also to
indicate the proportion of players using it.
A sensible idea for creating a discrete model is to make the system evolve by randomly choosing a pair of players who challenge each other. To obtain a continuous model more easily, we instead adopt a discrete model in which all players are paired as challengers at the same time.
Consider the example game reported in the original draft, with
strategies x, y, z. Assuming
a large even number N of players, it is reasonable (?)
to think that, by pairing them randomly as challengers, the
probability that in a challenge the strategies used are two distinct
strategies s_i and s_j is well
approximated by 2s_i s_j.
Consider the challenges involving the pair of strategies
xy: these will be 2xyN in number, of
which 0.7 · 2xyN are won by the player using
x, and 0.3 · 2xyN by the player using
y. Therefore there will be an increase of
(0.7 - 0.3) · 2xyN players using x, that
is, a relative increase of (0.7 - 0.3)2xy. Similarly,
for the pair xz, there will be a relative loss of
(0 - 1)2xz players using x. Challenges
of type xx do not modify the number of players using
x.
In general, in a game with set of strategies S, the
players using x ∈ S undergo a relative change of
2 · (∑s∈S (2pxs - 1)s) · x,
where pxs is the probability that
x wins against s.
Notice that the number 2p - 1 is equal to
1 for p = 1, to 0 for
p = 0.5, and to -1 for
p = 0.
1.2 Passage to the Continuous Model
Consider a game with set of strategies S. We now want
to think of the change in the number of players adopting each
strategy as a continuous problem, in which the relative change
2 · (∑s∈S (2pxs - 1)s) · x
indicates the derivative of x, and
∑s∈S s = 1. From now on, we ignore the
factor 2 in front of the first summation; the solution
will have the same orbits, traveled at half speed.
Let pij be the probability of winning
against s_j, and let s = (s_n) be the
vector of players adopting each strategy. Consider the matrix
A = (aij), with
aij = 2pij - 1. Notice that
A is a skew-symmetric matrix, with real entries between
-1 and 1. Then
(As)i = ∑j(2pij - 1)s_j.
The evolution of the players therefore follows the differential
system
s′ = (As) ⊙ s,
where ⊙ indicates componentwise multiplication.
We are therefore interested in studying differential systems of the
type s′ = (As) ⊙ s, with A a
skew-symmetric matrix with real entries between -1 and
1, and initial conditions
s_i ≥ 0, ∑ s_i = 1.
This type of system is a special case of generalized Lotka-Volterra systems.
1.3 Formalization
Definition 1.3.1. A game is a finite weighted
directed graph in which every pair of vertices is joined by exactly
two edges with opposite orientations, with weights
p and 1 - p, where
p ∈ [0,1], and p = 0.5 if the vertices
coincide. The vertices of the graph are also called strategies, and
the weight of the edge leaving vertex i and entering
j is called the probability of victory of
i against j, denoted
pij.
Note 1.3.1. Usually, we will draw a game while
ignoring edges with weight p ≤ 0.5. We will also call
directed edges arrows, and we may say invert an arrow to indicate
the operation of swapping the weights in the pair of edges joining
two strategies.
Definition 1.3.2. Given a game X with
ordered set of strategies S, we call the matrix
associated with X the matrix
A_X = (aij),
aij = 2pij - 1,
i,j ∈ S.
Observation 1.3.1. The matrix associated with a game is skew-symmetric.
Proof.
aij = 2pij - 1
= 2(1 - pji) - 1
= 2 - 2pji - 1
= -(2pji - 1)
= -aji.
Observation 1.3.2. The matrix associated with a
game has entries between -1 and 1.
Observation 1.3.3. Replacing the nonzero entries of
the matrix associated with a game by 1 gives the
adjacency matrix of the game, if the arrows with weight
0.5 were removed.
Definition 1.3.3. Given a game X with
ordered set of strategies S, and a vector of unknown
functions s = (s_i(t)), i ∈ S, we call the
system associated with X the system of differential
equations
s′ = (A_X s) ⊙ s,
where the symbol ⊙ indicates componentwise
multiplication. Initial conditions for the system are called
standard if they are of the type
s_i(0) = α_i, 0 ≤ α_i ≤ 1,
∑i∈S α_i = 1. The function
x_i(t) solving the system with standard initial
conditions indicates the players using strategy i at
time t.
2. General Characteristics
2.1 Global Existence and Uniqueness of the Solution
Observe that the system associated with a game is autonomous; therefore, to satisfy the hypotheses of the Picard-Lindelöf theorem and prove local existence and uniqueness of the solution, it is enough to verify local Lipschitzness with respect to the unknown functions.
Notice that the equations are homogeneous polynomials of degree two in the unknown functions, hence differentiable and in particular locally Lipschitz.
We will later prove that the solution is always contained in a simplex in the configuration space, first for standard initial conditions and then in general; by the principle of escape from compact sets, global existence of the solution follows. Alternatively, using the results of the next paragraph, it is a simple observation to verify that the derivatives are bounded.
2.2 Conservation of Players
Theorem 2.2.1. Given a game X with
strategies S, the number of players remains constant in
time; that is, for the associated system,
∑s∈S s(t) is constant.
Proof.
∂t ∑ s_i = ∑ ∂s_i
= ∑rows (As ⊙ s)_i
= sTAs = 0,
where the last equality follows because A is
skew-symmetric.
This agrees with our intuition, since we are modeling the change of strategy by a fixed number of players.
In the next paragraph we will prove that if one starts with a positive number of players for every strategy, no strategy will ever reach a negative number of players.
From this, remembering that the standard n-simplex is
defined as the convex combination of the vectors of the canonical
basis in Rn, and represents the region of
positive coordinates summing to 1, we obtain:
Observation 2.2.1. The state of a system associated
with a game with n strategies in configuration space
remains inside the standard n-simplex.
Observation 2.2.2. The coordinates in phase space can be interpreted as barycentric coordinates, so that the motion in the standard simplex can be seen inside a simplex described by the game.
Rescaling initial conditions so that their sum is different from
1, because of the homogeneity of the equations, also
rescales the derivatives. Therefore analogous orbits appear,
traveled at different speed on a larger simplex onto which they are
projected (?).
2.3 Subgames at the Boundary
Definition 2.3.1. Given a game X, a
subgame is a game obtained by deleting from X some
strategies, and the arrows entering and leaving them.
Suppose we have initial conditions in which there are strategies adopted by no player. Intuition would tell us that no player will ever convert to such strategies, since no one can lose against them, nor can they give players to others; therefore the presence of non-adopted strategies is entirely irrelevant to the movement of players, and so one can pass to the subgame in which they are wholly absent.
In our formalism, this idea corresponds to the following result.
Theorem 2.3.1. Given a system associated with a
game with standard initial conditions, if s_i = 0,
then s_i(t) = 0 for every t. Moreover,
the other strategies evolve as in the subgame in which strategy
s_i is absent.
Proof. If s_i = 0, because of the
componentwise product ⊙s, also
∂ts_i = 0, and therefore
s_i(t) = 0 for every t. Consequently, the
i-th summand of every component of Ps,
representing the contribution of s_i to
s′, will be zero for every t. Ignoring
the row and the zero summands, one obtains the system associated
with the subgame with s_i removed.
Observation 2.3.1. If multiple initial conditions are zero, one passes to the subgame in which all the corresponding strategies have been removed.
Observation 2.3.2. The portion of the standard simplex where some coordinates are imposed to be zero is the subsimplex given by the convex combination of the canonical vectors associated with the nonzero coordinates. Therefore the system associated with a subgame describes the motion in the corresponding subsimplex generated by the nonzero strategies.
Observation 2.3.3. The evolution of the system is tied only to how it evolves on the one-dimensional edges of the standard simplex, since each of them corresponds to the weight of the arrows in a pair of strategies; knowing the weights for every pair reconstructs the whole game. We will later prove that different weights between two strategies give rise to different behavior on the corresponding edge.
Observation 2.3.4. The system of a game with a single strategy has constant solutions. Therefore, subgames with a single strategy identify stationary points of the system associated with a game, namely the axes in configuration space. With standard initial conditions, this tells us that the vertices of the standard simplex are stationary points of the system.
We finally prove the following.
Theorem 2.3.2. The number of players per strategy
in a system associated with a game with standard initial conditions
remains positive for every t > 0.
Proof. A positive component of the solution, in
order to become negative, must be zero for some t. If
it became zero, the system would begin to behave like a subgame in
which only the number of players of the other strategies varies:
indeed, one would have s′_i = 0, and so it would remain
0. One iterates the reasoning by descending to
subcomplexes of lower and lower dimension, that is, by annihilating
more and more components of the solution, until all but one are
zero. But in that case we would be at a vertex, which is a
stationary point.
We can actually improve the result by proving that, starting from the interior of a simplex, the boundaries are never reached. We verify the following.
Lemma 2.3.1. Reversing time in a solution of the
system associated with a game X gives a solution of
the system associated with the game obtained by reversing all
arrows of X.
Proof. Reversing time is equivalent to changing the sign of the derivatives, which is equivalent to changing the sign of the associated skew-symmetric matrix, or equivalently to transposing it; hence it is equivalent to reversing the arrows of the game.
It follows that, given a system with standard initial conditions, every subsimplex is invariant both forward and backward in time. As a consequence, the motion in configuration space with standard initial conditions never reaches the boundaries of the standard subsimplex in which it starts.
2.4 Nontrivial Stationary Points
We are interested in finding stationary points with positive
coordinates summing to 1, since these are possible
standard initial conditions of the system. We ask in general who the
stationary points of the system
s′ = Ps ⊙ s are.
Definition 2.4.1. We call trivial the stationary points of the system associated with a game that have at least one zero coordinate.
Consider a configuration s with all entries nonzero,
that is, with s lying in the interior of the standard
simplex. It will be a stationary point of the system only if it is
one for the linear system s′ = Ps: indeed,
componentwise multiplication by nonzero numbers cannot otherwise
annihilate any nonzero entry. Therefore, an element with positive
entries in the kernel of the matrix associated with a game
corresponds to a nontrivial stationary point.
Considering instead s with at least one zero entry, the
problem reduces to looking for the stationary points of the
corresponding subgame.
2.5 Mirror Games
One might ask how a system behaves with initial conditions that are not all positive. The following helps us.
Observation 2.5.1. Changing the sign of the number of players adopting a strategy, thus allowing negative values, is equivalent to changing the sign of the entries in the corresponding row and column of the skew-symmetric matrix of the system; this is equivalent to reversing all arrows entering and leaving that strategy while keeping positive the number of players adopting it.
Consequently, the evolution of the system with not-all-positive initial conditions is analogous to the evolution of the system associated with another game obtained by reversing the appropriate arrows, with nonnegative initial conditions.
The observation suggests an action of (Z/2)^n on the
set of games with n ordered strategies, where the
i-th canonical generator reverses all arrows leaving or
entering the i-th strategy.
Definition 2.5.1. We call mirror games the games belonging to the same orbit of this action.
An orbit of this action contains at most 2^n - 1
different games, since two elements that differ by
(1,1,...,1) act in the same way.
For this reason, finding an element of the kernel of a skew-symmetric matrix with nonzero entries means finding a nontrivial stationary point in a mirror game of the one to which the matrix is associated.
As a consequence of the observations made, the behavior of a global system is determined by its behavior, and by the behavior of systems of mirror games, in standard conditions.
3. Small Games
3.1 Conversion
The simplest nontrivial game is composed of two strategies, one of
which wins against the other with probability
0.5 < p < 1.
For this game, calling α = 2p - 1 > 0, the
associated system is
x′ = αxy
y′ = -αxy.
Using y = 1 - x, one obtains
x′(t) = αx(1 - x), whose solution is a logistic curve:
x(t) = 1 / (1 + (1/x0 - 1)e-αt).
The case x(0) = 0 must be treated separately; one
obtains x(t) = 0. The function y is found
as 1 - x.
The growth of x satisfies the intuition coming from
the probabilistic interpretation: it is slow for small
x, since this means that few players of y
are challenged and converted; it is maximal when there are as many
players of x as of y, so that the
probability of an encounter between the two strategies is maximal;
and it becomes small again when players of y begin to
be scarce.
The only stationary points are therefore the vertices of the standard simplex, corresponding to subgames with a single strategy. One is asymptotically stable, the other is unstable.
3.2 Rock-Paper-Scissors
Rock-paper-scissors is the game with three strategies that defeat each other cyclically.
The associated differential system is
[x′, y′, z′]T
= [[0, 1, -1], [-1, 0, 1], [1, -1, 0]] · [x, y, z]T ⊙ [x, y, z]T.
This admits a stationary point internal to the standard simplex,
namely (1/3, 1/3, 1/3), which belongs to the kernel of
the skew-symmetric matrix. This corresponds to the configuration in
which each strategy has the same number of players; from the
symmetry of the arrows it is intuitive that one cannot move in any
particular direction. The subsimplices correspond to conversions,
whose stationary points are the vertices of the standard simplex,
which are unstable for the overall system.
We now see how some symmetries can be exploited to prove that the
orbits internal to the standard simplex are closed. We use the fact
that the behavior of the system behaves well (?) with respect to
cyclic permutations of the strategies, and that under time reversal
one still has rock-paper-scissors, with the victory relations
reversed. Put the system in standard conditions with
y > x = z and y ≠ 1, from which
x′ > 0, y′ < 0,
z′ < 0. The state of the system evolves until
y = x in finite time; indeed
x′ = x(y - z), but x grows, and
y - z cannot fall below the initial value of
x - z without having x = y. The same
trajectory repeats by cyclically permuting the coordinates, that is,
rotating the simplex by one third of a turn.
We now ask what happens with the same starting conditions but with
time reversed: the system behaves as if the arrows were reversed,
that is, as in a rock-paper-scissors game with x and
z exchanged; therefore the same behavior appears, and
this lets one conclude that the orbits are closed. The stationary
point at the center is therefore stable, but not asymptotically
stable.
As a variant of rock-paper-scissors, one can assign to the arrows a
probability of victory 0.5 < p_i < 1. The
skew-symmetric matrix associated with the system is of the type
[[0, a, -b], [-a, 0, c], [b, -c, 0]],
with a,b,c > 0 under the given hypotheses on
p_i. These games admit the nontrivial stationary point
(1 / √(a² + b² + c²)) · [c, b, a]T.
Numerical tests still suggest closed orbits (?).
3.3 Chain of Two Conversions
A chain of two conversions is the game represented in the original
draft. Writing y as 1 - x - z, the
associated system becomes
x′ = a x(1 - x - z)
z′ = -b z(1 - x - z).
We can find the trajectories as
dz/dx = -(b/a)(z/x).
Separating variables, one easily obtains the first integral
E = (b/a)log x + log z, assuming
x ≠ 0 and z ≠ 0; from this it follows
that the function xb/a · z also remains
constant.
This suggests that it is impossible for all players to tend to move
toward x, but that one converges to a state in which
y = 0 and xb/a · z = 1,
depending on the initial conditions. (?)
4. Category of Games
We want to construct a suitable category of games, and investigate the existence and properties of products and coproducts. In order to find a suitable definition for morphisms, it is useful to ask what a quotient of a game should be.
4.1 Quotients
Imagine having a game with strategies S, and a subset
V ⊆ S of strategies with the property that, as
v ∈ V varies, psv remains
invariant for every s ∈ S \ V. Intuitively, this means
that from the point of view of the players of a strategy
s ∈ S \ V, the strategies in V are
indistinguishable, and for the evolution of S it does
not matter how the players are distributed inside the strategies in
V, but only how many there are in total. This suggests
the possibility of simplifying the game in order to study more
easily the evolution of the strategies in S \ V, by
treating the set V as a single strategy. We call this
operation quotient. With the general idea that the right maps
between objects are those whose images are quotients, we give the
following.
Definition 4.1.1. A map between two games
X and Y is a function between their sets
of strategies
f : S_X -> S_Y
such that, for every
{x_1,...,x_k} = f-1(y) with
y ∈ S_Y, one has
pxx1 = pxx2 = ... = pxxk = pf(x)y
for every x ∈ S_X \ f-1{y}. If
f is surjective, we call it a quotient map and say that
Y is a quotient of X.
From the point of view of the system, calling
Q = {x_1,...,x_k} means that in all rows associated
with the derivative of a strategy not in Q, one can
factor out the coefficient common to {x_1,...,x_k},
and delete the rows of the strategies in Q by
replacing them with the one associated with the derivative of
x_1 + x_2 + ... + x_k, obtaining the system of the
quotient game. (?)
Let us see an example of applying quotients.
Example 4.1.1. Rigged rock-paper-scissors. Call the
game represented in the original draft rigged rock-paper-scissors.
The associated system is complicated at first sight, but we can
simplify it by quotienting the strategies x and
y into a single strategy x + y.
This is simply a conversion, which we have already solved.
Therefore, although it is difficult to understand how
x, y, and z evolve, we can
simply study how y and x + z evolve.
Definition 4.1.2. We call GAME the
category whose objects are games, and whose morphisms are maps of
games.
Observation 4.1.1. Games composed of a single strategy are isomorphic, and constitute the final object of the category. The initial object would want to be the empty game, but it is sensible to ask that there be at least one strategy.
4.2 Coproducts
It turns out that coproducts always exist, and are trivially the disjoint unions of games. Let us therefore see how games with disconnected components behave.
Let X be a disconnected game. Enumerate the strategies
so that the strategies of each connected component are placed next
to each other. The matrix associated with the game, like its
adjacency matrix, is block diagonal, one block for each component.
Consequently, in the associated system the equations split apart,
and one obtains systems independent of one another, one for each
component.
This represents the fact that players of strategies in different
connected components do not exchange players, and therefore one can
think about the problem by separating it for each connected
component. The possibility of drawing between players of strategies
in different connected components, however, slows the evolution of
players inside a component. This effect is tied to the fact that the
players of a subcomponent may not sum to 1, and we
have already observed that this translates into slower traversal of
orbits analogous to those that would occur if they did sum to
1.
Observation 4.2.1. A connected component can always be quotiented. It is evident (?) that a point is stationary only if it is stationary when restricted to a given connected component, and that the converse also holds; from this it follows (?) that the stationary points are the convex combinations of the stationary points of disconnected components.
It is simple to see that the disjoint union of games satisfies the universal property of the coproduct. (?)
4.3 Products
To construct the product of games, we will try to invert the quotient. While quotienting collapses several strategies to one, we want an operation that replaces a strategy with an entire game, and adds the missing arrows. We will call this operation explosion product. This may fail to be commutative, but in the case in which it is, it will coincide with the product (?).
Definition 4.3.1. Given games X and
Y, define the explosion of X on
Y, denoted X ⋉ Y, as the game obtained by
the following procedure: let S be the set of strategies
of Y; consider a copy of X for every
strategy of Y, and connect each strategy of copy
i with an outgoing edge of weight
pij to each strategy of copy j,
for every i,j ∈ S.
More formally, we are obtaining a game whose strategies are pairs
(x,y), x ∈ S_X, y ∈ S_Y,
with
p(x1,y0)(x2,y0) = px1x2,
p(x1,y1)(x2,y2) = p(x0,y1)(x0,y2) = py1y2.
We can similarly (?) define the explosion of X only on
a restricted set of strategies of Y, including a single
one. Explosion on a single strategy inverts the quotient operation.
(?).
We say that the explosion of X on Y
commutes if it is isomorphic to the explosion of Y on
X.
Note 4.3.1. For convenience, we may represent the
explosion X ⋉ Y by drawing a single arrow between the
copies of X.
Observe that quotienting by each copy of X gives
Y.
An important case in which explosion is commutative is when the two games being multiplied coincide, that is, in the case of squares.
A stationary point of X ⋉ Y must first be a stationary
point of the quotient Y, and in addition must be
stationary when restricted to each copy of X. In this
way there are no flows of players either among different copies of
X or inside the copies, so the condition is also
sufficient. To obtain a nontrivial stationary point in an explosion,
therefore, one first distributes the players among the copies of
X so as to have no flows among them while giving
players to each, and then distributes these inside each copy so that
they constitute a nontrivial stationary point for it.
In the cases in which the explosion X ⋉ Y is
commutative, one would have X and Y as
quotients, and plausibly the explosion is the product (?).
5. To-Do List
-
Investigate whether games with arrows of weight
1behave like games with the same oriented arrows but smaller probabilities, and what happens when the weights are changed continuously. -
Investigate elements of the kernel of skew-symmetric matrices
with entries different from
0; discuss spectral theory and the Pfaffian. - Investigate combinatorial questions such as how many classes of mirror games exist.
- Look for useful information in the literature on generalized Lotka-Volterra equations.
- Improve the chapter on the category of categories, and look for a link with homotopy theory.
- Look for conserved quantities or Lagrangians and prove the presence of closed orbits in some games.
- Investigate the relation between the behavior of the system on the boundary of a simplex and inside the simplex.
- Linearize the system of some games in order to study stationary points.
- Investigate the relation between the orbits of a system and of its associated linear system.
- Investigate the relation with Markov chains.
- Study further small games, looking for further analytic solutions, or introducing numerical analysis.
- Prove how compositions of arrows are tied to higher derivatives, and give a probabilistic interpretation.
- See the continuous case as a probabilistic limit of the discrete case.
- Find real situations well modeled by the theory.
- Structure the article well, add references.
-
Solve or explain better the questions accompanied by
(?).