Time-Consistent Equilibria in Common Access Resource Games with Asymmetric Players Under Partial Cooperation

Abstract

Given a differential game, if agents have different time preference rates, cooperative (Pareto optimum) solutions obtained by applying Pontryagin’s maximum principle become time inconsistent. We derive a set of dynamic programming equations in continuous time whose solutions are time-consistent equilibria for problems in which agents differ in their utility functions and also in their time preference rates. The solution assumes cooperation between agents at every time. Since coalitions at different times have different time preferences, equilibrium policies are calculated by looking for Markov (subgame perfect) equilibria in a (noncooperative) sequential game. The results are applied to the study of a cake-eating problem describing the management of a common property exhaustible natural resource. The extension of the results to a simple common property renewable natural resource model in infinite horizon is also discussed.

Appendix

Solution of the DPE (16) We guess for a value function of the form W(x,y,t) = A(t)ln (x) + B(t)y + C(t). If this choice proves to be consistent, the extraction rates for both agents are given by c ₁(t) = 1/W _x  = x/A(t) and c ₂(t) = W _y / W _x  = B(t)x/A(t). In order to solve Eq. 16, we calculate the expression for K(x,t). To do that, we substitute our “guessed” controls in Eq. 15. Hence, $x(s)=x_t \exp\left(\Lambda_t(s)\right)$, with $\Lambda_t(s)=-\int_t^s \frac{1+B(\tau)}{A(\tau)} d \tau$. Therefore, $K(x,y,t)= (r_1-r_2)\int_t^T e^{-r_1(s-t)} \ln \big( \frac{x_t e^{\Lambda_t(s)}}{A(s)} \big) ds = \frac{r_1-r_2}{r_1}\big( 1-e^{-r_1(T-t)}\big) \ln(x_t) + (r_1-r_2)\int_t^T e^{-r_1(s-t)} \ln \big( \frac{e^{\Lambda_t(s)}}{A(s)} \big) ds$. By substituting in Eq. 16 and simplifying, we obtain

$$\begin{array}{lll} r_2 \left[A(t)\ln(x)\!+\!B(t)y\!+\!C(t) \right]\!-\!\left[\! A'(t)\ln(x)\!+\!B'(t)y\!+\!C'(t)\! \right] \\ \quad+ \frac{r_1-r_2}{r_1}\left( 1-e^{-r_1(T-t)}\right) \ln(x) \\ \quad + (r_1-r_2)\int_t^T e^{-r_1(s-t)} \ln \left( \frac{e^{\Lambda_t(s)}}{A(s)} \right) ds \\\qquad=\ln (x)- \ln \left(A(t)\right) -1- B(t)\\\qquad+B(t)\left(r_2y+\ln(x)+\ln\left( \frac{B(t)}{A(t)} \right) \right). \end{array}$$

Since the above equation must be satisfied for every x and y, then

$$\label{aa} r_2 A(t)\!-\!A'(t) \!+\! \frac{r_1\!-\!r_2}{r_1}\!\left( 1\!-\!e^{-r_1(T-t)}\right) \!=\! 1\!+\!B(t)\; ,~B^\prime(t)\!=\!0. $$

(38)

Using the terminal condition B(T) = 1, we obtain B(t) = 1 and c ₁(t) = c ₂(t) = x/A(t), for every t ∈ [0,T]. With respect to A(t), note that if $A(t)=\sum_{i=1}^2 \frac{1-e^{-r_i(T-t)}}{r_i}$, which describes the solution for a naive coalition (see (14)), then (38) is satisfied and, in addition, the solution to the state equation $\dot{x}(t)=-2 x(t)/A(t)$ verifies the terminal condition lim _t→T x(t) = 0. Therefore, the naive solution verifies the DPE (16).

Derivation of the Dynamic Programming Algorithm in Discrete Time (21) In the final period, we define $V_n^*=0$, as usual. For j = n − 1, the optimal value for (19) will be given by the solution to the problem

$$\begin{array}{lll} V_{(n-1)}^*(x_{(n-1)},(n-1)\epsilon) \\ \quad = \max\limits_{\{c_{(n-1)}\}} \left\{ \sum\limits_{m=1}^N \lambda_m U^m(x_{(n-1)}, c_{(n-1)},(n-1)\epsilon)\epsilon \right\}\; , \end{array}$$

with x _n  = x _{(n − 1)} + f(x _{(n − 1)}, u _{(n − 1)},(n − 1)ε)ε. If $c^*_{(n-1)}(x_{(n-1)},(n-1)\epsilon)$ is the maximizer of the right-hand term of the above equation, let us denote

$$\begin{array}{lll}{\bar U}_{(n-1)}^m (x_{(n-1)},(n-1)\epsilon) \\ \quad=U^m(x_{(n-1)}, c_{(n-1)}^*(x_{(n-1)},(n-1)\epsilon),(n-1)\epsilon)\; . \end{array}$$

In general, for j = 1,...,n − 1, the value $V_j^*(x_j,j\epsilon)$ in (19) can be written as

$$\begin{array}{rll} V_j^* &=& \max\limits_{\{c_{j}\}} \left\{ \sum\limits_{m=1}^N \lambda_m U^m(x_j, c_j,j\epsilon)\epsilon \right.\\ &&\left. + \sum\limits_{k=1}^{n-j-1} \!\sum\limits_{m=1}^N \lambda_m e^{-r_m k \epsilon} {\bar U}_{(j+k)}^m (x_{(j+k)},(j\!+\!k)\epsilon)\epsilon \right\}.\notag\end{array}$$

(39)

Since

$$\begin{array}{lll}V_{(j+1)}^*(x_{(j+1)},(j+1)\epsilon) \\ \quad= \sum\limits_{i=0}^{n-j-2} \sum\limits_{m=1}^N \lambda_m e^{-r_m i \epsilon} {\bar U}_{(j+i+1)}^m (x_{(j+i+1)},(j+i+1)\epsilon)\epsilon , \end{array}$$

then we can write $V_{(j+1)}^*(x_{(j+1)},(j+1)\epsilon) - \sum_{i=0}^{n-j-2}$ $\sum_{m=1}^N \,\,\lambda_m e^{-r_m i \epsilon} \,\,{\bar U}_{(j+i+1)}^m \,\,(x_{(j+i+1)},\,\,(j\,+\,i\,+\,1)\epsilon)\epsilon \,=\,0$. Adding the former expression to (39), we obtain (21).

Derivation of the DPE in Continuous Time (22) Let W ^m(x,t) be a continuously differentiable function representing the value function of player m in the t coalition, and let $W(x,t)=\sum_{m=1}^N W^m(x,t)$ be the value function for the t coalition, with initial condition x(t) = x. Since s = jε, for sufficiently small ε, x(s + ε) − x(s) ≅ f(x(s),c(s),s)ε, W(x(t),t) ≅ V _j (x _j , jε), and $W(x(t+\epsilon),t+\epsilon)=W(x(t),t)+\nabla_xW(x(t),t)\cdot$ $f(x(t),c(t),t)\epsilon+\nabla_t W(x(t),t)\epsilon+o(\epsilon)$. Substituting in (21), we obtain

$$\begin{array}{lll} W(x(t),t)\\ \quad=\max\limits_{\{c_{t} \}} \!\left\{ \sum\limits_{m=1}^N \lambda_m U^m(x(t),c(t),t)\epsilon+W(x(t),t) \right. \\\quad\quad\left. +\nabla_{x} W(x(t),t) \cdot\! f(x(t),c(t),t)\epsilon + \nabla_{t}W(x(t),t)\epsilon \right.\\\quad\quad\left.- \sum_{m=1}^N \left(1-e^{r_m \epsilon}\right) W^m(x(t),t) +o(\epsilon) \right\}, \end{array}$$

(40)

where

$$ \label{kdis_N} W^m(x(t),t)=\sum_{k=1}^{n-j-1} \lambda_m e^{-r_m k \epsilon} {\bar U}_{(j+k)}^m(x_{(j+k)},(j+k)\epsilon)\epsilon . $$

(41)

Finally, by dividing (40) and (41) by ε and taking the limit ε→0, we obtain Eq. 22.

Derivation of the DPE (32) Without lack of generality, for simplicity, we take λ ₁ = ⋯ = λ _N  = 1. If c ^*(s) = φ(s,x(s)) is the equilibrium rule, then the value function is

$$\label{valuefunction} W(x,t)= \sum\limits_{m=1}^N \int_t^\tau e^{-r_m(s-t)} U^m(x(s),\phi(x(s),s),s)\, ds $$

(42)

where $\dot{x}(s)=f(x(s),\phi(x(s),s),s)$, x(t) = x _t . We assume that if τ = ∞, along the equilibrium rule, the value function (42) is finite (i.e., the integral converges). This is guaranteed if we restrict our attention to strategies φ(x,s) of class C ¹ such that, when t→ ∞, the state variables converge to a stationary state.

Next, for ε > 0, let us consider the variations

$$ c_\epsilon(s)=\left\{ \begin{array}{ccc} v(s) & \hbox{if} & s\in[t,t+\epsilon]\; , \\[4pt] \phi(x,s) & \hbox{if} & s>t+\epsilon\; . \end{array}\right. $$

If the t agent can precommit her behavior during the period [t,t + ε], the value function for the perturbed control path c _ε is given by

$$\begin{array}{lll}W_\epsilon(x,t)\\ \quad= \max\limits_{\{ v(s),\; s\in[t,t+\epsilon]\}} \left\{ \sum\limits_{m=1}^N \int_t^{t+\epsilon} e^{-r_m (s-t)}U^m\left(x(s),v(s),s\right)ds\right.\\ \quad\quad\left. + \sum\limits_{m=1}^N\int_{t+\epsilon}^\tau e^{-r_m(s-t)} U^m(x(s),\phi(x(s),s),s)\, ds\right\}\; . \end{array}$$

Let us assume that W _ε is differentiable in ε in a neighborhood of ε = 0. Then, c ^*(s) = φ(s,x(s)) is called an equilibrium rule if

$$\label{def1} \lim_{\epsilon\to 0^+}\frac{W(x,t)-W_{\epsilon}(x,t)}{\epsilon} \geq 0\; . $$

(43)

The above definition can be interpreted as follows: for sufficiently small ε, the maximum of W _ε in the limit when ε = 0 is precisely W(x,t). In order to prove that c ^*(t) = φ(x,t) solving the right-hand term in Eq. 32 is an equilibrium rule, we have to check Eq. 43. We do it in several steps:

If $\bar{x}(s)$ denotes the state trajectory corresponding to the decision rule c _ε (s), then

$$\begin{array}{lll} &&{\kern-6pt}W(x,t)-W_\epsilon(x,t) \\[6pt] &&= \sum\limits_{m=1}^N \int_t^{t+\epsilon} e^{-r_m(s-t)} \left[ U^m(x(s),\phi(x(s),s),s) \right.\\[6pt] &&\left.{\kern6pt}- U^m(\bar{x}(s),v(s),s)\right]\, ds\\[6pt] &&{\kern6pt}+\sum\limits_{m=1}^N \int_{t+\epsilon}^\tau e^{-r_m(s-t)} \left[ U^m(x(s),\phi(x(s),s),s) \right.\\[6pt] &&{\kern6pt}\left.- U^m(\bar{x}(s),\phi(\bar{x}(s),s),s)\right]\, ds\; . \end{array}$$

Note that

$$\begin{array}{lll}\sum\limits_{m=1}^N \int_{t+\epsilon}^\tau e^{-r_m(s-t)} U^m(x(s),\phi(x(s),s),s) \, ds\ \\\quad= W(x(t+\epsilon),t+\epsilon) -\sum_{m=1}^N \int_{t+\epsilon}^\tau \left[ e^{-r_m(s-t-\epsilon)} \right.\\\quad\quad\left.- e^{-r_m(s-t)}\right] U^m(x(s),\phi(x(s),s),s) \, ds\; . \end{array}$$

In a similar way,

$$\begin{array}{lll}\sum\limits_{m=1}^N \int_{t+\epsilon}^\tau e^{-r_m(s-t)} U^m({\bar x}(s),\phi({\bar x}(s),s),s) \, ds\ \\\quad= W({\bar x}(t+\epsilon),t+\epsilon)-\sum_{m=1}^N \int_{t+\epsilon}^\tau\left[ e^{-r_m(s-t-\epsilon)} \right.\\\quad\quad\left.- e^{-r_m(s-t)}\right] U^m({\bar x}(s),\phi({\bar x}(s),s),s) \, ds \; . \end{array}$$

Therefore,

$$\begin{array}{lll}\lim\limits_{\epsilon\to 0^+} \frac{W(x,t)-W_\epsilon(x,t)}{\epsilon} = \lim\limits_{\epsilon\to 0^+}\frac{\sum_{m=1}^N\int_t^{t+\epsilon} e^{-r_m(s-t)}\left[ U^m(x(s),\phi(x(s),s),s) - U^m(\bar{x}(s),v(s),s)\right] ds}{\epsilon}\\ \quad+\lim\limits_{\epsilon\to 0^+}\frac{1}{\epsilon}\sum\limits_{m=1}^N\left[ \int_{t+\epsilon}^\tau\left[ e^{-r_m(s-t)} - e^{-r_m(s-t-\epsilon)}\right]\left[ U^m(x(s),\phi(x(s),s),s) - U^m(\bar{x}(s),\phi(\bar{x}(s),s),s) \right] ds\right]\\ \quad+\lim\limits_{\epsilon\to 0^+} \frac{W(x(t+\epsilon),t+\epsilon) - W(\bar{x}(t+\epsilon),t+\epsilon)}{\epsilon} = \sum\limits_{m=1}^N \left[ U^m(x(t),\phi(x(t),t),t) - U^m(x(t),v(t),t)\right]\\ \quad+ 0 + \lim\limits_{\epsilon\to 0^+} \frac{W(x(t+\epsilon),t+\epsilon) - W(x(t),t)}{\epsilon} - \lim\limits_{\epsilon\to 0^+} \frac{W(\bar{x}(t+\epsilon),t+\epsilon)- W(x(t),t)}{\epsilon}\\ = \sum\limits_{m=1}^N \left[ U^m(x(t),\phi(x(t),t),t) - U^m(x(t),v(t),t)\right] + \left[\frac{\partial W(x,t)}{\partial t} + \nabla_x W(x,t)\cdot f(x,\phi(x,t),t)\right]\\ \quad- \left[ \frac{\partial W(x,t)}{\partial t} + \nabla_x W(x,t)\cdot f(x,v(t),t)\right] = \sum\limits_{m=1}^N\left[ U^m(x,\phi(x,t),t) + \nabla_x W^m(x,t)\cdot f(x,\phi(x,t),t)\right]\\ \quad- \sum\limits_{m=1}^N\left[ U^m(x,v(t),t) + \nabla_x W^m(x,t)\cdot f(x,v(t),t)\right]\geq 0\; , \end{array}$$

since c ^* = φ(x,t) is the maximizer of the right-hand term in Eq. 32.