Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies | Dynamic Games and Applications Skip to main content
Springer Nature Link
Log in

Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies

  • Published:
Dynamic Games and Applications Aims and scope Submit manuscript

Abstract

This work deals with a class of discrete-time zero-sum Markov games under a discounted optimality criterion with random state-actions-dependent discount factors of the form \(\tilde{\alpha }(x_{n},a_{n},b_{n},\xi _{n+1})\), where \(x_{n}, a_{n}, b_{n}\), and \(\xi _{n+1}\) are the state, the actions of players, and a random disturbance at time n, respectively, taking values in Borel spaces. Assuming possibly unbounded payoff, we prove the existence of a value of the game as well as a stationary pair of optimal strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ash RB, Doléans-Dade C (2000) Probability and measure theory, 2nd edn. Academic Press, London

    MATH  Google Scholar 

  2. Bäuerle N, Rieder U (2011) Markov decision processes with applications to finance. Springer, Berlin

    Book  MATH  Google Scholar 

  3. Carmon Y, Shwartz A (2009) Markov decision processes with exponentially representable discounting. Oper Res Lett 37(1):51–55

    Article  MathSciNet  MATH  Google Scholar 

  4. Cruz-Suárez H, Ilhuicatzi-Roldán R, Montes-de Oca R (2014) Markov decision processes on Borel spaces with total cost and random horizon. J Optim Theory Appl 162(1):329–346

    Article  MathSciNet  MATH  Google Scholar 

  5. Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2(2):197–214

    Article  MathSciNet  MATH  Google Scholar 

  6. Dynkin EB, Yushkevich AA (1979) Controlled Markov processes. Springer, Berlin

    Book  Google Scholar 

  7. Engwerda J (2005) LQ dynamic optimization and differential games. Wiley, New York

    Google Scholar 

  8. Feinberg EA, Shwartz A (1999) Constrained dynamic programming with two discount factors: applications and an algorithm. IEEE Trans Autom Control 44(3):628–631

    Article  MathSciNet  MATH  Google Scholar 

  9. Filar J, Vrieze K (2012) Competitive Markov decision processes. Springer, Berlin

    MATH  Google Scholar 

  10. González-Hernández J, López-Martínez RR, Minjárez-Sosa JA (2008) Adaptive policies for stochastic systems under a randomized discounted cost criterion. Bol Soc Mat Mexicana 3(14):149–163

    MathSciNet  MATH  Google Scholar 

  11. González-Hernández J, López-Martínez RR, Minjárez-Sosa JA (2009) Approximation, estimation and control of stochastic systems under a randomized discounted cost criterion. Kybernetika 45(5):737–754

    MathSciNet  MATH  Google Scholar 

  12. González-Hernández J, López-Martínez RR, Minjárez-Sosa JA, Gabriel-Arguelles JR (2013) Constrained Markov control processes with randomized discounted cost criteria: occupation measures and extremal points. Risk Decis Anal 4(3):163–176

    Article  MATH  Google Scholar 

  13. He W, Sun Y (2017) Stationary Markov perfect equilibria in discounted stochastic games. J Econ Theory 169:35–61

    Article  MathSciNet  MATH  Google Scholar 

  14. Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria, vol 30. Springer, Berlin

    Book  MATH  Google Scholar 

  15. Huang Y, Guo X (2012) Constrained optimality for first passage criteria in semi-Markov decision processes. In: Hernández-Hernández D, Minjárez-Sosa JA (eds) Optimization, control, and applications of stochastic systems, systems & control: foundations & applications. Birkhauser, Boston, pp 181–202 chap. 11

    Chapter  Google Scholar 

  16. Huang Y, Wei Q, Guo X (2013) Constrained Markov decision processes with first passage criteria. Ann Oper Res 206(1):197–219

    Article  MathSciNet  MATH  Google Scholar 

  17. Jaśkiewicz A, Nowak AS (2006) Approximation of noncooperative semi-Markov games. J Optim Theory Appl 131(1):115–134

    Article  MathSciNet  MATH  Google Scholar 

  18. Jaśkiewicz A, Nowak AS (2006) Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J Control Optim 45(3):773–789

    Article  MathSciNet  MATH  Google Scholar 

  19. Krausz A, Rieder U (1997) Markov games with incomplete information. Math Methods Oper Res 46(2):263–279

    Article  MathSciNet  MATH  Google Scholar 

  20. Luque-Vásquez F (2002) Zero-sum semi-Markov games in Borel spaces: discounted and average payoff. Bol Soc Mat Mexicana 8:227–241

    MathSciNet  MATH  Google Scholar 

  21. Maitra A, Parthasarathy T (1970) On stochastic games. J Optim Theory Appl 5(4):289–300

    Article  MATH  MathSciNet  Google Scholar 

  22. Maschler M, Solan E, Zamir S (2013) Game theory. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  23. Minjárez-Sosa JA (2015) Markov control models with unknown random state-action-dependent discount factors. Top 23(3):743–772

    Article  MathSciNet  MATH  Google Scholar 

  24. Minjárez-Sosa JA, Luque-Vásquez F (2008) Two person zero-sum semi-Markov games with unknown holding times distribution on one side: a discounted payoff criterion. Appl Math Optim 57(3):289–305

    Article  MathSciNet  MATH  Google Scholar 

  25. Minjárez-Sosa JA, Vega-Amaya O (2009) Asymptotically optimal strategies for adaptive zero-sum discounted Markov games. SIAM J Control Optim 48(3):1405–1421

    Article  MathSciNet  MATH  Google Scholar 

  26. Minjárez-Sosa JA, Vega-Amaya O (2013) Optimal strategies for adaptive zero-sum average Markov games. J Math Anal Appl 402(1):44–56

    Article  MathSciNet  MATH  Google Scholar 

  27. Neyman A, Sorin S (2003) Stochastic games and applications, vol 570. Kluwer, Dordrecht

    Book  MATH  Google Scholar 

  28. Nowak AS (1984) On zero-sum stochastic games with general state space. I. Prob Math Stat 4(1):13–32

    MathSciNet  MATH  Google Scholar 

  29. Nowak AS (1985) Measurable selection theorems for minimax stochastic optimization problems. SIAM J Control Optim 23(3):466–476

    Article  MathSciNet  MATH  Google Scholar 

  30. Nowak AS (1987) Nonrandomized strategy equilibria in noncooperative stochastic games with additive transition and reward structure. J Optim Theory Appl 52(3):429–441

    Article  MathSciNet  MATH  Google Scholar 

  31. Nowak AS, Szajowski K (1999) Nonzero-sum stochastic games. In: Stochastic and differential games. Annals of the international society of dynamic games, vol 4, chap 7. Springer, Berlin, pp 297–342

  32. Osborne MJ, Rubinstein A (1994) A course in game theory. MIT Press, Cambridge

    MATH  Google Scholar 

  33. Rieder U (1991) Non-cooperative dynamic games with general utility functions. In: Raghavan TES, Ferguson TS, Parthasarathy T, Vrieze OJ (eds) Stochastic games and related topics, theory and decision library, vol 7. Springer, Berlin, pp 161–174

    Chapter  Google Scholar 

  34. Schäl M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Probab Theory Rel Fields 32(3):179–196

    MathSciNet  MATH  Google Scholar 

  35. Shapley LS (1953) Stochastic games. Proc Natl Acad Sci USA 39(10):1095–1100

    Article  MathSciNet  MATH  Google Scholar 

  36. Shimkin N, Shwartz A (1995) Asymptotically efficient adaptive strategies in repeated games. Part I: certainty equivalence strategies. Math Oper Res 20(3):743–767

    Article  MathSciNet  MATH  Google Scholar 

  37. Shimkin N, Shwartz A (1996) Asymptotically efficient adaptive strategies in repeated games. Part II: asymptotic optimality. Math Oper Res 21(2):487–512

    Article  MathSciNet  MATH  Google Scholar 

  38. Wei Q, Guo X (2011) Markov decision processes with state-dependent discount factors and unbounded rewards/costs. Oper Res Lett 39(5):369–374

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. CONACYT–Universidad de Sonora, Rosales s/n, 83000, Hermosillo, Sonora, Mexico

    David González-Sánchez

  2. Departamento de Matemáticas, Universidad de Sonora, Rosales s/n, 83000, Hermosillo, Sonora, Mexico

    Fernando Luque-Vásquez & J. Adolfo Minjárez-Sosa

Authors
  1. David González-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fernando Luque-Vásquez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Adolfo Minjárez-Sosa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Adolfo Minjárez-Sosa.

Additional information

Work supported by Consejo Nacional de Ciencia y Tecnología (CONACYT) under Grant CB2015/254306.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

González-Sánchez, D., Luque-Vásquez, F. & Minjárez-Sosa, J.A. Zero-Sum Markov Games with Random State-Actions-Dependent Discount Factors: Existence of Optimal Strategies. Dyn Games Appl 9, 103–121 (2019). https://doi.org/10.1007/s13235-018-0248-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13235-018-0248-8

Keywords

Mathematics Subject Classification

Search

Navigation