Abstract
In this paper, we consider two-person nonzero-sum discrete-time stochastic games under the probability criterion. Taking \(\lambda \) for player 1 and \(\mu \) for player 2 as their profit goal, the two players are concerned with the probabilities that the rewards they earn before the first passage to some target state set are more than \(\lambda \) and \(\mu \), respectively. We firstly give a characterization of the probabilities, and then, under a mild condition, we show that the optimal value function for each player is the unique solution to the corresponding optimality equation by an iterative approximation, and then establish the existence of Nash equilibria. Finally, a queueing system is provided to show the application of our main result.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Altman E (1994) Flow control using the theory of zero-sum Markov games. IEEE Trans Autom Control 39:814–818
Bouakiz M, Kebir Y (1995) Target-level criterion in Markov decision processes. J Optim Theory Appl 86:1–15
Boda K, Filar JA, Lin YL, Spanjers L (2004) Stochastic target hitting time and the problem of early retirement. IEEE Trans Autom Control 49:409–419
Cao XR (2003) Semi-Markov decision problems and performance sensitivity analysis. IEEE Trans Autom Control 48:758–769
Fan K (1952) Fixed-point and minimax theorems in locally convex topological linear spaces. Proc Nat Acad Sci 38:121–126
Guo XP, Hernández-Lerma O (2003) Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates. J Appl Probab 40:327–345
Guo XP, Hernández-Lerma O (2005) Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs. J Appl Probab 42:303–320
Guo XP, Vykertas M, Zhang Y (2013) Absorbing continuous-time Markov decision processes with total cost criteria. Adv Appl Probab 45:490–519
Ghosh MK, Kumar KS, Pal C (2016) Zero-sum risk-sensitive stochastic games for continuous time Markov chains. Stoch Anal Appl 34:835–851
Huang YH, Guo XP, Song XY (2011) Performance analysis for controlled semi-Markov systems with application to maintenance. J Optim Theory Appl 150:395–415
Huo HF, Zou XL, Guo XP (2017) The risk probability criterion for discounted continuous-time Markov decision processes. Discrete Event Dyn Syst 27:675–699
Hernández-Lerma O, Lasserre JB (2001) Zero-sum stochastic games in Borel spaces: average payoff criterion. SIAM J Control Optim 39:1520–1539
Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes. Springer, New York
Hernández-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer, New York
Huang XX, Guo XP, Peng JP (2017) A probability criterion for zero-sum stochastic games. J Dyn Games 4:369–383
Kira A, Ueno T, Fujita T (2012) Threshold probability of non-terminal type in finite horizon Markov decision processes. J Math Anal Appl 386:461–472
Liu QL, Huang XX (2017) Discrete-time zero-sum Markov games with first passage criteria. Optimization 66:571–587
Shapley LS (1953) Stochastic games. Proc Nat Acad Sci 39:1095–1100
Sennott LI (1994) Zero-sum stochastic games with unbounded costs: discounted and average cost cases. Zeitschrift für Oper Res 39:209–225
Sakaguchi M, Ohtsubo Y (2013) Markov decision processes associated with two threshold probability criteria. J Control Appl 11:548–557
Sakaguchi M, Ohtsubo Y (2010) Optimal threshold probability and expectation in semi-Markov decision processes. Appl Math Comput 216:2947–2958
Wu CB, Lin YL (1999) Minimizing risk models in Markov decision processes with policies depending on target values. J Math Anal Appl 231:47–67
White DJ (1993) Minimizing a threshold probability in discounted Markov decision processes. J Math Anal Appl 173:634–646
Wei QD, Chen X (2016) Stochastic games for continuous-time jump processes under finite-horizon payoff criterion. Appl Math Optim 74:273–301
Zhang WZ, Wang BF, Chen DW (2018) Continuous-time constrained stochastic games with average criteria. Oper Res Lett. 46:109–115
Acknowledgements
The authors are grateful to the AE and the referees for their numerous valuable comments and suggestions that have improved this paper. This work was partially supported by the NSFC (Grant No. 11801073), the Natural Science Foundation of Guangdong Province (Grant No. 2017A030310598) and the Ministry of Education, Humanities and Social Sciences project (Grant No. 17JYJAZH011).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, X., Guo, X. Nonzero-Sum Stochastic Games with Probability Criteria. Dyn Games Appl 10, 509–527 (2020). https://doi.org/10.1007/s13235-019-00317-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13235-019-00317-z