Abstract
We adopt the Markov chain framework to model bilateral negotiations among agents in dynamic environments and use Bayesian learning to enable them to learn an optimal strategy in incomplete information settings. Specifically, an agent learns the optimal strategy to play against an opponent whose strategy varies with time, assuming no prior information about its negotiation parameters. In so doing, we present a new framework for adaptive negotiation in such non-stationary environments and develop a novel learning algorithm, which is guaranteed to converge, that an agent can use to negotiate optimally over time. We have implemented our algorithm and shown that it converges quickly in a wide range of cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, Heidelberg (1996)
Fudenberg, D., Levine, D.K.: The Theory of Learning in Games. MIT Press, Cambridge (1998)
Fudenberg, D., Tirole, J.: Game Theory. MIT Press, Cambridge (1991)
Hu, J., Wellman, M.P.: Multiagent reinforcement learning: Theoretical framework and an algorithm. In: Proceedings of the 11th International Conference on Machine Learning, pp. 242–250 (1998)
Kalai, E., Lehrer, E.: Rational learning leads to nash equilibrium. Econometrica 61(5), 1019–1045 (1993)
Karlin, S., Taylor, H.: First Course in Stochastic Processes. Academic Press, London (1974)
Kraus, S., Subrahmanian, V.S.: Multiagent reasoning with probability, time, and beliefs. International Journal of Intelligent Systems 10(5), 459–499 (1995)
Kulkarni, V.: Modelling and Analysis of Stochastic Systems. Chapman Hall/CRC (1996)
Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the 11th International Conference on Machine Learning, pp. 157–163 (1994)
Narayanan, V., Jennings, N.R.: An adaptive bilateral negotiation model for e-commerce settings. In: Proc. 7th Int. IEEE Conf. on E-Commerce Technology, Munich, Germany, pp. 34–39 (2005)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Weinberg, M., Rosenschein, J.: Best-response multiagent learning in non-stationary environments. In: The Third International Joint Conference on Autonomous Agents and Mutli-Agent Systems, pp. 506–513 (2004)
Zeng, D., Sycara, K.: Bayesian learning in negotiation. Int. J. Human-Computer Studies 48, 125–141 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Narayanan, V., Jennings, N.R. (2006). Learning to Negotiate Optimally in Non-stationary Environments. In: Klusch, M., Rovatsos, M., Payne, T.R. (eds) Cooperative Information Agents X. CIA 2006. Lecture Notes in Computer Science(), vol 4149. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11839354_21
Download citation
DOI: https://doi.org/10.1007/11839354_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38569-1
Online ISBN: 978-3-540-38570-7
eBook Packages: Computer ScienceComputer Science (R0)