Abstract
The tasks of optimizing asset allocation considering transaction costs can be formulated into the framework of Markov Decision Processes(MDPs) and reinforcement learning. In this paper, a risk-averse reinforcement learning algorithm is proposed which improves asset allocation strategy of portfolio management systems. The proposed algorithm alternates policy evaluation phases which take into account the mean and variance of return under a given policy and policy improvement phases which follow the variance-penalized criterion. The algorithm is tested on trading systems for a single future corresponding to a Japanese stock index.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Baba, N.: A user friendly decision support system for dealing stocks using neural network. Proc. of IJCNN, Vol.1, (1993) 762–765.
Mahadevan, S.: Average reward reinforcement learning: foundations, algorithms, and empirical results. Machine Learning, vol.22, (1996) 159–196.
Makoto, S., Hajime, K., Shigenobu, K.: TD algorithm for the variance of return and mean-variance reinforcement learning. Journal of Japanese Society for Artificial Intelligence, (in Japanese, To appear).
Moody, J., Saffell, M.: Reinforcement learning for trading systems and portfolios. Proc. of KDD-98, (1998) 279–283.
Neuneier, R.: Optimal asset allocation using adaptive dynamic programming. Proc. of NIPS 9, MIT Press (1996).
Neuneier, R.: Enhancing Q-Learning for optimal asset allocation. Proc. of NIPS 10, MIT Press (1997) 936–942.
Neuneier, R., Mihatsch, O.: Risk sensitive reinforcement learning. Proc. of NIPS 11, MIT Press (1998) 1031–1037.
Puterman, M.L.: Markov Decision Processes. John Wiley & Sons, Inc., New York, (1994).
Sutton, R. S., Barto, A. G.: Reinforcement Learning: An Introduction. MIT Press, (1998).
Watkins, C.: Learning from delayed rewards. PhD thesis, King’s College, UK(1989).
White, D. J.: Mean, variance, and probabilistic criteria in finite Markov decision processes: A review. Journal of Optimization Theory and Applications, vol.56(1), (1988) 1–29.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sato, M., Kobayashi, S. (2000). Variance-Penalized Reinforcement Learning for Risk-Averse Asset Allocation. In: Leung, K.S., Chan, LW., Meng, H. (eds) Intelligent Data Engineering and Automated Learning — IDEAL 2000. Data Mining, Financial Engineering, and Intelligent Agents. IDEAL 2000. Lecture Notes in Computer Science, vol 1983. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44491-2_34
Download citation
DOI: https://doi.org/10.1007/3-540-44491-2_34
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41450-6
Online ISBN: 978-3-540-44491-6
eBook Packages: Springer Book Archive