Reduction of Learning Time for Robots Using Automatic State Abstraction | SpringerLink
Skip to main content

Reduction of Learning Time for Robots Using Automatic State Abstraction

  • Conference paper
  • First Online:
European Robotics Symposium 2006

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 22))

Abstract

The required learning time and curse of dimensionality restrict the applicability of Reinforcement Learning(RL) on real robots. Difficulty in inclusion of initial knowledge and understanding the learned rules must be added to the mentioned problems. In this paper we address automatic state abstraction and creation of hierarchies in RL agent’s mind, as two major approaches for reducing the number of learning trials, simplifying inclusion of prior knowledge, and making the learned rules more abstract and understandable. We formalize automatic state abstraction and hierarchy creation as an optimization problem and derive a new algorithm that adapts decision tree learning techniques to state abstraction. The proof of performance is supported by strong evidences from simulation results in nondeterministic environments. Simulation results show encouraging enhancements in the required number of learning trials, agent’s performance, size of the learned trees, and computation time of the algorithm.

Keywords: State Abstraction, Hierarchical Reinforcement Learning

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

Author information

Authors and Affiliations

Authors

Editor information

Henrik I. Christensen

Rights and permissions

Reprints and permissions

About this paper

Cite this paper

Asadpour, M., Ahmadabadi, M.N., Siegwart, R. Reduction of Learning Time for Robots Using Automatic State Abstraction. In: Christensen, H.I. (eds) European Robotics Symposium 2006. Springer Tracts in Advanced Robotics, vol 22. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11681120_7

Download citation

  • DOI: https://doi.org/10.1007/11681120_7

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-32689-2

  • Online ISBN: 978-3-540-32689-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics