Invited Talk: PRISM – Practical RL: Representation, Interaction, Synthesis, and Mortality

Abstract

When scaling up RL to large continuous domains with imperfect representations and hierarchical structure, we often try applying algorithms that are proven to converge in small finite domains, and then just hope for the best. This talk will advocate instead designing algorithms that adhere to the constraints, and indeed take advantage of the opportunities, that might come with the problem at hand. Drawing on several different research threads within the Learning Agents Research Group at UT Austin, I will discuss four types of issues that arise from these contraints and opportunities: 1) Representation – choosing the algorithm for the problem’s representation and adapating the representation to fit the algorithm; 2) Interaction – with other agents and with human trainers; 3) Synthesis – of different algorithms for the same problem and of different concepts in the same algorithm; and 4) Mortality – the opportunity to improve learning based on past experience and the constraint that one can’t explore exhaustively.