The computational complexity of learning in sequential decision problems grows exponentially with the number of actions available to the agent at each state. We present a method for accelerating this process by learning action priors that express the usefulness of each action in each state. These are learned from a set of different optimal policies from many tasks in the same state space, and are used to bias exploration away from less useful actions. This is shown to improve performance for tasks in the same domain but with different goals. We extend our method to base action priors on perceptual cues rather than absolute states, allowing the transfer of these priors between tasks with differing state spaces and transition functions, and demonstrate experimentally the advantages of learning with action priors in a reinforcement learning context.
Reference:
Rosman, B.S. and Ramamoorthy, S. 2012. What good are actions? Accelerating learning using learned action priors. IEEE Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob 2012), San Diego, California, USA, 7-9 November 2012
Rosman, B. S., & Ramamoorthy, S. (2012). What good are actions? Accelerating learning using learned action priors. http://hdl.handle.net/10204/6475
Rosman, Benjamin S, and S Ramamoorthy. "What good are actions? Accelerating learning using learned action priors." (2012): http://hdl.handle.net/10204/6475
Rosman BS, Ramamoorthy S, What good are actions? Accelerating learning using learned action priors; 2012. http://hdl.handle.net/10204/6475 .