TY - GEN
T1 - Generalised discount functions applied to a Monte-Carlo AL implementation
AU - Lamont, Sean
AU - Aslanides, John
AU - Leike, Jan
AU - Hutter, Marcus
N1 - Publisher Copyright:
© Copyright 2017, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.
PY - 2017
Y1 - 2017
N2 - In recent years, work has been done to develop the theory of General Reinforcement Learning (GRL). However, there are no examples demonstrating the known results regarding generalised discounting. We have added to the GRL simulation platform (AIXIjs) the functionality to assign an agent arbitrary discount functions, and an environment which can be used to determine the effect of discounting on an agent's policy. Using this, we investigate how geometric, hyperbolic and power discounting affect an informed agent in a simple M DP. We experimentally reproduce a number of theoretical results, and discuss some related subtleties. It was found that the agent's behaviour followed what is expected theoretically, assuming appropriate parameters were chosen for the Monte-Carlo Tree Search (MOTS) planning algorithm.
AB - In recent years, work has been done to develop the theory of General Reinforcement Learning (GRL). However, there are no examples demonstrating the known results regarding generalised discounting. We have added to the GRL simulation platform (AIXIjs) the functionality to assign an agent arbitrary discount functions, and an environment which can be used to determine the effect of discounting on an agent's policy. Using this, we investigate how geometric, hyperbolic and power discounting affect an informed agent in a simple M DP. We experimentally reproduce a number of theoretical results, and discuss some related subtleties. It was found that the agent's behaviour followed what is expected theoretically, assuming appropriate parameters were chosen for the Monte-Carlo Tree Search (MOTS) planning algorithm.
UR - http://www.scopus.com/inward/record.url?scp=85031929132&partnerID=8YFLogxK
M3 - Conference contribution
T3 - Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
SP - 1589
EP - 1591
BT - 16th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2017
A2 - Durfee, Edmund
A2 - Winikoff, Michael
A2 - Larson, Kate
A2 - Das, Sanmay
PB - International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
T2 - 16th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2017
Y2 - 8 May 2017 through 12 May 2017
ER -