TY - GEN
T1 - Designing Curriculum for Deep Reinforcement Learning in StarCraft II
AU - Hao, Daniel
AU - Sweetser, Penny
AU - Aitchison, Matthew
N1 - Publisher Copyright:
© 2020, Springer Nature Switzerland AG.
PY - 2020
Y1 - 2020
N2 - Reinforcement learning (RL) has proven successful in games, but suffers from long training times when compared to other forms of machine learning. Curriculum learning, an optimisation technique that improves a model’s ability to learn by presenting training samples in a meaningful order, known as curricula, could offer a solution. Curricula are usually designed manually, due to limitations involved with automating curricula generation. However, as there is a lack of research into effective design of curricula, researchers often rely on intuition and the resulting performance can vary. In this paper, we explore different ways of manually designing curricula for RL in real-time strategy game StarCraft II. We propose four generalised methods of manually creating curricula and verify their effectiveness through experiments. Our results show that all four of our proposed methods can improve a RL agent’s learning process when used correctly. We demonstrate that using subtasks, or modifying the state space of the tasks, is the most effective way to create training samples for StarCraft II. We found that utilising subtasks during training consistently accelerated the learning process of the agent and improved the agent’s final performance.
AB - Reinforcement learning (RL) has proven successful in games, but suffers from long training times when compared to other forms of machine learning. Curriculum learning, an optimisation technique that improves a model’s ability to learn by presenting training samples in a meaningful order, known as curricula, could offer a solution. Curricula are usually designed manually, due to limitations involved with automating curricula generation. However, as there is a lack of research into effective design of curricula, researchers often rely on intuition and the resulting performance can vary. In this paper, we explore different ways of manually designing curricula for RL in real-time strategy game StarCraft II. We propose four generalised methods of manually creating curricula and verify their effectiveness through experiments. Our results show that all four of our proposed methods can improve a RL agent’s learning process when used correctly. We demonstrate that using subtasks, or modifying the state space of the tasks, is the most effective way to create training samples for StarCraft II. We found that utilising subtasks during training consistently accelerated the learning process of the agent and improved the agent’s final performance.
KW - Curriculum learning
KW - Game AI
KW - Real-time strategy games
KW - Reinforcement learning
KW - StarCraft II
UR - http://www.scopus.com/inward/record.url?scp=85097652502&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-64984-5_19
DO - 10.1007/978-3-030-64984-5_19
M3 - Conference contribution
SN - 9783030649838
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 243
EP - 255
BT - AI 2020
A2 - Gallagher, Marcus
A2 - Moustafa, Nour
A2 - Lakshika, Erandi
PB - Springer Science and Business Media Deutschland GmbH
T2 - 33rd Australasian Joint Conference on Artificial Intelligence, AI 2020
Y2 - 29 November 2020 through 30 November 2020
ER -