Designing Curriculum for Deep Reinforcement Learning in StarCraft II

Daniel Hao, Penny Sweetser*, Matthew Aitchison

*Corresponding author for this work

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    3 Citations (Scopus)

    Abstract

    Reinforcement learning (RL) has proven successful in games, but suffers from long training times when compared to other forms of machine learning. Curriculum learning, an optimisation technique that improves a model’s ability to learn by presenting training samples in a meaningful order, known as curricula, could offer a solution. Curricula are usually designed manually, due to limitations involved with automating curricula generation. However, as there is a lack of research into effective design of curricula, researchers often rely on intuition and the resulting performance can vary. In this paper, we explore different ways of manually designing curricula for RL in real-time strategy game StarCraft II. We propose four generalised methods of manually creating curricula and verify their effectiveness through experiments. Our results show that all four of our proposed methods can improve a RL agent’s learning process when used correctly. We demonstrate that using subtasks, or modifying the state space of the tasks, is the most effective way to create training samples for StarCraft II. We found that utilising subtasks during training consistently accelerated the learning process of the agent and improved the agent’s final performance.

    Original languageEnglish
    Title of host publicationAI 2020
    Subtitle of host publicationAdvances in Artificial Intelligence - 33rd Australasian Joint Conference, AI 2020, Proceedings
    EditorsMarcus Gallagher, Nour Moustafa, Erandi Lakshika
    PublisherSpringer Science and Business Media Deutschland GmbH
    Pages243-255
    Number of pages13
    ISBN (Print)9783030649838
    DOIs
    Publication statusPublished - 2020
    Event33rd Australasian Joint Conference on Artificial Intelligence, AI 2020 - Canberra, ACT, Australia
    Duration: 29 Nov 202030 Nov 2020

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume12576 LNAI
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference33rd Australasian Joint Conference on Artificial Intelligence, AI 2020
    Country/TerritoryAustralia
    CityCanberra, ACT
    Period29/11/2030/11/20

    Fingerprint

    Dive into the research topics of 'Designing Curriculum for Deep Reinforcement Learning in StarCraft II'. Together they form a unique fingerprint.

    Cite this