Comparing Three Data Representations for Music with a Sequence-to-Sequence Model

Sichao Li*, Charles Patrick Martin

*Corresponding author for this work

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


    The choices of neural network model and data representation, a mapping between musical notation and input signals for a neural network, have emerged as a major challenge in creating convincing models for melody generation. Music generation can inspire creativity in artists and the general public, but choosing a proper data representation is complicated because the same musical piece can be presented in a range of expressive ways. In this paper, we compare three different data representations on the task of generating melodies with a sequence-to-sequence model, which generates melodies with flexible length, to explore how they affect the performance of generated music. These three representations are: a monophonic representation, playing one note each time, a polyphonic representation, indicating simultaneous notes and a complex polyphonic representation, expanding the polyphonic representation with dynamics. The influences of three data representations on the generated performance are compared and evaluated by mathematical analysis and human-cantered evaluation. The results show that different data representations fed into the same model endow the generated music with various features, the monophonic representation makes the music sound more melodious to humans’ ears, the polyphonic representation provides expressiveness and the complex-polyphonic representation guarantees the complexity of the generated music.

    Original languageEnglish
    Title of host publicationAI 2020
    Subtitle of host publicationAdvances in Artificial Intelligence - 33rd Australasian Joint Conference, AI 2020, Proceedings
    EditorsMarcus Gallagher, Nour Moustafa, Erandi Lakshika
    PublisherSpringer Science and Business Media Deutschland GmbH
    Number of pages13
    ISBN (Print)9783030649838
    Publication statusPublished - 2020
    Event33rd Australasian Joint Conference on Artificial Intelligence, AI 2020 - Canberra, ACT, Australia
    Duration: 29 Nov 202030 Nov 2020

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume12576 LNAI
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    Conference33rd Australasian Joint Conference on Artificial Intelligence, AI 2020
    CityCanberra, ACT


    Dive into the research topics of 'Comparing Three Data Representations for Music with a Sequence-to-Sequence Model'. Together they form a unique fingerprint.

    Cite this