TY - GEN
T1 - Learning Embodied Sound-Motion Mappings
T2 - 13th Conference on Creativity and Cognition, C and C 2021
AU - Wallace, Benedikte
AU - Martin, Charles P.
AU - Tørresen, Jim
AU - Nymoen, Kristian
N1 - Publisher Copyright:
© 2021 ACM.
PY - 2021/6/22
Y1 - 2021/6/22
N2 - Through dance, a wide range of emotions can be expressed. As virtual agents and robots continue to become part of our daily lives, the need for them to efficiently convey emotion and intent increases. When trained to dance, to what extent can AI learn to model the tacit mappings between sound and motion? Here, we explore the creative capacity of a generative model trained on 3D motion capture recordings of improvised dance. We perform a perceptual judgment experiment wherein respondents rate movement generated by our model as well as human performances. While the sound-motion mappings remain somewhat elusive, particularly when compared to examples of human dance, our study shows that in certain aspects related to perceived dance-likeness and expressivity, the model successfully mimics human dance movement. By employing a perceptual study to evaluate our generative model, we aim to further our ability to understand the affordances and limitations of creative AI.
AB - Through dance, a wide range of emotions can be expressed. As virtual agents and robots continue to become part of our daily lives, the need for them to efficiently convey emotion and intent increases. When trained to dance, to what extent can AI learn to model the tacit mappings between sound and motion? Here, we explore the creative capacity of a generative model trained on 3D motion capture recordings of improvised dance. We perform a perceptual judgment experiment wherein respondents rate movement generated by our model as well as human performances. While the sound-motion mappings remain somewhat elusive, particularly when compared to examples of human dance, our study shows that in certain aspects related to perceived dance-likeness and expressivity, the model successfully mimics human dance movement. By employing a perceptual study to evaluate our generative model, we aim to further our ability to understand the affordances and limitations of creative AI.
KW - Dance
KW - Embodied Music Cognition
KW - Generative AI
KW - Mixture Density Networks
KW - Perceptual judgement experiment
UR - http://www.scopus.com/inward/record.url?scp=85109091499&partnerID=8YFLogxK
U2 - 10.1145/3450741.3465245
DO - 10.1145/3450741.3465245
M3 - Conference contribution
AN - SCOPUS:85109091499
T3 - ACM International Conference Proceeding Series
BT - C and C 2021 - Proceedings of the 13th Conference on Creativity and Cognition
PB - Association for Computing Machinery
Y2 - 22 June 2021 through 23 June 2021
ER -