Learning trajectory dependencies for human motion prediction

Wei Mao, Miaomiao Liu, Mathieu Salzmann, Hongdong Li

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    262 Citations (Scopus)

    Abstract

    Human motion prediction, i.e., forecasting future body poses given observed pose sequence, has typically been tackled with recurrent neural networks (RNNs). However, as evidenced by prior work, the resulted RNN models suffer from prediction errors accumulation, leading to undesired discontinuities in motion prediction. In this paper, we propose a simple feed-forward deep network for motion prediction, which takes into account both temporal smoothness and spatial dependencies among human body joints. In this context, we then propose to encode temporal information by working in trajectory space, instead of the traditionally-used pose space. This alleviates us from manually defining the range of temporal dependencies (or temporal convolutional filter size, as done in previous work). Moreover, spatial dependency of human pose is encoded by treating a human pose as a generic graph (rather than a human skeletal kinematic tree) formed by links between every pair of body joints. Instead of using a pre-defined graph structure, we design a new graph convolutional network to learn graph connectivity automatically. This allows the network to capture long range dependencies beyond that of human kinematic tree. We evaluate our approach on several standard benchmark datasets for motion prediction, including Human3.6M, the CMU motion capture dataset and 3DPW. Our experiments clearly demonstrate that the proposed approach achieves state of the art performance, and is applicable to both angle-based and position-based pose representations. The code is available at https://github.com/wei-mao-2019/LearnTrajDep.

    Original languageEnglish
    Title of host publicationProceedings - 2019 International Conference on Computer Vision, ICCV 2019
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages9488-9496
    Number of pages9
    ISBN (Electronic)9781728148038
    DOIs
    Publication statusPublished - Oct 2019
    Event17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 - Seoul, Korea, Republic of
    Duration: 27 Oct 20192 Nov 2019

    Publication series

    NameProceedings of the IEEE International Conference on Computer Vision
    Volume2019-October
    ISSN (Print)1550-5499

    Conference

    Conference17th IEEE/CVF International Conference on Computer Vision, ICCV 2019
    Country/TerritoryKorea, Republic of
    CitySeoul
    Period27/10/192/11/19

    Fingerprint

    Dive into the research topics of 'Learning trajectory dependencies for human motion prediction'. Together they form a unique fingerprint.

    Cite this