Towards accurate marker-less human shape and pose estimation over time

Yinghao Huang, Federica Bogo, Christoph Lassner, Angjoo Kanazawa, Peter V. Gehler, Javier Romero, Ijaz Akhter, Michael J. Black

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    150 Citations (Scopus)

    Abstract

    Existing markerless motion capture methods often assume known backgrounds, static cameras, and sequence specific motion priors, limiting their application scenarios. Here we present a fully automatic method that, given multi-view videos, estimates 3D human pose and body shape. We take the recently proposed SMPLify method \cite{bogo2016keep} as the base method and extend it in several ways. First we fit a 3D human body model to 2D features detected in multi-view images. Second, we use a CNN method to segment the person in each image and fit the 3D body model to the contours, further improving accuracy. Third we utilize a generic and robust DCT temporal prior to handle the left and right side swapping issue sometimes introduced by the 2D pose estimator. Validation on standard benchmarks shows our results are comparable to the state of the art and also provide a realistic 3D shape avatar. We also demonstrate accurate results on HumanEva and on challenging monocular sequences of dancing from YouTube.

    Original languageEnglish
    Title of host publicationProceedings - 2017 International Conference on 3D Vision, 3DV 2017
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages421-430
    Number of pages10
    ISBN (Electronic)9781538626108
    DOIs
    Publication statusPublished - 25 May 2018
    Event7th IEEE International Conference on 3D Vision, 3DV 2017 - Qingdao, China
    Duration: 10 Oct 201712 Oct 2017

    Publication series

    NameProceedings - 2017 International Conference on 3D Vision, 3DV 2017

    Conference

    Conference7th IEEE International Conference on 3D Vision, 3DV 2017
    Country/TerritoryChina
    CityQingdao
    Period10/10/1712/10/17

    Fingerprint

    Dive into the research topics of 'Towards accurate marker-less human shape and pose estimation over time'. Together they form a unique fingerprint.

    Cite this