Video Representation Learning Using Discriminative Pooling

Jue Wang, Anoop Cherian, Fatih Porikli, Stephen Gould

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    55 Citations (Scopus)

    Abstract

    Popular deep models for action recognition in videos generate independent predictions for short clips, which are then pooled heuristically to assign an action label to the full video segment. As not all frames may characterize the underlying action-indeed, many are common across multiple actions-pooling schemes that impose equal importance on all frames might be unfavorable. In an attempt to tackle this problem, we propose discriminative pooling, based on the notion that among the deep features generated on all short clips, there is at least one that characterizes the action. To this end, we learn a (nonlinear) hyperplane that separates this unknown, yet discriminative, feature from the rest. Applying multiple instance learning in a large-margin setup, we use the parameters of this separating hyperplane as a descriptor for the full video segment. Since these parameters are directly related to the support vectors in a max-margin framework, they serve as robust representations for pooling of the features. We formulate a joint objective and an efficient solver that learns these hyperplanes per video and the corresponding action classifiers over the hyperplanes. Our pooling scheme is end-to-end trainable within a deep framework. We report results from experiments on three benchmark datasets spanning a variety of challenges and demonstrate state-of-the-art performance across these tasks.

    Original languageEnglish
    Title of host publicationProceedings - 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018
    PublisherIEEE Computer Society
    Pages1149-1158
    Number of pages10
    ISBN (Electronic)9781538664209
    DOIs
    Publication statusPublished - 14 Dec 2018
    Event31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018 - Salt Lake City, United States
    Duration: 18 Jun 201822 Jun 2018

    Publication series

    NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
    ISSN (Print)1063-6919

    Conference

    Conference31st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2018
    Country/TerritoryUnited States
    CitySalt Lake City
    Period18/06/1822/06/18

    Fingerprint

    Dive into the research topics of 'Video Representation Learning Using Discriminative Pooling'. Together they form a unique fingerprint.

    Cite this