Learning image matching by simply watching video

Gucan Long*, Laurent Kneip, Jose M. Alvarez, Hongdong Li, Xiaohu Zhang, Qifeng Yu

*Corresponding author for this work

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    133 Citations (Scopus)


    This work presents an unsupervised learning based approach to the ubiquitous computer vision problem of image matching. We start from the insight that the problem of frame interpolation implicitly solves for inter-frame correspondences. This permits the application of analysisby-synthesis: we first train and apply a Convolutional Neural Network for frame interpolation, then obtain correspondences by inverting the learned CNN. The key benefit behind this strategy is that the CNN for frame interpolation can be trained in an unsupervised manner by exploiting the temporal coherence that is naturally contained in real-world video sequences. The present model therefore learns image matching by simply “watching videos”. Besides a promise to be more generally applicable, the presented approach achieves surprising performance comparable to traditional empirically designed methods.

    Original languageEnglish
    Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
    EditorsBastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
    PublisherSpringer Verlag
    Number of pages17
    ISBN (Print)9783319464657
    Publication statusPublished - 2016
    Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
    Duration: 8 Oct 201616 Oct 2016

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume9910 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349


    Conference14th European Conference on Computer Vision, ECCV 2016


    Dive into the research topics of 'Learning image matching by simply watching video'. Together they form a unique fingerprint.

    Cite this