Reducing the Sim-to-Real Gap for Event Cameras

Timo Stoffregen*, Cedric Scheerlinck, Davide Scaramuzza, Tom Drummond, Nick Barnes, Lindsay Kleeman, Robert Mahony

*Corresponding author for this work

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    57 Citations (Scopus)

    Abstract

    Event cameras are paradigm-shifting novel sensors that report asynchronous, per-pixel brightness changes called ‘events’ with unparalleled low latency. This makes them ideal for high speed, high dynamic range scenes where conventional cameras would fail. Recent work has demonstrated impressive results using Convolutional Neural Networks (CNNs) for video reconstruction and optic flow with events. We present strategies for improving training data for event based CNNs that result in 20–40% boost in performance of existing state-of-the-art (SOTA) video reconstruction networks retrained with our method, and up to 15% for optic flow networks. A challenge in evaluating event based video reconstruction is lack of quality ground truth images in existing datasets. To address this, we present a new High Quality Frames (HQF) dataset, containing events and ground truth frames from a DAVIS240C that are well-exposed and minimally motion-blurred. We evaluate our method on HQF + several existing major event camera datasets.

    Original languageEnglish
    Title of host publicationComputer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings
    EditorsAndrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
    PublisherSpringer Science and Business Media Deutschland GmbH
    Pages534-549
    Number of pages16
    ISBN (Print)9783030585822
    DOIs
    Publication statusPublished - 2020
    Event16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom
    Duration: 23 Aug 202028 Aug 2020

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume12372 LNCS
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference16th European Conference on Computer Vision, ECCV 2020
    Country/TerritoryUnited Kingdom
    CityGlasgow
    Period23/08/2028/08/20

    Fingerprint

    Dive into the research topics of 'Reducing the Sim-to-Real Gap for Event Cameras'. Together they form a unique fingerprint.

    Cite this