4D Light Field Superpixel and Segmentation

Hao Zhu, Qi Zhang, Qing Wang*, Hongdong Li

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review

    20 Citations (Scopus)

    Abstract

    Superpixel segmentation of 2D images has been widely used in many computer vision tasks. Previous algorithms model the color, position, or higher spectral information for segmenting a 2D image. However, limited to the Gaussian imaging principle in a traditional camera, where each pixel is formed by summing lots of light rays from different angles, there is not a thorough segmentation solution to eliminate the ambiguity in defocus and occlusion boundary areas. In this paper, we consider the essential element of image pixel, i.e., rays in light space, and propose light field superpixel (LFSP) to eliminate the ambiguity. The LFSP is first defined mathematically and then two evaluation metrics, named LFSP self-similarity and effective label ratio, are proposed to evaluate the refocus-invariant and full-sliced properties of segmentation. By building a clique system containing 80 neighbors in light field, a robust refocus-invariant LFSP segmentation algorithm is developed. Experimental results on both synthetic and real light field datasets demonstrate the advantages over the current state of the art in terms of traditional evaluation metrics. Additionally, the LFSP self-similarity evaluations under different light field refocus levels show the refocus-invariance of the proposed algorithm. The full-sliced property of the proposed LFSP algorithm is verified by comparing it with the classical supervoxel algorithms. Finally, an LFSP-based application is demonstrated to show the effectiveness of LFSP in light field editing.

    Original languageEnglish
    Article number8763926
    Pages (from-to)85-99
    Number of pages15
    JournalIEEE Transactions on Image Processing
    Volume29
    DOIs
    Publication statusPublished - 2020

    Fingerprint

    Dive into the research topics of '4D Light Field Superpixel and Segmentation'. Together they form a unique fingerprint.

    Cite this