Studying object naming with online photos and caption

Alexander Mathews, Lexing Xie, Xuming He

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    Abstract

    We explore what names people use to describe visual concepts and why these names are chosen. Choosing object names has been a topic of interest in cognitive psychology, but a systematic, data-driven approach for naming at the scale of thousands of objects does not yet exist. First, we -nd that visual context has interpretable e-ects on visual naming, by analysing the MSCOCO dataset that has manually annotated objects and captions containing the natural language names for the object. We show that taking into account other objects as context helps improve the prediction of object names. We then analyse the naming patterns on a large dataset from Flickr, using automatically detected concepts. Preliminary results indicate that naming patterns can be identi-ed on a large scale, but contrary to the conventional wisdom in cognitive psychology, are not dominated by genus for animals. We further validate the automatic method with a pilot Amazon Mechanical Turk naming experiment, and explore the impact of automatic concept detectors with t-SNE visualizations.

    Original languageEnglish
    Title of host publicationMMCommons 2015 - Proceedings of the 2015 Workshop on Community-Organized Multimodal Mining
    Subtitle of host publicationOpportunities for Novel Solutions, co-located with ACM MM 2015
    PublisherAssociation for Computing Machinery (ACM)
    Pages31-36
    Number of pages6
    ISBN (Electronic)9781450337441
    DOIs
    Publication statusPublished - 30 Oct 2015
    EventWorkshop on Community-Organized Multimodal Mining: Opportunities for Novel Solutions, MMCommons 2015 - Brisbane, Australia
    Duration: 30 Oct 2015 → …

    Publication series

    NameMMCommons 2015 - Proceedings of the 2015 Workshop on Community-Organized Multimodal Mining: Opportunities for Novel Solutions, co-located with ACM MM 2015

    Conference

    ConferenceWorkshop on Community-Organized Multimodal Mining: Opportunities for Novel Solutions, MMCommons 2015
    Country/TerritoryAustralia
    CityBrisbane
    Period30/10/15 → …

    Fingerprint

    Dive into the research topics of 'Studying object naming with online photos and caption'. Together they form a unique fingerprint.

    Cite this