Automatic training example selection for scalable unsupervised record linkage

Peter Christen*

*Corresponding author for this work

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    29 Citations (Scopus)

    Abstract

    Linking records from two or more databases is an increasingly important data preparation step in many data mining projects, as linked data can enable studies that are not feasible otherwise, or that would require expensive collection of specific data. The aim of such linkages is to match all records that refer to the same entity. One of the main challenges in record linkage is the accurate classification of record pairs into matches and non-matches. Many modern classification techniques are based on supervised machine learning and thus require training data, which is often not available in real world situations. A novel two-step approach to unsupervised record pair classification is presented in this paper. In the first step, training examples are selected automatically, and they are then used in the second step to train a binary classifier. An experimental evaluation shows that this approach can outperform k-means clustering and also be much faster than other classification techniques.

    Original languageEnglish
    Title of host publicationAdvances in Knowledge Discovery and Data Mining - 12th Pacific-Asia Conference, PAKDD 2008, Proceedings
    Pages511-518
    Number of pages8
    DOIs
    Publication statusPublished - 2008
    Event12th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2008 - Osaka, Japan
    Duration: 20 May 200823 May 2008

    Publication series

    NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume5012 LNAI
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference12th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2008
    Country/TerritoryJapan
    CityOsaka
    Period20/05/0823/05/08

    Fingerprint

    Dive into the research topics of 'Automatic training example selection for scalable unsupervised record linkage'. Together they form a unique fingerprint.

    Cite this