Scalable block scheduling for efficient multi-database record linkage

Thilina Ranbaduge; Dinusha Vatsalan; Peter Christen

doi:10.1109/ICDM.2016.40

Scalable block scheduling for efficient multi-database record linkage

Thilina Ranbaduge, Dinusha Vatsalan, Peter Christen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Citations (Scopus)

Abstract

Record linkage (RL) is a task in data integration that aims to identify matching records that refer to the same entity from different databases. When records from more than two databases are to be linked RL is significantly challenged by the intrinsic exponential growth in the number of potential record comparisons to be conducted.We propose a scalable metablocking protocol to be used for Multi-Database RL (MDRL) to significantly reduce the complexity of the matching (comparison and classification) phase. Our approach uses a graph structure to schedule the comparison of pairs of blocks with the aim of minimizing the number of repeated and superfluous comparisons between records. We provide an analysis of our approach and conduct an empirical study on large real-world databases.

Original language	English
Title of host publication	Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016
Editors	Francesco Bonchi, Josep Domingo-Ferrer, Ricardo Baeza-Yates, Zhi-Hua Zhou, Xindong Wu
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1161-1166
Number of pages	6
ISBN (Electronic)	9781509054725
DOIs	https://doi.org/10.1109/ICDM.2016.40
Publication status	Published - 2 Jul 2016
Event	16th IEEE International Conference on Data Mining, ICDM 2016 - Barcelona, Catalonia, Spain Duration: 12 Dec 2016 → 15 Dec 2016

Publication series

Name	Proceedings - IEEE International Conference on Data Mining, ICDM
Volume	0
ISSN (Print)	1550-4786

Conference

Conference	16th IEEE International Conference on Data Mining, ICDM 2016
Country/Territory	Spain
City	Barcelona, Catalonia
Period	12/12/16 → 15/12/16

Access to Document

10.1109/ICDM.2016.40

Cite this

Ranbaduge, T., Vatsalan, D., & Christen, P. (2016). Scalable block scheduling for efficient multi-database record linkage. In F. Bonchi, J. Domingo-Ferrer, R. Baeza-Yates, Z.-H. Zhou, & X. Wu (Eds.), Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016 (pp. 1161-1166). Article 7837966 (Proceedings - IEEE International Conference on Data Mining, ICDM; Vol. 0). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDM.2016.40

Ranbaduge, Thilina ; Vatsalan, Dinusha ; Christen, Peter. / Scalable block scheduling for efficient multi-database record linkage. Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016. editor / Francesco Bonchi ; Josep Domingo-Ferrer ; Ricardo Baeza-Yates ; Zhi-Hua Zhou ; Xindong Wu. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 1161-1166 (Proceedings - IEEE International Conference on Data Mining, ICDM).

@inproceedings{6b7bdb4e1df946a38d0b5f9d09de7c22,

title = "Scalable block scheduling for efficient multi-database record linkage",

abstract = "Record linkage (RL) is a task in data integration that aims to identify matching records that refer to the same entity from different databases. When records from more than two databases are to be linked RL is significantly challenged by the intrinsic exponential growth in the number of potential record comparisons to be conducted.We propose a scalable metablocking protocol to be used for Multi-Database RL (MDRL) to significantly reduce the complexity of the matching (comparison and classification) phase. Our approach uses a graph structure to schedule the comparison of pairs of blocks with the aim of minimizing the number of repeated and superfluous comparisons between records. We provide an analysis of our approach and conduct an empirical study on large real-world databases.",

author = "Thilina Ranbaduge and Dinusha Vatsalan and Peter Christen",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 16th IEEE International Conference on Data Mining, ICDM 2016 ; Conference date: 12-12-2016 Through 15-12-2016",

year = "2016",

month = jul,

day = "2",

doi = "10.1109/ICDM.2016.40",

language = "English",

series = "Proceedings - IEEE International Conference on Data Mining, ICDM",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1161--1166",

editor = "Francesco Bonchi and Josep Domingo-Ferrer and Ricardo Baeza-Yates and Zhi-Hua Zhou and Xindong Wu",

booktitle = "Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016",

address = "United States",

}

Ranbaduge, T, Vatsalan, D & Christen, P 2016, Scalable block scheduling for efficient multi-database record linkage. in F Bonchi, J Domingo-Ferrer, R Baeza-Yates, Z-H Zhou & X Wu (eds), Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016., 7837966, Proceedings - IEEE International Conference on Data Mining, ICDM, vol. 0, Institute of Electrical and Electronics Engineers Inc., pp. 1161-1166, 16th IEEE International Conference on Data Mining, ICDM 2016, Barcelona, Catalonia, Spain, 12/12/16. https://doi.org/10.1109/ICDM.2016.40

Scalable block scheduling for efficient multi-database record linkage. / Ranbaduge, Thilina; Vatsalan, Dinusha; Christen, Peter.
Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016. ed. / Francesco Bonchi; Josep Domingo-Ferrer; Ricardo Baeza-Yates; Zhi-Hua Zhou; Xindong Wu. Institute of Electrical and Electronics Engineers Inc., 2016. p. 1161-1166 7837966 (Proceedings - IEEE International Conference on Data Mining, ICDM; Vol. 0).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Scalable block scheduling for efficient multi-database record linkage

AU - Ranbaduge, Thilina

AU - Vatsalan, Dinusha

AU - Christen, Peter

PY - 2016/7/2

Y1 - 2016/7/2

N2 - Record linkage (RL) is a task in data integration that aims to identify matching records that refer to the same entity from different databases. When records from more than two databases are to be linked RL is significantly challenged by the intrinsic exponential growth in the number of potential record comparisons to be conducted.We propose a scalable metablocking protocol to be used for Multi-Database RL (MDRL) to significantly reduce the complexity of the matching (comparison and classification) phase. Our approach uses a graph structure to schedule the comparison of pairs of blocks with the aim of minimizing the number of repeated and superfluous comparisons between records. We provide an analysis of our approach and conduct an empirical study on large real-world databases.

AB - Record linkage (RL) is a task in data integration that aims to identify matching records that refer to the same entity from different databases. When records from more than two databases are to be linked RL is significantly challenged by the intrinsic exponential growth in the number of potential record comparisons to be conducted.We propose a scalable metablocking protocol to be used for Multi-Database RL (MDRL) to significantly reduce the complexity of the matching (comparison and classification) phase. Our approach uses a graph structure to schedule the comparison of pairs of blocks with the aim of minimizing the number of repeated and superfluous comparisons between records. We provide an analysis of our approach and conduct an empirical study on large real-world databases.

UR - http://www.scopus.com/inward/record.url?scp=85014517722&partnerID=8YFLogxK

U2 - 10.1109/ICDM.2016.40

DO - 10.1109/ICDM.2016.40

M3 - Conference contribution

T3 - Proceedings - IEEE International Conference on Data Mining, ICDM

SP - 1161

EP - 1166

BT - Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016

A2 - Bonchi, Francesco

A2 - Domingo-Ferrer, Josep

A2 - Baeza-Yates, Ricardo

A2 - Zhou, Zhi-Hua

A2 - Wu, Xindong

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 16th IEEE International Conference on Data Mining, ICDM 2016

Y2 - 12 December 2016 through 15 December 2016

ER -

Ranbaduge T, Vatsalan D, Christen P. Scalable block scheduling for efficient multi-database record linkage. In Bonchi F, Domingo-Ferrer J, Baeza-Yates R, Zhou ZH, Wu X, editors, Proceedings - 16th IEEE International Conference on Data Mining, ICDM 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 1161-1166. 7837966. (Proceedings - IEEE International Conference on Data Mining, ICDM). doi: 10.1109/ICDM.2016.40

Scalable block scheduling for efficient multi-database record linkage

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this