Active learning based entity resolution using Markov logic

Jeffrey Fisher; Peter Christen; Qing Wang

doi:10.1007/978-3-319-31750-2_27

Active learning based entity resolution using Markov logic

Jeffrey Fisher^*, Peter Christen, Qing Wang

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

12 Citations (Scopus)

Abstract

Entity resolution is a common data cleaning and data integration problem that involves determining which records in one or more data sets refer to the same real-world entities. It has numerous applications for commercial, academic and government organisations. For most practical entity resolution applications, training data does not exist which limits the type of classification models that can be applied. This also prevents complex techniques such as Markov logic networks from being used on real-world problems. In this paper we apply an active learning based technique to generate training data for a Markov logic network based entity resolution model and learn the weights for the formulae in a Markov logic network. We evaluate our technique on realworld data sets and show that we can generate balanced training data and learn and also learn approximate weights for the formulae in the Markov logic network.

Original language	English
Title of host publication	Advances in Knowledge Discovery and Data Mining - 20th Pacific-Asia Conference, PAKDD 2016, Proceedings
Editors	James Bailey, Latifur Khan, Takashi Washio, Gillian Dobbie, Joshua Zhexue Huang, Ruili Wang
Publisher	Springer Verlag
Pages	338-349
Number of pages	12
ISBN (Print)	9783319317496
DOIs	https://doi.org/10.1007/978-3-319-31750-2_27
Publication status	Published - 2016
Event	20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2016 - Auckland, New Zealand Duration: 19 Apr 2016 → 22 Apr 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9652 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2016
Country/Territory	New Zealand
City	Auckland
Period	19/04/16 → 22/04/16

Access to Document

10.1007/978-3-319-31750-2_27

Cite this

Fisher, J., Christen, P., & Wang, Q. (2016). Active learning based entity resolution using Markov logic. In J. Bailey, L. Khan, T. Washio, G. Dobbie, J. Z. Huang, & R. Wang (Eds.), Advances in Knowledge Discovery and Data Mining - 20th Pacific-Asia Conference, PAKDD 2016, Proceedings (pp. 338-349). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9652 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-31750-2_27

Fisher, Jeffrey ; Christen, Peter ; Wang, Qing. / Active learning based entity resolution using Markov logic. Advances in Knowledge Discovery and Data Mining - 20th Pacific-Asia Conference, PAKDD 2016, Proceedings. editor / James Bailey ; Latifur Khan ; Takashi Washio ; Gillian Dobbie ; Joshua Zhexue Huang ; Ruili Wang. Springer Verlag, 2016. pp. 338-349 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d6c99441be0b46aea068f1f36f3c1682,

title = "Active learning based entity resolution using Markov logic",

abstract = "Entity resolution is a common data cleaning and data integration problem that involves determining which records in one or more data sets refer to the same real-world entities. It has numerous applications for commercial, academic and government organisations. For most practical entity resolution applications, training data does not exist which limits the type of classification models that can be applied. This also prevents complex techniques such as Markov logic networks from being used on real-world problems. In this paper we apply an active learning based technique to generate training data for a Markov logic network based entity resolution model and learn the weights for the formulae in a Markov logic network. We evaluate our technique on realworld data sets and show that we can generate balanced training data and learn and also learn approximate weights for the formulae in the Markov logic network.",

author = "Jeffrey Fisher and Peter Christen and Qing Wang",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2016.; 20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2016 ; Conference date: 19-04-2016 Through 22-04-2016",

year = "2016",

doi = "10.1007/978-3-319-31750-2_27",

language = "English",

isbn = "9783319317496",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "338--349",

editor = "James Bailey and Latifur Khan and Takashi Washio and Gillian Dobbie and Huang, {Joshua Zhexue} and Ruili Wang",

booktitle = "Advances in Knowledge Discovery and Data Mining - 20th Pacific-Asia Conference, PAKDD 2016, Proceedings",

address = "Germany",

}

Fisher, J, Christen, P & Wang, Q 2016, Active learning based entity resolution using Markov logic. in J Bailey, L Khan, T Washio, G Dobbie, JZ Huang & R Wang (eds), Advances in Knowledge Discovery and Data Mining - 20th Pacific-Asia Conference, PAKDD 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9652 LNAI, Springer Verlag, pp. 338-349, 20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2016, Auckland, New Zealand, 19/04/16. https://doi.org/10.1007/978-3-319-31750-2_27

Active learning based entity resolution using Markov logic. / Fisher, Jeffrey; Christen, Peter ; Wang, Qing.
Advances in Knowledge Discovery and Data Mining - 20th Pacific-Asia Conference, PAKDD 2016, Proceedings. ed. / James Bailey; Latifur Khan; Takashi Washio; Gillian Dobbie; Joshua Zhexue Huang; Ruili Wang. Springer Verlag, 2016. p. 338-349 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9652 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Active learning based entity resolution using Markov logic

AU - Fisher, Jeffrey

AU - Christen, Peter

AU - Wang, Qing

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2016.

PY - 2016

Y1 - 2016

N2 - Entity resolution is a common data cleaning and data integration problem that involves determining which records in one or more data sets refer to the same real-world entities. It has numerous applications for commercial, academic and government organisations. For most practical entity resolution applications, training data does not exist which limits the type of classification models that can be applied. This also prevents complex techniques such as Markov logic networks from being used on real-world problems. In this paper we apply an active learning based technique to generate training data for a Markov logic network based entity resolution model and learn the weights for the formulae in a Markov logic network. We evaluate our technique on realworld data sets and show that we can generate balanced training data and learn and also learn approximate weights for the formulae in the Markov logic network.

AB - Entity resolution is a common data cleaning and data integration problem that involves determining which records in one or more data sets refer to the same real-world entities. It has numerous applications for commercial, academic and government organisations. For most practical entity resolution applications, training data does not exist which limits the type of classification models that can be applied. This also prevents complex techniques such as Markov logic networks from being used on real-world problems. In this paper we apply an active learning based technique to generate training data for a Markov logic network based entity resolution model and learn the weights for the formulae in a Markov logic network. We evaluate our technique on realworld data sets and show that we can generate balanced training data and learn and also learn approximate weights for the formulae in the Markov logic network.

UR - http://www.scopus.com/inward/record.url?scp=84987974387&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-31750-2_27

DO - 10.1007/978-3-319-31750-2_27

M3 - Conference contribution

SN - 9783319317496

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 338

EP - 349

BT - Advances in Knowledge Discovery and Data Mining - 20th Pacific-Asia Conference, PAKDD 2016, Proceedings

A2 - Bailey, James

A2 - Khan, Latifur

A2 - Washio, Takashi

A2 - Dobbie, Gillian

A2 - Huang, Joshua Zhexue

A2 - Wang, Ruili

PB - Springer Verlag

T2 - 20th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2016

Y2 - 19 April 2016 through 22 April 2016

ER -

Fisher J, Christen P , Wang Q. Active learning based entity resolution using Markov logic. In Bailey J, Khan L, Washio T, Dobbie G, Huang JZ, Wang R, editors, Advances in Knowledge Discovery and Data Mining - 20th Pacific-Asia Conference, PAKDD 2016, Proceedings. Springer Verlag. 2016. p. 338-349. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-31750-2_27

Active learning based entity resolution using Markov logic

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this