A new local distance-based outlier detection approach for scattered real-world data

Ke Zhang; Marcus Hutter; Huidong Jin

doi:10.1007/978-3-642-01307-2_84

A new local distance-based outlier detection approach for scattered real-world data

Ke Zhang^*, Marcus Hutter, Huidong Jin

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

330 Citations (Scopus)

Abstract

Detecting outliers which are grossly different from or inconsistent with the remaining dataset is a major challenge in real-world KDD applications. Existing outlier detection methods are ineffective on scattered real-world datasets due to implicit data patterns and parameter setting issues. We define a novel Local Distance-based Outlier Factor (LDOF) to measure the outlier-ness of objects in scattered datasets which addresses these issues. LDOF uses the relative location of an object to its neighbours to determine the degree to which the object deviates from its neighbourhood. We present theoretical bounds on LDOF's false-detection probability. Experimentally, LDOF compares favorably to classical KNN and LOF based outlier detection. In particular it is less sensitive to parameter values.

Original language	English
Title of host publication	13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009
Pages	813-822
Number of pages	10
DOIs	https://doi.org/10.1007/978-3-642-01307-2_84
Publication status	Published - 2009
Event	13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009 - Bangkok, Thailand Duration: 27 Apr 2009 → 30 Apr 2009

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5476 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009
Country/Territory	Thailand
City	Bangkok
Period	27/04/09 → 30/04/09

Access to Document

10.1007/978-3-642-01307-2_84

Cite this

Zhang, K., Hutter, M., & Jin, H. (2009). A new local distance-based outlier detection approach for scattered real-world data. In 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009 (pp. 813-822). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5476 LNAI). https://doi.org/10.1007/978-3-642-01307-2_84

@inproceedings{a3ec80ab69334acfb4d0d458d379f17a,

title = "A new local distance-based outlier detection approach for scattered real-world data",

abstract = "Detecting outliers which are grossly different from or inconsistent with the remaining dataset is a major challenge in real-world KDD applications. Existing outlier detection methods are ineffective on scattered real-world datasets due to implicit data patterns and parameter setting issues. We define a novel Local Distance-based Outlier Factor (LDOF) to measure the outlier-ness of objects in scattered datasets which addresses these issues. LDOF uses the relative location of an object to its neighbours to determine the degree to which the object deviates from its neighbourhood. We present theoretical bounds on LDOF's false-detection probability. Experimentally, LDOF compares favorably to classical KNN and LOF based outlier detection. In particular it is less sensitive to parameter values.",

keywords = "K-distance, KNN, LDOF, LOF, Scattered data, local outlier",

author = "Ke Zhang and Marcus Hutter and Huidong Jin",

year = "2009",

doi = "10.1007/978-3-642-01307-2_84",

language = "English",

isbn = "3642013066",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "813--822",

booktitle = "13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009",

note = "13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009 ; Conference date: 27-04-2009 Through 30-04-2009",

}

Zhang, K, Hutter, M & Jin, H 2009, A new local distance-based outlier detection approach for scattered real-world data. in 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5476 LNAI, pp. 813-822, 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009, Bangkok, Thailand, 27/04/09. https://doi.org/10.1007/978-3-642-01307-2_84

A new local distance-based outlier detection approach for scattered real-world data. / Zhang, Ke; Hutter, Marcus; Jin, Huidong.
13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009. 2009. p. 813-822 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5476 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A new local distance-based outlier detection approach for scattered real-world data

AU - Zhang, Ke

AU - Hutter, Marcus

AU - Jin, Huidong

PY - 2009

Y1 - 2009

N2 - Detecting outliers which are grossly different from or inconsistent with the remaining dataset is a major challenge in real-world KDD applications. Existing outlier detection methods are ineffective on scattered real-world datasets due to implicit data patterns and parameter setting issues. We define a novel Local Distance-based Outlier Factor (LDOF) to measure the outlier-ness of objects in scattered datasets which addresses these issues. LDOF uses the relative location of an object to its neighbours to determine the degree to which the object deviates from its neighbourhood. We present theoretical bounds on LDOF's false-detection probability. Experimentally, LDOF compares favorably to classical KNN and LOF based outlier detection. In particular it is less sensitive to parameter values.

AB - Detecting outliers which are grossly different from or inconsistent with the remaining dataset is a major challenge in real-world KDD applications. Existing outlier detection methods are ineffective on scattered real-world datasets due to implicit data patterns and parameter setting issues. We define a novel Local Distance-based Outlier Factor (LDOF) to measure the outlier-ness of objects in scattered datasets which addresses these issues. LDOF uses the relative location of an object to its neighbours to determine the degree to which the object deviates from its neighbourhood. We present theoretical bounds on LDOF's false-detection probability. Experimentally, LDOF compares favorably to classical KNN and LOF based outlier detection. In particular it is less sensitive to parameter values.

KW - K-distance

KW - KNN

KW - LDOF

KW - LOF

KW - Scattered data

KW - local outlier

UR - http://www.scopus.com/inward/record.url?scp=67650671551&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-01307-2_84

DO - 10.1007/978-3-642-01307-2_84

M3 - Conference contribution

SN - 3642013066

SN - 9783642013065

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 813

EP - 822

BT - 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009

T2 - 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009

Y2 - 27 April 2009 through 30 April 2009

ER -

Zhang K, Hutter M, Jin H. A new local distance-based outlier detection approach for scattered real-world data. In 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2009. 2009. p. 813-822. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-01307-2_84

A new local distance-based outlier detection approach for scattered real-world data

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this