The loss rank principle for model selection

Marcus Hutter

doi:10.1007/978-3-540-72927-3_42

The loss rank principle for model selection

Marcus Hutter^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

A key issue in statistics and machine learning is to automatically select the "right" model complexity, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polynomials. We suggest a novel principle (LoRP) for model selection in regression and classification. It is based on the loss rank, which counts how many other (fictitious) data would be fitted better. LoRP selects the model that has minimal loss rank. Unlike most penalized maximum likelihood variants (AIC,BIC,MDL), LoRP only depends on the regression functions and the loss function. It works without a stochastic noise model, and is directly applicable to any non-parametric regressor, like kNN.

Original language	English
Title of host publication	Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings
Publisher	Springer Verlag
Pages	589-603
Number of pages	15
ISBN (Print)	9783540729259
DOIs	https://doi.org/10.1007/978-3-540-72927-3_42
Publication status	Published - 2007
Event	20th Annual Conference on Learning Theory, COLT 2007 - San Diego, CA, United States Duration: 13 Jun 2007 → 15 Jun 2007

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	4539 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	20th Annual Conference on Learning Theory, COLT 2007
Country/Territory	United States
City	San Diego, CA
Period	13/06/07 → 15/06/07

Access to Document

10.1007/978-3-540-72927-3_42

Cite this

Hutter, M. (2007). The loss rank principle for model selection. In Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings (pp. 589-603). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4539 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-540-72927-3_42

@inproceedings{44d6fa5845784398b6116ac291836098,

title = "The loss rank principle for model selection",

abstract = "A key issue in statistics and machine learning is to automatically select the {"}right{"} model complexity, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polynomials. We suggest a novel principle (LoRP) for model selection in regression and classification. It is based on the loss rank, which counts how many other (fictitious) data would be fitted better. LoRP selects the model that has minimal loss rank. Unlike most penalized maximum likelihood variants (AIC,BIC,MDL), LoRP only depends on the regression functions and the loss function. It works without a stochastic noise model, and is directly applicable to any non-parametric regressor, like kNN.",

author = "Marcus Hutter",

year = "2007",

doi = "10.1007/978-3-540-72927-3_42",

language = "English",

isbn = "9783540729259",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "589--603",

booktitle = "Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings",

address = "Germany",

note = "20th Annual Conference on Learning Theory, COLT 2007 ; Conference date: 13-06-2007 Through 15-06-2007",

}

Hutter, M 2007, The loss rank principle for model selection. in Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4539 LNAI, Springer Verlag, pp. 589-603, 20th Annual Conference on Learning Theory, COLT 2007, San Diego, CA, United States, 13/06/07. https://doi.org/10.1007/978-3-540-72927-3_42

The loss rank principle for model selection. / Hutter, Marcus.
Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings. Springer Verlag, 2007. p. 589-603 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4539 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The loss rank principle for model selection

AU - Hutter, Marcus

PY - 2007

Y1 - 2007

N2 - A key issue in statistics and machine learning is to automatically select the "right" model complexity, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polynomials. We suggest a novel principle (LoRP) for model selection in regression and classification. It is based on the loss rank, which counts how many other (fictitious) data would be fitted better. LoRP selects the model that has minimal loss rank. Unlike most penalized maximum likelihood variants (AIC,BIC,MDL), LoRP only depends on the regression functions and the loss function. It works without a stochastic noise model, and is directly applicable to any non-parametric regressor, like kNN.

AB - A key issue in statistics and machine learning is to automatically select the "right" model complexity, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polynomials. We suggest a novel principle (LoRP) for model selection in regression and classification. It is based on the loss rank, which counts how many other (fictitious) data would be fitted better. LoRP selects the model that has minimal loss rank. Unlike most penalized maximum likelihood variants (AIC,BIC,MDL), LoRP only depends on the regression functions and the loss function. It works without a stochastic noise model, and is directly applicable to any non-parametric regressor, like kNN.

UR - http://www.scopus.com/inward/record.url?scp=38049041556&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-72927-3_42

DO - 10.1007/978-3-540-72927-3_42

M3 - Conference contribution

SN - 9783540729259

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 589

EP - 603

BT - Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings

PB - Springer Verlag

T2 - 20th Annual Conference on Learning Theory, COLT 2007

Y2 - 13 June 2007 through 15 June 2007

ER -

The loss rank principle for model selection

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this