TY - GEN

T1 - The loss rank principle for model selection

AU - Hutter, Marcus

PY - 2007

Y1 - 2007

N2 - A key issue in statistics and machine learning is to automatically select the "right" model complexity, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polynomials. We suggest a novel principle (LoRP) for model selection in regression and classification. It is based on the loss rank, which counts how many other (fictitious) data would be fitted better. LoRP selects the model that has minimal loss rank. Unlike most penalized maximum likelihood variants (AIC,BIC,MDL), LoRP only depends on the regression functions and the loss function. It works without a stochastic noise model, and is directly applicable to any non-parametric regressor, like kNN.

AB - A key issue in statistics and machine learning is to automatically select the "right" model complexity, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polynomials. We suggest a novel principle (LoRP) for model selection in regression and classification. It is based on the loss rank, which counts how many other (fictitious) data would be fitted better. LoRP selects the model that has minimal loss rank. Unlike most penalized maximum likelihood variants (AIC,BIC,MDL), LoRP only depends on the regression functions and the loss function. It works without a stochastic noise model, and is directly applicable to any non-parametric regressor, like kNN.

UR - http://www.scopus.com/inward/record.url?scp=38049041556&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-72927-3_42

DO - 10.1007/978-3-540-72927-3_42

M3 - Conference contribution

SN - 9783540729259

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 589

EP - 603

BT - Learning Theory - 20th Annual Conference on Learning Theory, COLT 2007, Proceedings

PB - Springer Verlag

T2 - 20th Annual Conference on Learning Theory, COLT 2007

Y2 - 13 June 2007 through 15 June 2007

ER -