A scalable modular convex solver for regularized risk minimization

Choon Hui Teo; Alex Smola; S. V.N. Vishwanathan; Quoc Viet Le

doi:10.1145/1281192.1281270

A scalable modular convex solver for regularized risk minimization

Choon Hui Teo^*, Alex Smola, S. V.N. Vishwanathan, Quoc Viet Le

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

108 Citations (Scopus)

Abstract

A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a highly scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for data-locality, and can deal with regularizers such as l₁ and l ₂ penalties. At present, our solver implements 20 different estimation problems, can be easily extended, scales to millions of observations, and is up to 10 times faster than specialized solvers for many applications. The open source code is freely available as part of the ELEFANT toolbox.

Original language	English
Title of host publication	KDD-2007
Subtitle of host publication	Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Pages	727-736
Number of pages	10
DOIs	https://doi.org/10.1145/1281192.1281270
Publication status	Published - 2007
Externally published	Yes
Event	KDD-2007: 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - San Jose, CA, United States Duration: 12 Aug 2007 → 15 Aug 2007

Publication series

Name	Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Conference

Conference	KDD-2007: 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Country/Territory	United States
City	San Jose, CA
Period	12/08/07 → 15/08/07

Access to Document

10.1145/1281192.1281270

Cite this

Teo, C. H., Smola, A., Vishwanathan, S. V. N., & Le, Q. V. (2007). A scalable modular convex solver for regularized risk minimization. In KDD-2007: Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 727-736). (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). https://doi.org/10.1145/1281192.1281270

@inproceedings{00c6fe6d903d4477bbd36b44d816bbd5,

title = "A scalable modular convex solver for regularized risk minimization",

abstract = "A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a highly scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for data-locality, and can deal with regularizers such as l1 and l 2 penalties. At present, our solver implements 20 different estimation problems, can be easily extended, scales to millions of observations, and is up to 10 times faster than specialized solvers for many applications. The open source code is freely available as part of the ELEFANT toolbox.",

keywords = "Algorithms, Convexity, Optimization",

author = "Teo, {Choon Hui} and Alex Smola and Vishwanathan, {S. V.N.} and Le, {Quoc Viet}",

year = "2007",

doi = "10.1145/1281192.1281270",

language = "English",

isbn = "1595936092",

series = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

pages = "727--736",

booktitle = "KDD-2007",

note = "KDD-2007: 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining ; Conference date: 12-08-2007 Through 15-08-2007",

}

Teo, CH, Smola, A, Vishwanathan, SVN & Le, QV 2007, A scalable modular convex solver for regularized risk minimization. in KDD-2007: Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 727-736, KDD-2007: 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, CA, United States, 12/08/07. https://doi.org/10.1145/1281192.1281270

A scalable modular convex solver for regularized risk minimization. / Teo, Choon Hui; Smola, Alex; Vishwanathan, S. V.N. et al.
KDD-2007: Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2007. p. 727-736 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A scalable modular convex solver for regularized risk minimization

AU - Teo, Choon Hui

AU - Smola, Alex

AU - Vishwanathan, S. V.N.

AU - Le, Quoc Viet

PY - 2007

Y1 - 2007

N2 - A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a highly scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for data-locality, and can deal with regularizers such as l1 and l 2 penalties. At present, our solver implements 20 different estimation problems, can be easily extended, scales to millions of observations, and is up to 10 times faster than specialized solvers for many applications. The open source code is freely available as part of the ELEFANT toolbox.

AB - A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a highly scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for data-locality, and can deal with regularizers such as l1 and l 2 penalties. At present, our solver implements 20 different estimation problems, can be easily extended, scales to millions of observations, and is up to 10 times faster than specialized solvers for many applications. The open source code is freely available as part of the ELEFANT toolbox.

KW - Algorithms

KW - Convexity

KW - Optimization

UR - http://www.scopus.com/inward/record.url?scp=36849059715&partnerID=8YFLogxK

U2 - 10.1145/1281192.1281270

DO - 10.1145/1281192.1281270

M3 - Conference contribution

SN - 1595936092

SN - 9781595936097

T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

SP - 727

EP - 736

BT - KDD-2007

T2 - KDD-2007: 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Y2 - 12 August 2007 through 15 August 2007

ER -

Teo CH, Smola A, Vishwanathan SVN, Le QV. A scalable modular convex solver for regularized risk minimization. In KDD-2007: Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2007. p. 727-736. (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). doi: 10.1145/1281192.1281270

A scalable modular convex solver for regularized risk minimization

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this