Implicit online learning with kernels

Li Cheng; S. V.N. Vishwanathan; Dale Schuurmans; Shaojun Wang; Terry Caelli

Implicit online learning with kernels

Li Cheng^*, S. V.N. Vishwanathan, Dale Schuurmans, Shaojun Wang, Terry Caelli

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

50 Citations (Scopus)

Abstract

We present two new algorithms for online learning in reproducing kernel Hilbert spaces. Our first algorithm, ILK (implicit online learning with kernels), employs a new, implicit update technique that can be applied to a wide variety of convex loss functions. We then introduce a bounded memory version, SILK (sparse ILK), that maintains a compact representation of the predictor without compromising solution quality, even in non-stationary environments. We prove loss bounds and analyze the convergence rate of both. Experimental evidence shows that our proposed algorithms outperform current methods on synthetic and real data.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 19 - Proceedings of the 2006 Conference
Pages	249-256
Number of pages	8
Publication status	Published - 2007
Externally published	Yes
Event	20th Annual Conference on Neural Information Processing Systems, NIPS 2006 - Vancouver, BC, Canada Duration: 4 Dec 2006 → 7 Dec 2006

Publication series

Name	Advances in Neural Information Processing Systems
ISSN (Print)	1049-5258

Conference

Conference	20th Annual Conference on Neural Information Processing Systems, NIPS 2006
Country/Territory	Canada
City	Vancouver, BC
Period	4/12/06 → 7/12/06

Cite this

@inproceedings{0a334e706a57475b8a5c7168bdb129a8,

title = "Implicit online learning with kernels",

abstract = "We present two new algorithms for online learning in reproducing kernel Hilbert spaces. Our first algorithm, ILK (implicit online learning with kernels), employs a new, implicit update technique that can be applied to a wide variety of convex loss functions. We then introduce a bounded memory version, SILK (sparse ILK), that maintains a compact representation of the predictor without compromising solution quality, even in non-stationary environments. We prove loss bounds and analyze the convergence rate of both. Experimental evidence shows that our proposed algorithms outperform current methods on synthetic and real data.",

author = "Li Cheng and Vishwanathan, {S. V.N.} and Dale Schuurmans and Shaojun Wang and Terry Caelli",

year = "2007",

language = "English",

isbn = "9780262195683",

series = "Advances in Neural Information Processing Systems",

pages = "249--256",

booktitle = "Advances in Neural Information Processing Systems 19 - Proceedings of the 2006 Conference",

note = "20th Annual Conference on Neural Information Processing Systems, NIPS 2006 ; Conference date: 04-12-2006 Through 07-12-2006",

}

Cheng, L, Vishwanathan, SVN, Schuurmans, D, Wang, S & Caelli, T 2007, Implicit online learning with kernels. in Advances in Neural Information Processing Systems 19 - Proceedings of the 2006 Conference. Advances in Neural Information Processing Systems, pp. 249-256, 20th Annual Conference on Neural Information Processing Systems, NIPS 2006, Vancouver, BC, Canada, 4/12/06.

TY - GEN

T1 - Implicit online learning with kernels

AU - Cheng, Li

AU - Vishwanathan, S. V.N.

AU - Schuurmans, Dale

AU - Wang, Shaojun

AU - Caelli, Terry

PY - 2007

Y1 - 2007

N2 - We present two new algorithms for online learning in reproducing kernel Hilbert spaces. Our first algorithm, ILK (implicit online learning with kernels), employs a new, implicit update technique that can be applied to a wide variety of convex loss functions. We then introduce a bounded memory version, SILK (sparse ILK), that maintains a compact representation of the predictor without compromising solution quality, even in non-stationary environments. We prove loss bounds and analyze the convergence rate of both. Experimental evidence shows that our proposed algorithms outperform current methods on synthetic and real data.

AB - We present two new algorithms for online learning in reproducing kernel Hilbert spaces. Our first algorithm, ILK (implicit online learning with kernels), employs a new, implicit update technique that can be applied to a wide variety of convex loss functions. We then introduce a bounded memory version, SILK (sparse ILK), that maintains a compact representation of the predictor without compromising solution quality, even in non-stationary environments. We prove loss bounds and analyze the convergence rate of both. Experimental evidence shows that our proposed algorithms outperform current methods on synthetic and real data.

UR - http://www.scopus.com/inward/record.url?scp=84864074626&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9780262195683

T3 - Advances in Neural Information Processing Systems

SP - 249

EP - 256

BT - Advances in Neural Information Processing Systems 19 - Proceedings of the 2006 Conference

T2 - 20th Annual Conference on Neural Information Processing Systems, NIPS 2006

Y2 - 4 December 2006 through 7 December 2006

ER -

Implicit online learning with kernels

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this