Accelerated training of conditional random fields with stochastic gradient methods

S. V.N. Vishwanathan; Nicol N. Schraudolph; Mark W. Schmidt; Kevin P. Murphy

Accelerated training of conditional random fields with stochastic gradient methods

S. V.N. Vishwanathan^*, Nicol N. Schraudolph, Mark W. Schmidt, Kevin P. Murphy

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

148 Citations (Scopus)

Abstract

We apply Stochastic Meta-Descent (SMD), a stochastic gradient optimization method with gain vector adaptation, to the training of Conditional Random Fields (CRFs). On several large data sets, the resulting optimizer converges to the same quality of solution over an order of magnitude faster than limited-memory BFGS, the leading method reported to date. We report results for both exact and inexact inference techniques.

Original language	English
Title of host publication	ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning
Pages	969-976
Number of pages	8
Publication status	Published - 2006
Event	ICML 2006: 23rd International Conference on Machine Learning - Pittsburgh, PA, United States Duration: 25 Jun 2006 → 29 Jun 2006

Publication series

Name	ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning
Volume	2006

Conference

Conference	ICML 2006: 23rd International Conference on Machine Learning
Country/Territory	United States
City	Pittsburgh, PA
Period	25/06/06 → 29/06/06

Cite this

@inproceedings{5db2e9915e2c4213a5cf960045d5b966,

title = "Accelerated training of conditional random fields with stochastic gradient methods",

abstract = "We apply Stochastic Meta-Descent (SMD), a stochastic gradient optimization method with gain vector adaptation, to the training of Conditional Random Fields (CRFs). On several large data sets, the resulting optimizer converges to the same quality of solution over an order of magnitude faster than limited-memory BFGS, the leading method reported to date. We report results for both exact and inexact inference techniques.",

author = "Vishwanathan, {S. V.N.} and Schraudolph, {Nicol N.} and Schmidt, {Mark W.} and Murphy, {Kevin P.}",

year = "2006",

language = "English",

isbn = "1595933832",

series = "ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning",

pages = "969--976",

booktitle = "ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning",

note = "ICML 2006: 23rd International Conference on Machine Learning ; Conference date: 25-06-2006 Through 29-06-2006",

}

Vishwanathan, SVN, Schraudolph, NN, Schmidt, MW & Murphy, KP 2006, Accelerated training of conditional random fields with stochastic gradient methods. in ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning. ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning, vol. 2006, pp. 969-976, ICML 2006: 23rd International Conference on Machine Learning, Pittsburgh, PA, United States, 25/06/06.

Accelerated training of conditional random fields with stochastic gradient methods. / Vishwanathan, S. V.N.; Schraudolph, Nicol N.; Schmidt, Mark W. et al.
ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning. 2006. p. 969-976 (ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning; Vol. 2006).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Accelerated training of conditional random fields with stochastic gradient methods

AU - Vishwanathan, S. V.N.

AU - Schraudolph, Nicol N.

AU - Schmidt, Mark W.

AU - Murphy, Kevin P.

PY - 2006

Y1 - 2006

N2 - We apply Stochastic Meta-Descent (SMD), a stochastic gradient optimization method with gain vector adaptation, to the training of Conditional Random Fields (CRFs). On several large data sets, the resulting optimizer converges to the same quality of solution over an order of magnitude faster than limited-memory BFGS, the leading method reported to date. We report results for both exact and inexact inference techniques.

AB - We apply Stochastic Meta-Descent (SMD), a stochastic gradient optimization method with gain vector adaptation, to the training of Conditional Random Fields (CRFs). On several large data sets, the resulting optimizer converges to the same quality of solution over an order of magnitude faster than limited-memory BFGS, the leading method reported to date. We report results for both exact and inexact inference techniques.

UR - http://www.scopus.com/inward/record.url?scp=33749243756&partnerID=8YFLogxK

M3 - Conference contribution

SN - 1595933832

SN - 9781595933836

T3 - ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning

SP - 969

EP - 976

BT - ICML 2006 - Proceedings of the 23rd International Conference on Machine Learning

T2 - ICML 2006: 23rd International Conference on Machine Learning

Y2 - 25 June 2006 through 29 June 2006

ER -

Accelerated training of conditional random fields with stochastic gradient methods

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this