Sparse Kernel-SARSA(Î») with an Eligibility Trace

Matthew Robards; Peter Sunehag; Scott Sanner; Bhaskara Marthi

doi:10.1007/978-3-642-23808-6_1

Sparse Kernel-SARSA(Î») with an Eligibility Trace

Matthew Robards, Peter Sunehag, Scott Sanner, Bhaskara Marthi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We introduce the first online kernelized version of SARSA(?) to permit sparsifi- cation for arbitrary ? for 0 = ? = 1; this is possible via a novel kernelization of the eligibility trace that is maintained separately from the kernelized value function. This separation is crucial for preserving the functional structure of the eligibility trace when using sparse kernel projection techniques that are essential for memory efficiency and capacity control. The result is a simple and practical Kernel-SARSA(?) algorithm for general 0 = ? = 1 that is memory-efficient in comparison to standard SARSA(?) (using various basis functions) on a range of domains including a real robotics task running on a Willow Garage PR2 robot.

Original language	English
Title of host publication	Proceedings of Machine Learning and Knowledge Discovery in Databases - European Conference (ECML PKDD 2011)
Editors	Dimitrios Gunopulos
Place of Publication	Berlin, Heidelberg
Publisher	Springer
Pages	16
Edition	Peer Reviewed
ISBN (Print)	9783642237799
DOIs	https://doi.org/10.1007/978-3-642-23808-6_1
Publication status	Published - 2011
Event	Machine Learning and Knowledge Discovery in Databases European Conference (ECML PKDD 2011) - Athens Greece Duration: 1 Jan 2011 → …

Conference

Conference	Machine Learning and Knowledge Discovery in Databases European Conference (ECML PKDD 2011)
Period	1/01/11 → …
Other	September 5-9 2011

Access to Document

10.1007/978-3-642-23808-6_1

Cite this

@inproceedings{5cb8adbf035746b786508c074d74bd13,

title = "Sparse Kernel-SARSA({\^I}») with an Eligibility Trace",

abstract = "We introduce the first online kernelized version of SARSA(?) to permit sparsifi- cation for arbitrary ? for 0 = ? = 1; this is possible via a novel kernelization of the eligibility trace that is maintained separately from the kernelized value function. This separation is crucial for preserving the functional structure of the eligibility trace when using sparse kernel projection techniques that are essential for memory efficiency and capacity control. The result is a simple and practical Kernel-SARSA(?) algorithm for general 0 = ? = 1 that is memory-efficient in comparison to standard SARSA(?) (using various basis functions) on a range of domains including a real robotics task running on a Willow Garage PR2 robot.",

author = "Matthew Robards and Peter Sunehag and Scott Sanner and Bhaskara Marthi",

year = "2011",

doi = "10.1007/978-3-642-23808-6_1",

language = "English",

isbn = "9783642237799",

pages = "16",

editor = "{Dimitrios Gunopulos}",

booktitle = "Proceedings of Machine Learning and Knowledge Discovery in Databases - European Conference (ECML PKDD 2011)",

publisher = "Springer",

edition = "Peer Reviewed",

note = "Machine Learning and Knowledge Discovery in Databases European Conference (ECML PKDD 2011) ; Conference date: 01-01-2011",

}

Robards, M, Sunehag, P, Sanner, S & Marthi, B 2011, Sparse Kernel-SARSA(Î») with an Eligibility Trace. in DG (ed.), Proceedings of Machine Learning and Knowledge Discovery in Databases - European Conference (ECML PKDD 2011). Peer Reviewed edn, Springer, Berlin, Heidelberg, pp. 16, Machine Learning and Knowledge Discovery in Databases European Conference (ECML PKDD 2011), 1/01/11. https://doi.org/10.1007/978-3-642-23808-6_1

Sparse Kernel-SARSA(Î») with an Eligibility Trace. / Robards, Matthew; Sunehag, Peter; Sanner, Scott et al.
Proceedings of Machine Learning and Knowledge Discovery in Databases - European Conference (ECML PKDD 2011). ed. / Dimitrios Gunopulos. Peer Reviewed. ed. Berlin, Heidelberg: Springer, 2011. p. 16.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Sparse Kernel-SARSA(Î») with an Eligibility Trace

AU - Robards, Matthew

AU - Sunehag, Peter

AU - Sanner, Scott

AU - Marthi, Bhaskara

PY - 2011

Y1 - 2011

N2 - We introduce the first online kernelized version of SARSA(?) to permit sparsifi- cation for arbitrary ? for 0 = ? = 1; this is possible via a novel kernelization of the eligibility trace that is maintained separately from the kernelized value function. This separation is crucial for preserving the functional structure of the eligibility trace when using sparse kernel projection techniques that are essential for memory efficiency and capacity control. The result is a simple and practical Kernel-SARSA(?) algorithm for general 0 = ? = 1 that is memory-efficient in comparison to standard SARSA(?) (using various basis functions) on a range of domains including a real robotics task running on a Willow Garage PR2 robot.

AB - We introduce the first online kernelized version of SARSA(?) to permit sparsifi- cation for arbitrary ? for 0 = ? = 1; this is possible via a novel kernelization of the eligibility trace that is maintained separately from the kernelized value function. This separation is crucial for preserving the functional structure of the eligibility trace when using sparse kernel projection techniques that are essential for memory efficiency and capacity control. The result is a simple and practical Kernel-SARSA(?) algorithm for general 0 = ? = 1 that is memory-efficient in comparison to standard SARSA(?) (using various basis functions) on a range of domains including a real robotics task running on a Willow Garage PR2 robot.

U2 - 10.1007/978-3-642-23808-6_1

DO - 10.1007/978-3-642-23808-6_1

M3 - Conference contribution

SN - 9783642237799

SP - 16

BT - Proceedings of Machine Learning and Knowledge Discovery in Databases - European Conference (ECML PKDD 2011)

A2 - null, Dimitrios Gunopulos

PB - Springer

CY - Berlin, Heidelberg

T2 - Machine Learning and Knowledge Discovery in Databases European Conference (ECML PKDD 2011)

Y2 - 1 January 2011

ER -

Sparse Kernel-SARSA(Î») with an Eligibility Trace

Abstract

Conference

Access to Document

Fingerprint

Cite this