Gradient based algorithms with loss functions and kernels for improved on-policy control

Matthew Robards; Peter Sunehag

doi:10.1007/978-3-642-29946-9_7

Gradient based algorithms with loss functions and kernels for improved on-policy control

Matthew Robards^*, Peter Sunehag

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We introduce and empirically evaluate two novel online gradient-based reinforcement learning algorithms with function approximation - one model based, and the other model free. These algorithms come with the possibility of having non-squared loss functions which is novel in reinforcement learning, and seems to come with empirical advantages. We further extend a previous gradient based algorithm to the case of full control, by using generalized policy iteration. Theoretical properties of these algorithms are studied in a companion paper.

Original language	English
Title of host publication	Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Revised Selected Papers
Pages	30-41
Number of pages	12
DOIs	https://doi.org/10.1007/978-3-642-29946-9_7
Publication status	Published - 2012
Event	9th European Workshop on Reinforcement Learning, EWRL 2011 - Athens, Greece Duration: 9 Sept 2011 → 11 Sept 2011

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	7188 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th European Workshop on Reinforcement Learning, EWRL 2011
Country/Territory	Greece
City	Athens
Period	9/09/11 → 11/09/11

Access to Document

10.1007/978-3-642-29946-9_7

Cite this

Robards, M., & Sunehag, P. (2012). Gradient based algorithms with loss functions and kernels for improved on-policy control. In Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Revised Selected Papers (pp. 30-41). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7188 LNAI). https://doi.org/10.1007/978-3-642-29946-9_7

Robards, Matthew ; Sunehag, Peter. / Gradient based algorithms with loss functions and kernels for improved on-policy control. Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Revised Selected Papers. 2012. pp. 30-41 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b3500bc06de1408f99369221f7f89489,

title = "Gradient based algorithms with loss functions and kernels for improved on-policy control",

abstract = "We introduce and empirically evaluate two novel online gradient-based reinforcement learning algorithms with function approximation - one model based, and the other model free. These algorithms come with the possibility of having non-squared loss functions which is novel in reinforcement learning, and seems to come with empirical advantages. We further extend a previous gradient based algorithm to the case of full control, by using generalized policy iteration. Theoretical properties of these algorithms are studied in a companion paper.",

author = "Matthew Robards and Peter Sunehag",

year = "2012",

doi = "10.1007/978-3-642-29946-9_7",

language = "English",

isbn = "9783642299452",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "30--41",

booktitle = "Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Revised Selected Papers",

note = "9th European Workshop on Reinforcement Learning, EWRL 2011 ; Conference date: 09-09-2011 Through 11-09-2011",

}

Robards, M & Sunehag, P 2012, Gradient based algorithms with loss functions and kernels for improved on-policy control. in Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7188 LNAI, pp. 30-41, 9th European Workshop on Reinforcement Learning, EWRL 2011, Athens, Greece, 9/09/11. https://doi.org/10.1007/978-3-642-29946-9_7

Gradient based algorithms with loss functions and kernels for improved on-policy control. / Robards, Matthew; Sunehag, Peter.
Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Revised Selected Papers. 2012. p. 30-41 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7188 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Gradient based algorithms with loss functions and kernels for improved on-policy control

AU - Robards, Matthew

AU - Sunehag, Peter

PY - 2012

Y1 - 2012

N2 - We introduce and empirically evaluate two novel online gradient-based reinforcement learning algorithms with function approximation - one model based, and the other model free. These algorithms come with the possibility of having non-squared loss functions which is novel in reinforcement learning, and seems to come with empirical advantages. We further extend a previous gradient based algorithm to the case of full control, by using generalized policy iteration. Theoretical properties of these algorithms are studied in a companion paper.

AB - We introduce and empirically evaluate two novel online gradient-based reinforcement learning algorithms with function approximation - one model based, and the other model free. These algorithms come with the possibility of having non-squared loss functions which is novel in reinforcement learning, and seems to come with empirical advantages. We further extend a previous gradient based algorithm to the case of full control, by using generalized policy iteration. Theoretical properties of these algorithms are studied in a companion paper.

UR - http://www.scopus.com/inward/record.url?scp=84861701646&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-29946-9_7

DO - 10.1007/978-3-642-29946-9_7

M3 - Conference contribution

AN - SCOPUS:84861701646

SN - 9783642299452

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 30

EP - 41

BT - Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Revised Selected Papers

T2 - 9th European Workshop on Reinforcement Learning, EWRL 2011

Y2 - 9 September 2011 through 11 September 2011

ER -

Robards M, Sunehag P. Gradient based algorithms with loss functions and kernels for improved on-policy control. In Recent Advances in Reinforcement Learning - 9th European Workshop, EWRL 2011, Revised Selected Papers. 2012. p. 30-41. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-29946-9_7

Gradient based algorithms with loss functions and kernels for improved on-policy control

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this