Survival-oriented reinforcement learning model: An effcient and robust deep reinforcement learning algorithm for autonomous driving problem

Changkun Ye; Huimin Ma; Xiaoqin Zhang; Kai Zhang; Shaodi You

doi:10.1007/978-3-319-71589-6_36

Survival-oriented reinforcement learning model: An effcient and robust deep reinforcement learning algorithm for autonomous driving problem

Changkun Ye, Huimin Ma^*, Xiaoqin Zhang, Kai Zhang, Shaodi You

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Citations (Scopus)

Abstract

Using Deep Reinforcement Learning (DRL) algorithm to deal with autonomous driving tasks usually have unsatisfied performance due to lack of robustness and means to escape local optimum. In this article, we designs a Survival-Oriented Reinforcement Learning (SORL) model that tackle these problems by setting survival rather than maximize total reward as first priority. In SORL model, we model autonomous driving task as Constrained Markov Decision Process (CMDP) and introduce Negative-Avoidance Function to learn from previous failure. The SORL model greatly speed up the training process and improve the robustness of normal Deep Reinforcement Learning algorithm.

Original language	English
Title of host publication	Image and Graphics - 9th International Conference, ICIG 2017, Revised Selected Papers
Editors	Xiangwei Kong, Yao Zhao, David Taubman
Publisher	Springer Verlag
Pages	417-429
Number of pages	13
ISBN (Print)	9783319715889
DOIs	https://doi.org/10.1007/978-3-319-71589-6_36
Publication status	Published - 2017
Event	9th International Conference on Image and Graphics, ICIG 2017 - Shanghai, China Duration: 13 Sept 2017 → 15 Sept 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10667 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Conference on Image and Graphics, ICIG 2017
Country/Territory	China
City	Shanghai
Period	13/09/17 → 15/09/17

Access to Document

10.1007/978-3-319-71589-6_36

Cite this

Ye, C., Ma, H., Zhang, X., Zhang, K., & You, S. (2017). Survival-oriented reinforcement learning model: An effcient and robust deep reinforcement learning algorithm for autonomous driving problem. In X. Kong, Y. Zhao, & D. Taubman (Eds.), Image and Graphics - 9th International Conference, ICIG 2017, Revised Selected Papers (pp. 417-429). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10667 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-71589-6_36

Ye, Changkun ; Ma, Huimin ; Zhang, Xiaoqin et al. / Survival-oriented reinforcement learning model : An effcient and robust deep reinforcement learning algorithm for autonomous driving problem. Image and Graphics - 9th International Conference, ICIG 2017, Revised Selected Papers. editor / Xiangwei Kong ; Yao Zhao ; David Taubman. Springer Verlag, 2017. pp. 417-429 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{9206a2a23cff42ebb23dfd39937e425b,

title = "Survival-oriented reinforcement learning model: An effcient and robust deep reinforcement learning algorithm for autonomous driving problem",

abstract = "Using Deep Reinforcement Learning (DRL) algorithm to deal with autonomous driving tasks usually have unsatisfied performance due to lack of robustness and means to escape local optimum. In this article, we designs a Survival-Oriented Reinforcement Learning (SORL) model that tackle these problems by setting survival rather than maximize total reward as first priority. In SORL model, we model autonomous driving task as Constrained Markov Decision Process (CMDP) and introduce Negative-Avoidance Function to learn from previous failure. The SORL model greatly speed up the training process and improve the robustness of normal Deep Reinforcement Learning algorithm.",

keywords = "Local optimum, Negative-avoidance, Reinforcement learning, Robustness",

author = "Changkun Ye and Huimin Ma and Xiaoqin Zhang and Kai Zhang and Shaodi You",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing AG 2017.; 9th International Conference on Image and Graphics, ICIG 2017 ; Conference date: 13-09-2017 Through 15-09-2017",

year = "2017",

doi = "10.1007/978-3-319-71589-6_36",

language = "English",

isbn = "9783319715889",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "417--429",

editor = "Xiangwei Kong and Yao Zhao and David Taubman",

booktitle = "Image and Graphics - 9th International Conference, ICIG 2017, Revised Selected Papers",

address = "Germany",

}

Ye, C, Ma, H, Zhang, X, Zhang, K & You, S 2017, Survival-oriented reinforcement learning model: An effcient and robust deep reinforcement learning algorithm for autonomous driving problem. in X Kong, Y Zhao & D Taubman (eds), Image and Graphics - 9th International Conference, ICIG 2017, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10667 LNCS, Springer Verlag, pp. 417-429, 9th International Conference on Image and Graphics, ICIG 2017, Shanghai, China, 13/09/17. https://doi.org/10.1007/978-3-319-71589-6_36

Survival-oriented reinforcement learning model: An effcient and robust deep reinforcement learning algorithm for autonomous driving problem. / Ye, Changkun; Ma, Huimin; Zhang, Xiaoqin et al.
Image and Graphics - 9th International Conference, ICIG 2017, Revised Selected Papers. ed. / Xiangwei Kong; Yao Zhao; David Taubman. Springer Verlag, 2017. p. 417-429 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10667 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Survival-oriented reinforcement learning model

T2 - 9th International Conference on Image and Graphics, ICIG 2017

AU - Ye, Changkun

AU - Ma, Huimin

AU - Zhang, Xiaoqin

AU - Zhang, Kai

AU - You, Shaodi

N1 - Publisher Copyright: © Springer International Publishing AG 2017.

PY - 2017

Y1 - 2017

N2 - Using Deep Reinforcement Learning (DRL) algorithm to deal with autonomous driving tasks usually have unsatisfied performance due to lack of robustness and means to escape local optimum. In this article, we designs a Survival-Oriented Reinforcement Learning (SORL) model that tackle these problems by setting survival rather than maximize total reward as first priority. In SORL model, we model autonomous driving task as Constrained Markov Decision Process (CMDP) and introduce Negative-Avoidance Function to learn from previous failure. The SORL model greatly speed up the training process and improve the robustness of normal Deep Reinforcement Learning algorithm.

AB - Using Deep Reinforcement Learning (DRL) algorithm to deal with autonomous driving tasks usually have unsatisfied performance due to lack of robustness and means to escape local optimum. In this article, we designs a Survival-Oriented Reinforcement Learning (SORL) model that tackle these problems by setting survival rather than maximize total reward as first priority. In SORL model, we model autonomous driving task as Constrained Markov Decision Process (CMDP) and introduce Negative-Avoidance Function to learn from previous failure. The SORL model greatly speed up the training process and improve the robustness of normal Deep Reinforcement Learning algorithm.

KW - Local optimum

KW - Negative-avoidance

KW - Reinforcement learning

KW - Robustness

UR - http://www.scopus.com/inward/record.url?scp=85041809553&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-71589-6_36

DO - 10.1007/978-3-319-71589-6_36

M3 - Conference contribution

SN - 9783319715889

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 417

EP - 429

BT - Image and Graphics - 9th International Conference, ICIG 2017, Revised Selected Papers

A2 - Kong, Xiangwei

A2 - Zhao, Yao

A2 - Taubman, David

PB - Springer Verlag

Y2 - 13 September 2017 through 15 September 2017

ER -

Ye C, Ma H, Zhang X, Zhang K, You S. Survival-oriented reinforcement learning model: An effcient and robust deep reinforcement learning algorithm for autonomous driving problem. In Kong X, Zhao Y, Taubman D, editors, Image and Graphics - 9th International Conference, ICIG 2017, Revised Selected Papers. Springer Verlag. 2017. p. 417-429. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-71589-6_36

Survival-oriented reinforcement learning model: An effcient and robust deep reinforcement learning algorithm for autonomous driving problem

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this