An online and approximate solver for POMDPs with continuous action space

Konstantin M. Seiler; Hanna Kurniawati; Surya P. N. Singh

doi:10.1109/ICRA.2015.7139503

An online and approximate solver for POMDPs with continuous action space

Konstantin M. Seiler, Hanna Kurniawati, Surya P. N. Singh

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › peer-review

54 Citations (Scopus)

Abstract

For agile, accurate autonomous robotics, it is desirable to plan motion in the presence of uncertainty. The Partially Observable Markov Decision Process (POMDP) provides a principled framework for this. Despite the tremendous advances of POMDP-based planning, most can only solve problems with a small and discrete set of actions. This paper presents General Pattern Search in Adaptive Belief Tree (GPS-ABT), an approximate and online POMDP solver for problems with continuous action spaces. Generalized Pattern Search (GPS) is used as a search strategy for action selection. Under certain conditions, GPS-ABT converges to the optimal solution in probability. Results on a box pushing and an extended Tag benchmark problem are promising.

Original language	English
Title of host publication	IEEE International Conference on Robotics and Automation, ICRA 2015, Seattle, WA, USA, 26-30 May, 2015
Publisher	IEEE
Pages	2290-2297
Number of pages	8
DOIs	https://doi.org/10.1109/ICRA.2015.7139503
Publication status	Published - 2015
Externally published	Yes

Access to Document

10.1109/ICRA.2015.7139503

https://doi.org/10.1109/ICRA.2015.7139503

Cite this

@inproceedings{55059b03c8824a44b2557530ed2cc2c9,

title = "An online and approximate solver for POMDPs with continuous action space",

abstract = "For agile, accurate autonomous robotics, it is desirable to plan motion in the presence of uncertainty. The Partially Observable Markov Decision Process (POMDP) provides a principled framework for this. Despite the tremendous advances of POMDP-based planning, most can only solve problems with a small and discrete set of actions. This paper presents General Pattern Search in Adaptive Belief Tree (GPS-ABT), an approximate and online POMDP solver for problems with continuous action spaces. Generalized Pattern Search (GPS) is used as a search strategy for action selection. Under certain conditions, GPS-ABT converges to the optimal solution in probability. Results on a box pushing and an extended Tag benchmark problem are promising.",

author = "Seiler, \{Konstantin M.\} and Hanna Kurniawati and Singh, \{Surya P. N.\}",

year = "2015",

doi = "10.1109/ICRA.2015.7139503",

language = "English",

pages = "2290--2297",

booktitle = "IEEE International Conference on Robotics and Automation, ICRA 2015, Seattle, WA, USA, 26-30 May, 2015",

publisher = "IEEE",

}

TY - GEN

T1 - An online and approximate solver for POMDPs with continuous action space

AU - Seiler, Konstantin M.

AU - Kurniawati, Hanna

AU - Singh, Surya P. N.

PY - 2015

Y1 - 2015

N2 - For agile, accurate autonomous robotics, it is desirable to plan motion in the presence of uncertainty. The Partially Observable Markov Decision Process (POMDP) provides a principled framework for this. Despite the tremendous advances of POMDP-based planning, most can only solve problems with a small and discrete set of actions. This paper presents General Pattern Search in Adaptive Belief Tree (GPS-ABT), an approximate and online POMDP solver for problems with continuous action spaces. Generalized Pattern Search (GPS) is used as a search strategy for action selection. Under certain conditions, GPS-ABT converges to the optimal solution in probability. Results on a box pushing and an extended Tag benchmark problem are promising.

AB - For agile, accurate autonomous robotics, it is desirable to plan motion in the presence of uncertainty. The Partially Observable Markov Decision Process (POMDP) provides a principled framework for this. Despite the tremendous advances of POMDP-based planning, most can only solve problems with a small and discrete set of actions. This paper presents General Pattern Search in Adaptive Belief Tree (GPS-ABT), an approximate and online POMDP solver for problems with continuous action spaces. Generalized Pattern Search (GPS) is used as a search strategy for action selection. Under certain conditions, GPS-ABT converges to the optimal solution in probability. Results on a box pushing and an extended Tag benchmark problem are promising.

U2 - 10.1109/ICRA.2015.7139503

DO - 10.1109/ICRA.2015.7139503

M3 - Conference Paper

SP - 2290

EP - 2297

BT - IEEE International Conference on Robotics and Automation, ICRA 2015, Seattle, WA, USA, 26-30 May, 2015

PB - IEEE

ER -

An online and approximate solver for POMDPs with continuous action space

Abstract

Access to Document

Fingerprint

Cite this