An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach

Erli Wang; Hanna Kurniawati; Dirk P. Kroese

An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach

Erli Wang, Hanna Kurniawati, Dirk P. Kroese

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › peer-review

9 Citations (Scopus)

Abstract

Making principled decisions in the presence of uncertainty
is often facilitated by Partially Observable Markov Decision Processes (POMDPs). Despite tremendous advances
in POMDP solvers, finding good policies with large action
spaces remains difficult. To alleviate this difficulty, this paper presents an on-line approximate solver, called QuantileBased Action Selector (QBASE). It uses quantile-statistics to
adaptively evaluate a small subset of the action space without
sacrificing the quality of the generated decision strategies by
much. Experiments on four different robotics tasks with up
to 10,000 actions indicate that QBASE can generate substantially better strategies than a state-of-the-art method.

Original language	English
Title of host publication	Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, ICAPS 2018, Delft, The Netherlands, June 24-29, 2018
Editors	Mathijs de Weerdt, Sven Koenig, Gabriele Röger, Matthijs T. J. Spaan
Publisher	AAAI Press
Pages	273-277
Number of pages	5
Publication status	Published - 2018
Externally published	Yes

Access to Document

https://aaai.org/ocs/index.php/ICAPS/ICAPS18/paper/view/17774

Cite this

Wang, E., Kurniawati, H., & Kroese, D. P. (2018). An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach. In M. D. Weerdt, S. Koenig, G. Röger, & M. T. J. Spaan (Eds.), Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, ICAPS 2018, Delft, The Netherlands, June 24-29, 2018 (pp. 273-277). AAAI Press. https://aaai.org/ocs/index.php/ICAPS/ICAPS18/paper/view/17774

Wang, Erli ; Kurniawati, Hanna ; Kroese, Dirk P. / An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach. Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, ICAPS 2018, Delft, The Netherlands, June 24-29, 2018. editor / Mathijs de Weerdt ; Sven Koenig ; Gabriele Röger ; Matthijs T. J. Spaan. AAAI Press, 2018. pp. 273-277

@inproceedings{81a1836353d84c608459a04e30eb2fb8,

title = "An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach",

abstract = "Making principled decisions in the presence of uncertainty is often facilitated by Partially Observable Markov Decision Processes (POMDPs). Despite tremendous advances in POMDP solvers, finding good policies with large action spaces remains difficult. To alleviate this difficulty, this paper presents an on-line approximate solver, called QuantileBased Action Selector (QBASE). It uses quantile-statistics to adaptively evaluate a small subset of the action space without sacrificing the quality of the generated decision strategies by much. Experiments on four different robotics tasks with up to 10,000 actions indicate that QBASE can generate substantially better strategies than a state-of-the-art method.",

author = "Erli Wang and Hanna Kurniawati and Kroese, \{Dirk P.\}",

year = "2018",

language = "English",

pages = "273--277",

editor = "Weerdt, \{Mathijs de\} and Sven Koenig and Gabriele R{\"o}ger and Spaan, \{Matthijs T. J.\}",

booktitle = "Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, ICAPS 2018, Delft, The Netherlands, June 24-29, 2018",

publisher = "AAAI Press",

}

Wang, E, Kurniawati, H & Kroese, DP 2018, An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach. in MD Weerdt, S Koenig, G Röger & MTJ Spaan (eds), Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, ICAPS 2018, Delft, The Netherlands, June 24-29, 2018. AAAI Press, pp. 273-277. <https://aaai.org/ocs/index.php/ICAPS/ICAPS18/paper/view/17774>

An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach. / Wang, Erli; Kurniawati, Hanna; Kroese, Dirk P.
Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, ICAPS 2018, Delft, The Netherlands, June 24-29, 2018. ed. / Mathijs de Weerdt; Sven Koenig; Gabriele Röger; Matthijs T. J. Spaan. AAAI Press, 2018. p. 273-277.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › peer-review

TY - GEN

T1 - An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach

AU - Wang, Erli

AU - Kurniawati, Hanna

AU - Kroese, Dirk P.

PY - 2018

Y1 - 2018

N2 - Making principled decisions in the presence of uncertainty is often facilitated by Partially Observable Markov Decision Processes (POMDPs). Despite tremendous advances in POMDP solvers, finding good policies with large action spaces remains difficult. To alleviate this difficulty, this paper presents an on-line approximate solver, called QuantileBased Action Selector (QBASE). It uses quantile-statistics to adaptively evaluate a small subset of the action space without sacrificing the quality of the generated decision strategies by much. Experiments on four different robotics tasks with up to 10,000 actions indicate that QBASE can generate substantially better strategies than a state-of-the-art method.

AB - Making principled decisions in the presence of uncertainty is often facilitated by Partially Observable Markov Decision Processes (POMDPs). Despite tremendous advances in POMDP solvers, finding good policies with large action spaces remains difficult. To alleviate this difficulty, this paper presents an on-line approximate solver, called QuantileBased Action Selector (QBASE). It uses quantile-statistics to adaptively evaluate a small subset of the action space without sacrificing the quality of the generated decision strategies by much. Experiments on four different robotics tasks with up to 10,000 actions indicate that QBASE can generate substantially better strategies than a state-of-the-art method.

M3 - Conference Paper

SP - 273

EP - 277

BT - Proceedings of the Twenty-Eighth International Conference on Automated Planning and Scheduling, ICAPS 2018, Delft, The Netherlands, June 24-29, 2018

A2 - Weerdt, Mathijs de

A2 - Koenig, Sven

A2 - Röger, Gabriele

A2 - Spaan, Matthijs T. J.

PB - AAAI Press

ER -

An On-Line Planner for POMDPs with Large Discrete Action Space: A Quantile-Based Approach

Abstract

Access to Document

Fingerprint

Cite this