Partially Observable Markov Decision Processes and Robotics

Hanna Kurniawati*

*Corresponding author for this work

    Research output: Contribution to journalReview articlepeer-review

    43 Citations (Scopus)


    Planning under uncertainty is critical to robotics. The partially observable Markov decision process (POMDP) is a mathematical framework for such planning problems. POMDPs are powerful because of their careful quantification of the nondeterministic effects of actions and the partial observability of the states. But for the same reason, they are notorious for their high computational complexity and have been deemed impractical for robotics. However, over the past two decades, the development of sampling-based approximate solvers has led to tremendous advances in POMDP-solving capabilities. Although these solvers do not generate the optimal solution, they can compute good POMDP solutions that significantly improve the robustness of robotics systems within reasonable computational resources, thereby making POMDPs practical for many realistic robotics problems. This article presents a review of POMDPs, emphasizing computational issues that have hindered their practicality in robotics and ideas in sampling-based solvers that have alleviated such difficulties, together with lessons learned from applying POMDPs to physical robots.

    Original languageEnglish
    Pages (from-to)253-277
    Number of pages25
    JournalAnnual Review of Control, Robotics, and Autonomous Systems
    Publication statusPublished - 2022


    Dive into the research topics of 'Partially Observable Markov Decision Processes and Robotics'. Together they form a unique fingerprint.

    Cite this