Fitted value function iteration with probability one contractions

Jen Pál; John Stachurski

doi:10.1016/j.jedc.2012.08.003

Fitted value function iteration with probability one contractions

Jen Pál, John Stachurski^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

5 Citations (Scopus)

Abstract

This paper studies a value function iteration algorithm based on nonexpansive function approximation and Monte Carlo integration that can be applied to almost all stationary dynamic programming problems. The method can be represented using a randomized fitted Bellman operator and a corresponding algorithm that is shown to be globally convergent with probability one. When additional restrictions are imposed, an O _P(n ^-1/2) rate of convergence for Monte Carlo error is obtained.

Original language	English
Pages (from-to)	251-264
Number of pages	14
Journal	Journal of Economic Dynamics and Control
Volume	37
Issue number	1
DOIs	https://doi.org/10.1016/j.jedc.2012.08.003
Publication status	Published - Jan 2013

Access to Document

10.1016/j.jedc.2012.08.003

Cite this

@article{871b59f08a8543fbbaf11399d2c235b1,

title = "Fitted value function iteration with probability one contractions",

abstract = "This paper studies a value function iteration algorithm based on nonexpansive function approximation and Monte Carlo integration that can be applied to almost all stationary dynamic programming problems. The method can be represented using a randomized fitted Bellman operator and a corresponding algorithm that is shown to be globally convergent with probability one. When additional restrictions are imposed, an O P(n -1/2) rate of convergence for Monte Carlo error is obtained.",

keywords = "C61, C63, Dynamic programming, Monte Carlo, Value function iteration",

author = "Jen P{\'a}l and John Stachurski",

year = "2013",

month = jan,

doi = "10.1016/j.jedc.2012.08.003",

language = "English",

volume = "37",

pages = "251--264",

journal = "Journal of Economic Dynamics and Control",

issn = "0165-1889",

publisher = "Elsevier B.V.",

number = "1",

}

TY - JOUR

T1 - Fitted value function iteration with probability one contractions

AU - Pál, Jen

AU - Stachurski, John

PY - 2013/1

Y1 - 2013/1

N2 - This paper studies a value function iteration algorithm based on nonexpansive function approximation and Monte Carlo integration that can be applied to almost all stationary dynamic programming problems. The method can be represented using a randomized fitted Bellman operator and a corresponding algorithm that is shown to be globally convergent with probability one. When additional restrictions are imposed, an O P(n -1/2) rate of convergence for Monte Carlo error is obtained.

AB - This paper studies a value function iteration algorithm based on nonexpansive function approximation and Monte Carlo integration that can be applied to almost all stationary dynamic programming problems. The method can be represented using a randomized fitted Bellman operator and a corresponding algorithm that is shown to be globally convergent with probability one. When additional restrictions are imposed, an O P(n -1/2) rate of convergence for Monte Carlo error is obtained.

KW - C61

KW - C63

KW - Dynamic programming

KW - Monte Carlo

KW - Value function iteration

UR - https://www.scopus.com/pages/publications/84868342069

U2 - 10.1016/j.jedc.2012.08.003

DO - 10.1016/j.jedc.2012.08.003

M3 - Article

SN - 0165-1889

VL - 37

SP - 251

EP - 264

JO - Journal of Economic Dynamics and Control

JF - Journal of Economic Dynamics and Control

IS - 1

ER -

Fitted value function iteration with probability one contractions

Abstract

Access to Document

Other files and links

Fingerprint

Cite this