Conditional random fields for multi-agent reinforcement learning

Xinhua Zhang*, Douglas Aberdeen, S. V.N. Vishwanathan

*Corresponding author for this work

    Research output: Contribution to conferencePaperpeer-review

    12 Citations (Scopus)

    Abstract

    Conditional random fields (CRFs) are graphical models for modeling the probability of labels given the observations. They have traditionally been trained with using a set of observation and label pairs. Underlying all CRFs is the assumption that, conditioned on the training data, the labels are independent and identically distributed (iid). In this paper we explore the use of CRFs in a class of temporal learning algorithms, namely policy-gradient reinforcement learning (RL). Now the labels are no longer iid. They are actions that update the environment and affect the next observation. From an RL point of view, CRFs provide a natural way to model joint actions in a decentralized Markov decision process. They define how agents can communicate with each other to choose the optimal joint action. Our experiments include a synthetic network alignment problem, a distributed sensor network, and road traffic control; clearly outperforming RL methods which do not model the proper joint policy.

    Original languageEnglish
    Pages1143-1150
    Number of pages8
    DOIs
    Publication statusPublished - 2007
    Event24th International Conference on Machine Learning, ICML 2007 - Corvalis, OR, United States
    Duration: 20 Jun 200724 Jun 2007

    Conference

    Conference24th International Conference on Machine Learning, ICML 2007
    Country/TerritoryUnited States
    CityCorvalis, OR
    Period20/06/0724/06/07

    Fingerprint

    Dive into the research topics of 'Conditional random fields for multi-agent reinforcement learning'. Together they form a unique fingerprint.

    Cite this