Mixability in statistical learning

Tim Van Erven, Peter D. Grünwald, Mark D. Reid, Robert C. Williamson

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    8 Citations (Scopus)

    Abstract

    Statistical learning and sequential prediction are two different but related formalisms to study the quality of predictions. Mapping out their relations and transferring ideas is an active area of investigation. We provide another piece of the puzzle by showing that an important concept in sequential prediction, the mixability of a loss, has a natural counterpart in the statistical setting, which we call stochastic mixability. Just as ordinary mixability characterizes fast rates for the worst-case regret in sequential prediction, stochastic mixability characterizes fast rates in statistical learning. We show that, in the special case of log-loss, stochastic mixability reduces to a well-known (but usually unnamed) martingale condition, which is used in existing convergence theorems for minimum description length and Bayesian inference. In the case of 0/1-loss, it reduces to the margin condition of Mammen and Tsybakov, and in the case that the model under consideration contains all possible predictors, it is equivalent to ordinary mixability.

    Original languageEnglish
    Title of host publicationAdvances in Neural Information Processing Systems 25
    Subtitle of host publication26th Annual Conference on Neural Information Processing Systems 2012, NIPS 2012
    Pages1691-1699
    Number of pages9
    Publication statusPublished - 2012
    Event26th Annual Conference on Neural Information Processing Systems 2012, NIPS 2012 - Lake Tahoe, NV, United States
    Duration: 3 Dec 20126 Dec 2012

    Publication series

    NameAdvances in Neural Information Processing Systems
    Volume3
    ISSN (Print)1049-5258

    Conference

    Conference26th Annual Conference on Neural Information Processing Systems 2012, NIPS 2012
    Country/TerritoryUnited States
    CityLake Tahoe, NV
    Period3/12/126/12/12

    Fingerprint

    Dive into the research topics of 'Mixability in statistical learning'. Together they form a unique fingerprint.

    Cite this