Tighter variational representations of f-divergences via restriction to probability measures

Avraham Ruderman; Mark D. Reid; Dario García-García; James Petterson

Tighter variational representations of f-divergences via restriction to probability measures

Avraham Ruderman^*, Mark D. Reid, Dario García-García, James Petterson

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › peer-review

23 Citations (SciVal)

Abstract

We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on two i.i.d. samples and derive the dual program for this estimator that performs well empirically. We also point out a connection between our estimator and MMD.

Original language	English
Title of host publication	Proceedings of the 29th International Conference on Machine Learning, ICML 2012
Pages	671-678
Number of pages	8
Publication status	Published - 2012
Event	29th International Conference on Machine Learning, ICML 2012 - Edinburgh, United Kingdom Duration: 26 Jun 2012 → 1 Jul 2012

Publication series

Name	Proceedings of the 29th International Conference on Machine Learning, ICML 2012
Volume	1

Conference

Conference	29th International Conference on Machine Learning, ICML 2012
Country/Territory	United Kingdom
City	Edinburgh
Period	26/06/12 → 1/07/12

Cite this

@inproceedings{01757f71191e47b6bd2e2431eb26b577,

title = "Tighter variational representations of f-divergences via restriction to probability measures",

abstract = "We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on two i.i.d. samples and derive the dual program for this estimator that performs well empirically. We also point out a connection between our estimator and MMD.",

author = "Avraham Ruderman and Reid, \{Mark D.\} and Dario Garc{\'i}a-Garc{\'i}a and James Petterson",

year = "2012",

language = "English",

isbn = "9781450312851",

series = "Proceedings of the 29th International Conference on Machine Learning, ICML 2012",

pages = "671--678",

booktitle = "Proceedings of the 29th International Conference on Machine Learning, ICML 2012",

note = "29th International Conference on Machine Learning, ICML 2012 ; Conference date: 26-06-2012 Through 01-07-2012",

}

Ruderman, A, Reid, MD, García-García, D & Petterson, J 2012, Tighter variational representations of f-divergences via restriction to probability measures. in Proceedings of the 29th International Conference on Machine Learning, ICML 2012. Proceedings of the 29th International Conference on Machine Learning, ICML 2012, vol. 1, pp. 671-678, 29th International Conference on Machine Learning, ICML 2012, Edinburgh, United Kingdom, 26/06/12.

Tighter variational representations of f-divergences via restriction to probability measures. / Ruderman, Avraham; Reid, Mark D.; García-García, Dario et al.
Proceedings of the 29th International Conference on Machine Learning, ICML 2012. 2012. p. 671-678 (Proceedings of the 29th International Conference on Machine Learning, ICML 2012; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › peer-review

TY - GEN

T1 - Tighter variational representations of f-divergences via restriction to probability measures

AU - Ruderman, Avraham

AU - Reid, Mark D.

AU - García-García, Dario

AU - Petterson, James

PY - 2012

Y1 - 2012

N2 - We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on two i.i.d. samples and derive the dual program for this estimator that performs well empirically. We also point out a connection between our estimator and MMD.

AB - We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on two i.i.d. samples and derive the dual program for this estimator that performs well empirically. We also point out a connection between our estimator and MMD.

UR - http://www.scopus.com/inward/record.url?scp=84867136405&partnerID=8YFLogxK

M3 - Conference Paper

SN - 9781450312851

T3 - Proceedings of the 29th International Conference on Machine Learning, ICML 2012

SP - 671

EP - 678

BT - Proceedings of the 29th International Conference on Machine Learning, ICML 2012

T2 - 29th International Conference on Machine Learning, ICML 2012

Y2 - 26 June 2012 through 1 July 2012

ER -

Tighter variational representations of f-divergences via restriction to probability measures

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this