MaxEnt Upper Bounds for the Differential Entropy of Univariate Continuous Distributions

Frank Nielsen; Richard Nock

doi:10.1109/LSP.2017.2666792

MaxEnt Upper Bounds for the Differential Entropy of Univariate Continuous Distributions

Frank Nielsen^*, Richard Nock

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

11 Citations (Scopus)

Abstract

We present a series of closed-form upper bounds of the differential entropy of univariate continuous distributions based on the maximum entropy principle. We apply those bounds to Gaussian mixture models, and study their tightness properties.

Original language	English
Article number	7849175
Pages (from-to)	402-406
Number of pages	5
Journal	IEEE Signal Processing Letters
Volume	24
Issue number	4
DOIs	https://doi.org/10.1109/LSP.2017.2666792
Publication status	Published - Apr 2017

Access to Document

10.1109/LSP.2017.2666792

Cite this

@article{e0c382c049534d0a88fec95bf681c480,

title = "MaxEnt Upper Bounds for the Differential Entropy of Univariate Continuous Distributions",

abstract = "We present a series of closed-form upper bounds of the differential entropy of univariate continuous distributions based on the maximum entropy principle. We apply those bounds to Gaussian mixture models, and study their tightness properties.",

keywords = "Absolute geometric moment, absolute monomial exponential family (EF), differential entropy, maximum entropy principle, mixture model",

author = "Frank Nielsen and Richard Nock",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.",

year = "2017",

month = apr,

doi = "10.1109/LSP.2017.2666792",

language = "English",

volume = "24",

pages = "402--406",

journal = "IEEE Signal Processing Letters",

issn = "1070-9908",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "4",

}

TY - JOUR

T1 - MaxEnt Upper Bounds for the Differential Entropy of Univariate Continuous Distributions

AU - Nielsen, Frank

AU - Nock, Richard

PY - 2017/4

Y1 - 2017/4

N2 - We present a series of closed-form upper bounds of the differential entropy of univariate continuous distributions based on the maximum entropy principle. We apply those bounds to Gaussian mixture models, and study their tightness properties.

AB - We present a series of closed-form upper bounds of the differential entropy of univariate continuous distributions based on the maximum entropy principle. We apply those bounds to Gaussian mixture models, and study their tightness properties.

KW - Absolute geometric moment

KW - absolute monomial exponential family (EF)

KW - differential entropy

KW - maximum entropy principle

KW - mixture model

UR - https://www.scopus.com/pages/publications/85017509578

U2 - 10.1109/LSP.2017.2666792

DO - 10.1109/LSP.2017.2666792

M3 - Article

SN - 1070-9908

VL - 24

SP - 402

EP - 406

JO - IEEE Signal Processing Letters

JF - IEEE Signal Processing Letters

IS - 4

M1 - 7849175

ER -

MaxEnt Upper Bounds for the Differential Entropy of Univariate Continuous Distributions

Abstract

Access to Document

Other files and links

Fingerprint

Cite this