PlanetMath: Gaussian distribution maximizes entropy for given covariance

(more info)

Math for the people, by the people.

Main Menu

sections Encyclopædia
Papers
Books
Expositions

meta Requests (210)
Orphanage
Unclass'd (1)
Unproven (473)
Corrections (37)
Classification

talkback Polls
Forums
Feedback
Bug Reports

downloads Snapshots
PM Book

information News
Docs
Wiki
ChangeLog
TODO List
Legalese
About

Entry average rating: No information on entry rating

Gaussian distribution maximizes entropy for given covariance

(Theorem)

Theorem 1 Let $f:\mathbb{R}^n \to \mathbb{R}$ be a continuous probability density function. Let $X_1, \ldots, X_n$ be random variables with density and with covariance matrix $\mathbf{K}$ , $K_{ij} = \mathrm{cov}(X_i, X_j)$ . Let $\phi$ be the distribution of the multidimensional Gaussian with mean $\mathbf{0}$ and covariance matrix $\mathbf{K}$ . Then the Gaussian distribution maximizes the differential entropy for a given covariance matrix $\mathbf{K}$ . That is, $h(\phi) \ge h(f)$ .

"Gaussian distribution maximizes entropy for given covariance" is owned by Mathprof. [ full author list (2) | owner history (1) ]

(view preamble)

Attachments:: proof of Gaussian maximizes entropy for given covariance (Proof) by Mathprof

Log in to rate this entry.
(view current ratings)

Cross-references: differential entropy, Gaussian, mean, covariance matrix, density, random variables, probability density function, continuous
There is 1 reference to this entry.

This is version 7 of Gaussian distribution maximizes entropy for given covariance, born on 2002-02-13, modified 2006-10-04.
Object id is 1942, canonical name is GaussianMaximizesEntropyForGivenCovariance.
Accessed 3997 times total.

Classification:

AMS MSC:

94A17 (Information and communication, circuits :: Communication, information :: Measures of information, entropy)

Pending Errata and Addenda

None.[ View all 3 ]

Discussion

forum policy

No messages.

Interact