Gaussian distribution maximizes entropy for given covariance

Theorem 1

Let $f\mathrm{:}{\mathrm{R}}^n\mathrm{\to }\mathrm{R}$ be a continuous probability density function. Let $X_{\mathrm{1}}\mathrm{,}\mathrm{\dots }\mathrm{,}{X}_n$ be random variables with density $f$ and with covariance matrix $\mathrm{K}$, $K_{i\mathit{}j}\mathrm{=}\mathrm{cov}\mathit{}\mathrm{(}{X}_i\mathrm{,}{X}_j\mathrm{)}$. Let $\varphi$ be the distribution of the multidimensional Gaussian (http://planetmath.org/JointNormalDistribution) with mean $\mathrm{0}$ and covariance matrix $\mathrm{K}$. Then the Gaussian distribution maximizes the differential entropy for a given covariance matrix $\mathrm{K}$. That is, $h\mathit{}\mathrm{(}\varphi \mathrm{)}\mathrm{\ge }h\mathit{}\mathrm{(}f\mathrm{)}$.

Title	Gaussian distribution maximizes entropy for given covariance
Canonical name	GaussianDistributionMaximizesEntropyForGivenCovariance
Date of creation	2013-03-22 12:19:26
Last modified on	2013-03-22 12:19:26
Owner	Mathprof (13753)
Last modified by	Mathprof (13753)
Numerical id	10
Author	Mathprof (13753)
Entry type	Theorem
Classification	msc 94A17