PTAH inequality

Let $\sigma =\{({\theta }_1,\mathrm{\dots },{\theta }_n)\in {ℝ}^n|{\theta }_i\ge 0,{\sum }_{i=1}^n{\theta }_i=1\}$.

Let $X$ be a measure space with measure $m$. Let $a_i:X\to ℝ$ be measurable functions such that $a_i(x)\ge 0$ a.e [m] for $i=1,\mathrm{\dots },n$.

Note: the notation “a.e. [m]” means that the condition holds almost everywhere with respect to the measure $m$.

Define $p:X\times \sigma \to ℝ$ by

$$p(x,\lambda )=\prod _{i=1}^n\lambda _i{}^{a_i(x)}$$

where $\lambda =({\lambda }_1,\mathrm{\dots },{\lambda }_n)$.

And define $P:\sigma \to ℝ$ and $Q:\sigma \times \sigma \to ℝ$ by

$$P(\lambda )=\int p(x,\lambda )𝑑m(x)$$

and

$$Q(\lambda ,{\lambda }^{\prime })=\int p(x,\lambda )\log p(x,{\lambda }^{\prime })𝑑m(x).$$

Define $\overline{{\lambda }_i}$ by

$$\overline{{\lambda }_i}=\frac{{\lambda }_i\partial P/\partial {\lambda }_i}{{\sum }_j{\lambda }_j\partial P/\partial {\lambda }_j}.$$

Theorem. Let $\lambda \in \sigma$ and $\overline{\lambda }$ be defined as above. Then for every ${\lambda }^{\prime }\in \sigma$ we have

$$Q(\lambda ,{\lambda }^{\prime })\le Q(\lambda ,\overline{\lambda })$$

with strict inequality unless ${\lambda }^{\prime }=\overline{\lambda }$. Also,

$$P(\lambda )\le P(\overline{\lambda })$$

with strict inequality unless $\lambda =\overline{\lambda }$. The second inequality is known as the PTAH inequality.

The significance of the PTAH inequality is that some of the classical inequalities are all special cases of PTAH.

Consider:

(A) The arithmetic-geometric mean inequality:

$$\prod x_i{}^{\frac{1}{n}}\le \sum \frac{x_i}{n}$$

(B) the concavity of $\log x$:

$$\sum {\theta }_i\log x_i\le \log \sum {\theta }_i{x}_i$$

(C) the Kullback-Leibler inequality:

$$\prod \theta _i{}^{r_i}\le \prod {(\frac{r_i}{\sum r_j})}^{r_i}$$

(D) the convexity of $x\log x$:

$$(\sum {\theta }_i{x}_i)\log (\sum {\theta }_i{x}_i)\le \sum {\theta }_i{x}_i\log x_i$$

(E)

$$Q(\lambda ,{\lambda }^{\prime })\le Q(\lambda ,\overline{\lambda })$$

(F)

$$Q(\lambda ,{\lambda }^{\prime })-Q(\lambda ,\lambda )\le P(\lambda )\log \frac{P({\lambda }^{\prime })}{P(\lambda )}$$

(G) the maximum-entropy inequality (in logarithmic form)

$$-\sum _{i=1}^n{p}_i\log p_i\le \log n$$

(H) Hölder’s generalized inequality (http://planetmath.org/GeneralizedHolderInequality)

$$\sum _{j=1}^n\prod _{i=1}^m{a}_{i,j}^{{\theta }_i}\le \prod _{i=1}^m{\left(\sum _{j=1}^n{a}_{i,j}\right)}^{{\theta }_i}$$

(P) The PTAH inequality:

$$P(\lambda )\le P(\overline{\lambda })$$

All the sums and products range from 1 to $n$, all the ${\theta }_i,x_i,r_i$ are positive and $({\theta }_i),\lambda ,{\lambda }^{\prime }$ are in $\sigma$ and the set $X$ is discrete, so that

$$P(\lambda )=\sum _{x}p(x,\lambda )m(x)$$

$$Q(\lambda ,{\lambda }^{\prime })=\sum _i{r}_i(\lambda )\log \lambda _i{}^{\prime }$$

where $m(x)>0$ $p(x,\lambda )={\prod }_i\lambda _i{}^{a_i(x)}$ and $a_i(x)\ge 0$, $r_i(\lambda )=\sum a_i(x)p(x,\lambda )m(x)$ and

$$\overline{{\lambda }_i}=\frac{{\lambda }_i\partial P/\partial {\lambda }_i}{\sum {\lambda }_j\partial P/\partial {\lambda }_j},$$

and $\overline{\lambda }=(\overline{{\lambda }_i})$. Then it turns out that (A) to (G) are all special cases of (H), and in fact (A) to (G) are all equivalent, in the sense that given any two of them, each is a special case of the other. (H) is a special case of (P), However, it appears that none of the reverse implications holds. According to George Soules:

”The folklore at the Institute for Defense Analyses in Princeton NJ is that the first program to maximize a function P(z) by iterating the growth transformation

$$z\to \overline{z}$$

was written while the programmer was listening to the opera Aida, in which the Egyptian god of creation Ptah is mentioned, and that became the name of the program (and of the inequality). The name is in upper case because the word processor in use in the middle 1960’s had no lower case.”

Title	PTAH inequality
Canonical name	PTAHInequality
Date of creation	2013-03-22 16:54:32
Last modified on	2013-03-22 16:54:32
Owner	Mathprof (13753)
Last modified by	Mathprof (13753)
Numerical id	27
Author	Mathprof (13753)
Entry type	Theorem
Classification	msc 26D15