PTAH inequality

Let $\sigma=\{(\theta_{1},\ldots,\theta_{n})\in{\mathbb{R}}^{n}|\theta_{i}\geq 0,% \sum_{i=1}^{n}\theta_{i}=1\}$.

Let $X$ be a measure space  with measure $m$. Let $a_{i}:X\to\mathbb{R}$ be measurable functions  such that $a_{i}(x)\geq 0$ a.e [m] for $i=1,\ldots,n$.

Note: the notation “a.e. [m]” means that the condition holds almost everywhere with respect to the measure $m$.

Define $p:X\times\sigma\to\mathbb{R}$ by

 $p(x,\lambda)=\prod_{i=1}^{n}{\lambda_{i}}^{a_{i}(x)}$

where $\lambda=(\lambda_{1},\ldots,\lambda_{n})$.

And define $P:\sigma\to\mathbb{R}$ and $Q:\sigma\times\sigma\to\mathbb{R}$ by

 $P(\lambda)=\int p(x,\lambda)dm(x)$

and

 $Q(\lambda,\lambda^{\prime})=\int p(x,\lambda)\log p(x,\lambda^{\prime})dm(x).$

Define $\overline{\lambda_{i}}$ by

 $\overline{\lambda_{i}}=\frac{\lambda_{i}\partial P/\partial\lambda_{i}}{\sum_{% j}\lambda_{j}\partial P/\partial\lambda_{j}}.$

Let $\lambda\in\sigma$ and $\overline{\lambda}$ be defined as above. Then for every $\lambda^{\prime}\in\sigma$ we have

 $Q(\lambda,\lambda^{\prime})\leq Q(\lambda,\overline{\lambda})$

with strict inequality  unless $\lambda^{\prime}=\overline{\lambda}$. Also,

 $P(\lambda)\leq P(\overline{\lambda})$

with strict inequality unless $\lambda=\overline{\lambda}$. The second inequality is known as the PTAH inequality.

The significance of the PTAH inequality is that some of the classical inequalities are all special cases of PTAH.

Consider:

 $\prod{x_{i}}^{\frac{1}{n}}\leq\sum\frac{x_{i}}{n}$

(B) the concavity of $\log x$:

 $\sum\theta_{i}\log x_{i}\leq\log\sum\theta_{i}x_{i}$

(C) the Kullback-Leibler inequality:

 $\prod{\theta_{i}}^{r_{i}}\leq\prod(\frac{r_{i}}{\sum r_{j}})^{r_{i}}$

(D) the convexity of $x\log x$:

 $(\sum\theta_{i}x_{i})\log(\sum\theta_{i}x_{i})\leq\sum\theta_{i}x_{i}\log x_{i}$

(E)

 $Q(\lambda,\lambda^{\prime})\leq Q(\lambda,\overline{\lambda})$

(F)

 $Q(\lambda,\lambda^{\prime})-Q(\lambda,\lambda)\leq P(\lambda)\log\frac{P(% \lambda^{\prime})}{P(\lambda)}$

(G) the maximum-entropy inequality (in logarithmic form)

 $-\sum_{i=1}^{n}p_{i}\log p_{i}\leq\log n$

(H) Hölder’s generalized inequality (http://planetmath.org/GeneralizedHolderInequality)

 $\sum_{j=1}^{n}\prod_{i=1}^{m}a_{i,j}^{\theta_{i}}\leq\prod_{i=1}^{m}\left(\sum% _{j=1}^{n}a_{i,j}\right)^{\theta_{i}}$

(P) The PTAH inequality:

 $P(\lambda)\leq P(\overline{\lambda})$

All the sums and products   range from 1 to $n$, all the $\theta_{i},x_{i},r_{i}$ are positive and $(\theta_{i}),\lambda,\lambda^{\prime}$ are in $\sigma$ and the set $X$ is discrete, so that

 $P(\lambda)=\sum_{x}p(x,\lambda)m(x)$
 $Q(\lambda,\lambda^{\prime})=\sum_{i}r_{i}(\lambda)\log{\lambda_{i}}^{\prime}$

where $m(x)>0$ $p(x,\lambda)=\prod_{i}{\lambda_{i}}^{a_{i}(x)}$ and $a_{i}(x)\geq 0$, $r_{i}(\lambda)=\sum a_{i}(x)p(x,\lambda)m(x)$ and

 $\overline{\lambda_{i}}=\frac{\lambda_{i}\partial P/\partial\lambda_{i}}{\sum% \lambda_{j}\partial P/\partial\lambda_{j}},$

and $\overline{\lambda}=(\overline{\lambda_{i}})$. Then it turns out that (A) to (G) are all special cases of (H), and in fact (A) to (G) are all equivalent     , in the sense that given any two of them, each is a special case of the other. (H) is a special case of (P), However, it appears that none of the reverse implications  holds. According to George Soules:

”The folklore at the Institute for Defense Analyses in Princeton NJ is that the first program to maximize a function P(z) by iterating the growth transformation  $z\to\overline{z}$

was written while the programmer was listening to the opera Aida, in which the Egyptian god of creation Ptah is mentioned, and that became the name of the program (and of the inequality). The name is in upper case because the word processor in use in the middle 1960’s had no lower case.”

References

Title PTAH inequality PTAHInequality 2013-03-22 16:54:32 2013-03-22 16:54:32 Mathprof (13753) Mathprof (13753) 27 Mathprof (13753) Theorem msc 26D15