factorization criterion

Let $𝑿=(X_1,\mathrm{\dots },X_n)$ be a random vector whose coordinates are observations, and whose probability (density) function is, $f(𝒙\mid \theta )$ where $\theta$ is an unknown parameter. Then a statistic $T(𝑿)$ for $\theta$ is a sufficient statistic iff $f$ can be expressed as a product of (or factored into) two functions $g,h$, $f=gh$ where $g$ is a function of $T(𝑿)$ and $\theta$, and $h$ is a function of $𝒙$. In symbol, we have

$$f(𝒙\mid \theta )=g(T(𝑿),\theta )h(𝒙).$$

Applications.

1. In view of the above statement, let’s show that the sample mean $\overline{X}$ of $n$ independent observations from a normal distribution $N(\mu ,{\sigma }^2)$ is a sufficient statistic for the unknown mean $\mu$. Since the $X_i$’s are independent random variables, then the probability density function $f(𝒙\mid \mu )$, being the joint probability density function of each of the $X_i$, is the product of the individual density functions $f(x\mid \mu )$:

   $f(𝒙\mid \mu )$ $=$ ${\displaystyle \prod _{i=1}^n}f(x\mid \mu )={\displaystyle \prod _{i=1}^n}{\displaystyle \frac{1}{\sqrt{2\pi {\sigma }^2}}}\exp [-{\displaystyle \frac{{(x_i-\mu )}^2}{2{\sigma }^2}}]$ (1)

   $=$ ${\displaystyle \frac{1}{\sqrt{{(2\pi )}^n{\sigma }^{2n}}}}\exp \left[{\displaystyle \sum _{i=1}^n}-{\displaystyle \frac{{(x_i-\mu )}^2}{2{\sigma }^2}}\right]$ (2)

   $=$ ${\displaystyle \frac{1}{\sqrt{{(2\pi )}^n{\sigma }^{2n}}}}\exp \left[{\displaystyle \frac{-1}{2{\sigma }^2}}{\displaystyle \sum _{i=1}^n}{x}_i^2\right]\exp \left[{\displaystyle \frac{\mu }{{\sigma }^2}}{\displaystyle \sum _{i=1}^n}{x}_i-{\displaystyle \frac{n{\mu }^2}{2{\sigma }^2}}\right]$ (3)

   $=$ $h(𝒙)\exp \left[{\displaystyle \frac{n\mu }{{\sigma }^2}}T(𝒙)-{\displaystyle \frac{n{\mu }^2}{2{\sigma }^2}}\right]$ (4)

   $=$ $h(𝒙)g(T(𝒙),\mu )$ (5)

   where $g$ is the last exponential expression and $h$ is the rest of the expression in $(3)$. By the factorization criterion, $T(𝑿)=\overline{X}$ is a sufficient statistic.

2. Similarly, the above shows that the sample variance $s^2$ is not a sufficient statistic for ${\sigma }^2$ if $\mu$ is unknown.

3. But, if $\mu$ is a known constant, then the statistic

   $$T(X_1,\mathrm{\dots },X_n)=\frac{1}{n-1}\sum _{i=1}^n{(X_i-\mu )}^2$$

   is sufficient for ${\sigma }^2$ by observing in $(2)$ above, and letting $h(𝒙)=1$ and $g(T,{\sigma }^2)$ be all of expression $(2)$.

Title	factorization criterion
Canonical name	FactorizationCriterion
Date of creation	2013-03-22 15:02:48
Last modified on	2013-03-22 15:02:48
Owner	CWoo (3771)
Last modified by	CWoo (3771)
Numerical id	4
Author	CWoo (3771)
Entry type	Theorem
Classification	msc 62B05
Synonym	factorization theorem
Synonym	Fisher-Neyman factorization theorem