A probability (density) function $f_X(x\mid\theta)$ given a parameter $\theta$ is said to belong to the (one parameter) exponential family of distributions if it can be written in one of the following two equivalent forms:

1. $a(x)b(\theta)\operatorname{exp}\big[ c(x)d(\theta)\big ]$
2. $\operatorname{exp}\big[ a(x)+b(\theta)+c(x)d(\theta) \big]$

Examples:

• The normal distribution, $N(\mu,\sigma^2)$ , treating $\sigma^2$ as a nuisance parameter, belongs to the exponential family. To see this, take the natural logarithm of $N(\mu,\sigma^2)$ to get $$-\frac{1}{2}\operatorname{ln}(2\pi\sigma^2)-\frac{1}{2\sigma^2}(x-\mu)^2$$ Rearrange the above expression and we have $$\frac{x\mu}{\sigma^2}-\frac{\mu^2}{2\sigma^2}-\frac{1}{2}\Big[\frac{x^2}{\sigma^2}+\operatorname{ln}(2\pi\sigma^2)\Big]$$ Set $c(x)=x$ , $d(\mu)=\mu/\sigma^2$ , $b(\mu)=-\mu^2/(2\sigma^2)$ , and $a(x)=-1/2\big[x^2/\sigma^2+\operatorname{ln}(2\pi\sigma^2)\big]$ . Then we see that $N(\mu,\sigma^2)$ does indeed belong to the exponential family. Furthermore, it is in canonical form. The natural parameter is $d(\mu)=\mu/\sigma^2$ .
• Similarly, the Poisson, binomial, Gamma, and inverse Gaussian distributions all belong to the exponential family and they are all in canonical form.
• Lognormal and Weibull distributions also belong to the exponential family but they are not in canonical form.

Remarks

• If the p.d.f of a random variable $X$ belongs to an exponential family, and it is expressed in the second of the two above forms, then \begin{equation} \operatorname{E}[c(X)]=-\frac{b'(\theta)}{d'(\theta)}, \end{equation}
  and
  \begin{equation} \operatorname{Var}[c(X)]=\frac{d''(\theta)b'(\theta)-d'(\theta)b''(\theta)}{d'(\theta)^3}, \end{equation}provided that functions $b$ and $d$ are appropriately conditioned.
• Given a member from the exponential family of distributions, we have $\operatorname{E}[U]=0$ and $I=-\operatorname{E}[U']$ , where $U$ is the score function and $I$ the Fisher information. To see this, first observe that the log-likelihood function from a member of the exponential family of distributions is given by $$\ell(\theta\mid x)=a(x)+b(\theta)+c(x)d(\theta),$$ and hence the score function is $$U(\theta)=b'(\theta)+c(X)d'(\theta).$$ From (1), $\operatorname{E}[U]=0$ . Next, we obtain the Fisher information $I$ . By definition, we have \begin{eqnarray*} I&=&\operatorname{E}[U^2]-\operatorname{E}[U]^2\\ &=&\operatorname{E}[U^2]\\ &=&d'(\theta)^2\operatorname{Var}[c(X)]\\ &=&\frac{d''(\theta)b'(\theta)-d'(\theta)b''(\theta)}{d'(\theta)} \end{eqnarray*}On the other hand, $$\frac{\partial U}{\partial\theta}=b''(\theta)+c(X)d''(\theta)$$ so \begin{eqnarray*} \operatorname{E}\Big[\frac{\partial U}{\partial\theta}\Big] &=&b''(\theta)+\operatorname{E}[c(X)]d''(\theta)\\ &=&b''(\theta)-\frac{b'(\theta)}{d'(\theta)}d''(\theta)\\ &=&\frac{b''(\theta)d'(\theta)-b'(\theta)d''(\theta)}{d'(\theta)}\\ &=&-I \end{eqnarray*}
• For example, for a Poisson distribution $$f_X(x\mid\theta) = \frac{\theta^x e^{-\theta}}{x!},$$ the natural parameter $d(\theta)$ is $\operatorname{ln}\theta$ and $b(\theta)=-\theta$ . $c(x)=x$ since Poisson is in canonical form. Then $$U(\theta)=-1+\frac{X}{\theta}\mbox{ and }I=-\operatorname{E}\Big[\frac{-X}{\theta^2}\Big]=\frac{1}{\theta}$$ as expected.