|
|
|
|
Fisher information matrix
|
(Definition)
|
|
|
Given a statistical model
 of a random vector
 , the Fisher information matrix,  , is the variance of the score function  . So,
If there is only one parameter involved, then  is simply called the Fisher information or information of
 .
Remarks
- If
 belongs to the exponential family,
![$ I=\operatorname{E}\big[U^{\operatorname{T}}U\big]$](http://images.planetmath.org:8080/cache/objects/6041/l2h/img9.png "$ I=\operatorname{E}\big[U^{\operatorname{T}}U\big]$") . Furthermore, with some regularity conditions imposed, we have
- As an example, the normal distribution,
 , belongs to the exponential family and its log-likelihood function
 is
where
 . Then the score function
 is given by
Taking the derivative with respect to
 , we have
Therefore, the Fisher information matrix  is
- Now, in linear regression model with constant variance
 , it can be shown that the Fisher information matrix  is
where
 is the design matrix of the regression model.
- In general, the Fisher information meansures how much “information” is known about a parameter
 . If  is an unbiased estimator of  , it can be shown that
This is known as the Cramer-Rao inequality, and the number
 is known as the Cramer-Rao lower bound. The smaller the variance of the estimate of  , the more information we have on  . If there is more than one parameter, the above can be generalized by saying that
is positive semidefinite, where  is the Fisher information matrix.
|
"Fisher information matrix" is owned by CWoo.
|
|
(view preamble)
| Other names: |
information matrix |
| Also defines: |
Fisher information, information, Cramer-Rao inequality, Cramer-Rao lower bound |
|
|
Cross-references: positive semidefinite, estimate, unbiased estimator, regression model, matrix, design, linear regression model, derivative, log-likelihood function, normal distribution, regularity, exponential family, parameter, score function, variance, random vector, statistical model
There are 30 references to this entry.
This is version 11 of Fisher information matrix, born on 2004-07-27, modified 2007-04-21.
Object id is 6041, canonical name is FisherInformationMatrix.
Accessed 22743 times total.
Classification:
| AMS MSC: | 62A01 (Statistics :: Foundational and philosophical topics) | | | 62B10 (Statistics :: Sufficiency and information :: Information-theoretic topics) | | | 62H99 (Statistics :: Multivariate analysis :: Miscellaneous) |
|
|
|
|
|
|
Pending Errata and Addenda
|
|
|
|
|
|
|
|
|
|
|
![$\displaystyle I=\operatorname{Var}[U].$](http://images.planetmath.org:8080/cache/objects/6041/l2h/img5.png "$\displaystyle I=\operatorname{Var}[U].$"){kind=small}
![$\displaystyle I=-\operatorname{E}\Big[\frac{\partial U}{\partial\boldsymbol{\theta}}\Big].$](http://images.planetmath.org:8080/cache/objects/6041/l2h/img10.png "$\displaystyle I=-\operatorname{E}\Big[\frac{\partial U}{\partial\boldsymbol{\theta}}\Big].$")
![$\displaystyle -\operatorname{E}\Big[\frac{\partial U}{\partial\boldsymbol{\thet... ...ig]=\frac{1}{2\sigma^4} \begin{pmatrix} 2\sigma^2 & 0 \ 0 & -1 \end{pmatrix}.$](http://images.planetmath.org:8080/cache/objects/6041/l2h/img20.png "$\displaystyle -\operatorname{E}\Big[\frac{\partial U}{\partial\boldsymbol{\thet... ...ig]=\frac{1}{2\sigma^4} \begin{pmatrix} 2\sigma^2 & 0 \ 0 & -1 \end{pmatrix}.$")
![$\displaystyle \operatorname{Var}\big[T(X)\big]\ge\frac{1}{I(\theta)}$](http://images.planetmath.org:8080/cache/objects/6041/l2h/img28.png "$\displaystyle \operatorname{Var}\big[T(X)\big]\ge\frac{1}{I(\theta)}$")
![$\displaystyle \operatorname{Var}\big[T(X)\big]-I(\boldsymbol{\theta})^{-1}$](http://images.planetmath.org:8080/cache/objects/6041/l2h/img32.png "$\displaystyle \operatorname{Var}\big[T(X)\big]-I(\boldsymbol{\theta})^{-1}$"){kind=small}