# score function

Given a statistical model $\{{f}_{\mathbf{X}}(\bm{x}\mid \bm{\theta}):\bm{\theta}\in \mathrm{\Theta}\}$ with log-likelihood function^{} $\mathrm{\ell}(\bm{\theta}\mid \bm{x})$, the *score function ^{}* $U$ is defined to be the gradient of $\mathrm{\ell}$:

$$U(\bm{\theta})=\nabla \mathrm{\ell}=\frac{\partial \mathrm{\ell}}{\partial \bm{\theta}}.$$ |

Since the score function $U$ is also a function of the random vector $\bm{x}$, $U$ is itself a random vector.
By setting $U$ to 0, we have a system of $k$ equation(s), otherwise known as the *likelihood equation(s)*:

$$U(\bm{\theta})=(\frac{\partial \mathrm{\ell}}{\partial {\theta}_{1}},\mathrm{\dots},\frac{\partial \mathrm{\ell}}{\partial {\theta}_{k}})=(0,\mathrm{\dots},0).$$ |

If $\bm{\theta}=\theta $ is one-dimensional, then the score function is simply referred to as the *score* of $\theta $.

The maximum likelihood estimate (MLE) $\widehat{\bm{\theta}}$ of the parameter vector $\bm{\theta}$ can usually be found by finding the solutions of the likelihood equations. The likelihood equations may also be formed by setting the gradient of the plain likelihood function to zero. The use of the log function often facilitates the algebra as many distributions^{} are exponential^{} in nature. For some distributions it may also be necessary to test that the solution to the likelihood equations is really a minimum as opposed to a point of inflection.

Example. $n$ independent observations are made from a random variable^{} $X$ with a Poisson distribution^{} with parameter $\lambda $. The observed values are ${x}_{1},\mathrm{\dots},{x}_{n}$. The log-likelihood of the joint pdf is

$$\mathrm{\ell}(\lambda \mid \bm{x})=\sum _{i=1}^{n}-\lambda +{x}_{i}\mathrm{ln}(\lambda )-\mathrm{ln}({x}_{i}!)$$ |

and so the score function is

$$U(\lambda )=\frac{d\mathrm{\ell}}{d\lambda}=\sum _{i=1}^{n}\left(-1+\frac{{x}_{i}}{\lambda}\right)=-n+\frac{n\overline{x}}{\lambda},$$ |

where $n\overline{x}=\sum {x}_{i}$. To find the MLE of $\lambda $, we set $U=0$ and solve for $\lambda $. So the MLE $\widehat{\lambda}$ of $\lambda =\overline{x}$.

Title | score function |
---|---|

Canonical name | ScoreFunction |

Date of creation | 2013-03-22 14:28:02 |

Last modified on | 2013-03-22 14:28:02 |

Owner | CWoo (3771) |

Last modified by | CWoo (3771) |

Numerical id | 13 |

Author | CWoo (3771) |

Entry type | Definition |

Classification | msc 62A01 |

Synonym | score |

Synonym | score statistic |

Defines | likelihood equation |