score function

Given a statistical model $\{f_{𝐗}(𝒙\mid 𝜽):𝜽\in \mathrm{\Theta }\}$ with log-likelihood function $\mathrm{\ell }(𝜽\mid 𝒙)$, the score function $U$ is defined to be the gradient of $\mathrm{\ell }$:

$$U(𝜽)=\nabla \mathrm{\ell }=\frac{\partial \mathrm{\ell }}{\partial 𝜽}.$$

Since the score function $U$ is also a function of the random vector $𝒙$, $U$ is itself a random vector. By setting $U$ to 0, we have a system of $k$ equation(s), otherwise known as the likelihood equation(s):

$$U(𝜽)=(\frac{\partial \mathrm{\ell }}{\partial {\theta }_1},\mathrm{\dots },\frac{\partial \mathrm{\ell }}{\partial {\theta }_k})=(0,\mathrm{\dots },0).$$

If $𝜽=\theta$ is one-dimensional, then the score function is simply referred to as the score of $\theta$.

The maximum likelihood estimate (MLE) $\widehat{𝜽}$ of the parameter vector $𝜽$ can usually be found by finding the solutions of the likelihood equations. The likelihood equations may also be formed by setting the gradient of the plain likelihood function to zero. The use of the log function often facilitates the algebra as many distributions are exponential in nature. For some distributions it may also be necessary to test that the solution to the likelihood equations is really a minimum as opposed to a point of inflection.

Example. $n$ independent observations are made from a random variable $X$ with a Poisson distribution with parameter $\lambda$. The observed values are $x_1,\mathrm{\dots },x_n$. The log-likelihood of the joint pdf is

$$\mathrm{\ell }(\lambda \mid 𝒙)=\sum _{i=1}^n-\lambda +x_i\ln (\lambda )-\ln (x_i!)$$

and so the score function is

$$U(\lambda )=\frac{d\mathrm{\ell }}{d\lambda }=\sum _{i=1}^n\left(-1+\frac{x_i}{\lambda }\right)=-n+\frac{n\overline{x}}{\lambda },$$

where $n\overline{x}=\sum x_i$. To find the MLE of $\lambda$, we set $U=0$ and solve for $\lambda$. So the MLE $\widehat{\lambda }$ of $\lambda =\overline{x}$.

Title	score function
Canonical name	ScoreFunction
Date of creation	2013-03-22 14:28:02
Last modified on	2013-03-22 14:28:02
Owner	CWoo (3771)
Last modified by	CWoo (3771)
Numerical id	13
Author	CWoo (3771)
Entry type	Definition
Classification	msc 62A01
Synonym	score
Synonym	score statistic
Defines	likelihood equation