PlanetMath (more info)
 Math for the people, by the people.
Encyclopedia | Requests | Forums | Docs | Wiki | Random | RSS  
Login
create new user
name:
pass:
forget your password?
Main Menu
sections
Encyclopædia
Papers
Books
Expositions

meta
Requests (210)
Orphanage
Unclass'd (1)
Unproven (472)
Corrections (38)
Classification

talkback
Polls
Forums
Feedback
Bug Reports

downloads
Snapshots
PM Book

information
News
Docs
Wiki
ChangeLog
TODO List
Legalese
About
Owner confidence rating: Very high Entry average rating: No information on entry rating
score function (Definition)

Given a statistical model $ \lbrace f_{\mathbf{X}}(\boldsymbol{x}\mid\boldsymbol{\theta}) : \boldsymbol{\theta} \in \Theta \rbrace$ with log-likelihood function $ \ell(\boldsymbol{\theta}\mid\boldsymbol{x})$, the score function $ U$ is defined to be the gradient of $ \ell$:

$\displaystyle U(\boldsymbol{\theta})=\nabla\ell=\frac{\partial\ell}{\partial\boldsymbol{\theta}}.$
Since the score function $ U$ is also a function of the random vector $ \boldsymbol{x}$, $ U$ is itself a random vector. By setting $ U$ to 0, we have a system of $ k$ equation(s), otherwise known as the likelihood equation(s):
$\displaystyle U(\boldsymbol{\theta})=\Big(\frac{\partial\ell}{\partial\theta_1},\ldots,\frac{\partial\ell}{\partial\theta_k}\Big)= (0,\ldots,0).$
If $ \boldsymbol{\theta}=\theta$ is one-dimensional, then the score function is simply referred to as the score of $ \theta$.

The maximum likelihood estimate (MLE) $ \hat{\boldsymbol{\theta}}$ of the parameter vector $ \boldsymbol{\theta}$ can usually be found by finding the solutions of the likelihood equations. The likelihood equations may also be formed by setting the gradient of the plain likelihood function to zero. The use of the log function often facilitates the algebra as many distributions are exponential in nature. For some distributions it may also be necessary to test that the solution to the likelihood equations is really a minimum as opposed to a point of inflection.

Example. $ n$ independent observations are made from a random variable $ X$ with a Poisson distribution with parameter $ \lambda$. The observed values are $ x_1,\ldots,x_n$. The log-likelihood of the joint pdf is

$\displaystyle \ell(\lambda\mid\boldsymbol{x})=\sum_{i=1}^{n}-\lambda+x_i\ln(\lambda)-\ln(x_i!)$
and so the score function is
$\displaystyle U(\lambda)=\frac{d\ell}{d\lambda}=\sum_{i=1}^{n}\big(-1+\frac{x_i}{\lambda}\big)=-n+\frac{n\overline{x}}{\lambda},$
where $ n\overline{x}=\sum x_i$. To find the MLE of $ \lambda$, we set $ U=0$ and solve for $ \lambda$. So the MLE $ \hat{\lambda}$ of $ \lambda=\overline{x}$.



"score function" is owned by CWoo. [ full author list (2) ]
(view preamble)

View style:

Other names:  score, score statistic
Also defines:  likelihood equation
Log in to rate this entry.
(view current ratings)

Cross-references: Poisson distribution, random variable, observations, independent, point of inflection, necessary, exponential, distributions, algebra, log, likelihood function, solutions, vector, parameter, MLE, maximum likelihood estimate, equation, random vector, function, gradient, log-likelihood function, statistical model
There are 5 references to this entry.

This is version 10 of score function, born on 2004-07-08, modified 2007-04-21.
Object id is 5988, canonical name is ScoreFunction.
Accessed 12633 times total.

Classification:
AMS MSC: 62A01 (Statistics :: Foundational and philosophical topics)
Pending Errata and Addenda
None.
[ View all 1 ]
Discussion
Style: Expand: Order:
forum policy
Likelihood Equations by gmr001 on 2005-06-01 07:45:58
Hi,

I think the expection of the score function will always be zero, only the variance is affected by the parameter.

I think the likelihood equations should be "score(\theta) = 0", not "expect(score(\theta)) = 0". If you do the integration necessary in computing the expectation, you end up differentiating a constant and thus get zero.

Am I wrong?

Thanks,

Greg

[ reply | up ]
Expectation evaluated with respect to what? by yaroslavvb on 2004-12-06 18:03:31
It says we evaluate expectation of the score function, and set it to zero. But which distribution is expectation evaluated with respect to?

If we have a Bernoulli variable and observe a heads and b tails,
then

log-likelihood: l(t)=a log(t) + b log(1-t)
score function: U(t) = a/t + b/(1-t)

maximum likelihood solution is a/(a+b) which is the solution of U(t)=0

So it looks like here we are setting the score function to 0 directly, and not it's expectation. Am I missing something here?


[ reply | up ]
Interact
post | correct | update request | add derivation | add example | add (any)