likelihood function
Let X=(${X}_{1},\mathrm{\dots},{X}_{n}$) be a random vector and
$$\{{f}_{\mathbf{X}}(\bm{x}\mid \bm{\theta}):\bm{\theta}\in \mathrm{\Theta}\}$$ 
a statistical model parametrized by $\bm{\theta}=({\theta}_{1},\mathrm{\dots},{\theta}_{k})$, the parameter vector in the parameter space $\mathrm{\Theta}$. The likelihood function^{} is a map $L:\mathrm{\Theta}\to \mathbb{R}$ given by
$$L(\bm{\theta}\mid \bm{x})={f}_{\mathbf{X}}(\bm{x}\mid \bm{\theta}).$$ 
In other words, the likelikhood function^{} is functionally the same in form as a probability density function^{}. However, the emphasis is changed from the $\bm{x}$ to the $\bm{\theta}$. The pdf is a function of the $x$’s while holding the parameters $\theta $’s constant, $L$ is a function of the parameters $\theta $’s, while holding the $x$’s constant.
When there is no confusion, $L(\bm{\theta}\mid \bm{x})$ is abbreviated to be $L(\bm{\theta})$.
The parameter vector $\widehat{\bm{\theta}}$ such that $L(\widehat{\bm{\theta}})\ge L(\bm{\theta})$ for all $\bm{\theta}\in \mathrm{\Theta}$ is called a maximum likelihood estimate, or MLE, of $\bm{\theta}$.
Many of the density functions are exponential^{} in nature, it is therefore easier to compute the MLE of a likelihood function $L$ by finding the maximum of the natural log of $L$, known as the loglikelihood function:
$$\mathrm{\ell}(\bm{\theta}\mid \bm{x})=\mathrm{ln}(L(\bm{\theta}\mid \bm{x}))$$ 
due to the monotonicity of the log function.
Examples:

1.
A coin is tossed $n$ times and $m$ heads are observed. Assume that the probability of a head after one toss is $\pi $. What is the MLE of $\pi $?
Solution: Define the outcome of a toss be 0 if a tail is observed and 1 if a head is observed. Next, let ${X}_{i}$ be the outcome of the $i$th toss. For any single toss, the density function is ${\pi}^{x}{(1\pi )}^{1x}$ where $x\in \{0,1\}$. Assume that the tosses are independent events, then the joint probability density is
$${f}_{\mathbf{X}}(\bm{x}\mid \pi )=\left(\genfrac{}{}{0pt}{}{n}{\mathrm{\Sigma}{x}_{i}}\right){\pi}^{\mathrm{\Sigma}{x}_{i}}{(1\pi )}^{\mathrm{\Sigma}(1{x}_{i})}=\left(\genfrac{}{}{0pt}{}{n}{m}\right){\pi}^{m}{(1\pi )}^{nm},$$ which is also the likelihood function $L(\pi )$. Therefore, the loglikelihood function has the form
$$\mathrm{\ell}(\pi \mid \bm{x})=\mathrm{\ell}(\pi )=\mathrm{ln}\left(\genfrac{}{}{0pt}{}{n}{m}\right)+m\mathrm{ln}(\pi )+(nm)\mathrm{ln}(1\pi ).$$ Using standard calculus, we get that the MLE of $\pi $ is
$$\widehat{\pi}=\frac{m}{n}=\overline{x}.$$ 
2.
Suppose a sample of $n$ data points ${X}_{i}$ are collected. Assume that the ${X}_{i}\sim N(\mu ,{\sigma}^{2})$ and the ${X}_{i}$’s are independent of each other. What is the MLE of the parameter vector $\bm{\theta}=(\mu ,{\sigma}^{2})$?
Solution: The joint pdf of the ${X}_{i}$, and hence the likelihood function, is
$$L(\bm{\theta}\mid \bm{x})=\frac{1}{{\sigma}^{n}{(2\pi )}^{n/2}}\mathrm{exp}(\frac{\mathrm{\Sigma}{({x}_{i}\mu )}^{2}}{2{\sigma}^{2}}).$$ The loglikelihood function is
$$\mathrm{\ell}(\bm{\theta}\mid \bm{x})=\frac{\mathrm{\Sigma}{({x}_{i}\mu )}^{2}}{2{\sigma}^{2}}\frac{n}{2}\mathrm{ln}({\sigma}^{2})\frac{n}{2}\mathrm{ln}(2\pi ).$$ Taking the first derivative^{} (gradient), we get
$$\frac{\partial \mathrm{\ell}}{\partial \bm{\theta}}=(\frac{\mathrm{\Sigma}({x}_{i}\mu )}{{\sigma}^{2}},\frac{\mathrm{\Sigma}{({x}_{i}\mu )}^{2}}{2{\sigma}^{4}}\frac{n}{2{\sigma}^{2}}).$$ Setting
$$\frac{\partial \mathrm{\ell}}{\partial \bm{\theta}}=\mathrm{\U0001d7ce}\text{See score function}$$ and solve for $\bm{\theta}=(\mu ,{\sigma}^{2})$ we have
$$\widehat{\bm{\theta}}=(\widehat{\mu},{\widehat{\sigma}}^{2})=(\overline{x},\frac{n1}{n}{s}^{2}),$$ where $\overline{x}=\mathrm{\Sigma}{x}_{i}/n$ is the sample mean^{} and ${s}^{2}=\mathrm{\Sigma}{({x}_{i}\overline{x})}^{2}/(n1)$ is the sample variance. Finally, we verify that $\widehat{\bm{\theta}}$ is indeed the MLE of $\bm{\theta}$ by checking the negativity of the 2nd derivatives (for each parameter).
Title  likelihood function 

Canonical name  LikelihoodFunction 
Date of creation  20130322 14:27:58 
Last modified on  20130322 14:27:58 
Owner  CWoo (3771) 
Last modified by  CWoo (3771) 
Numerical id  13 
Author  CWoo (3771) 
Entry type  Definition 
Classification  msc 62A01 
Synonym  likelihood statistic 
Synonym  likelihood 
Defines  maximum likelihood estimate 
Defines  MLE 
Defines  loglikelihood function 