# Gauss-Markov theorem

A Gauss-Markov linear model is a linear statistical model that satisfies all the conditions of a general linear model except the normality of the error terms. Formally, if $\boldsymbol{Y}$ is an $m$-dimensional response variable vector, and $\boldsymbol{Z_{i}}=z_{i}(\boldsymbol{X})$, $i=1,\ldots,k$ are the $m$-dimensional functions of the explanatory variable vector $\boldsymbol{X}$, a Gauss-Markov linear model has the form:

 $\boldsymbol{Y}=\beta_{0}\boldsymbol{Z_{0}}+\cdots+\beta_{k}\boldsymbol{Z_{k}}+% \boldsymbol{\epsilon},$

with $\boldsymbol{\epsilon}$ the error vector such that

1. 1.

$\operatorname{E}[\boldsymbol{\epsilon}]=\boldsymbol{0}$, and

2. 2.

$\operatorname{Var}[\boldsymbol{\epsilon}]=\sigma^{2}\boldsymbol{I}$.

In other words, the observed responses $Y_{i}$, $i=1,\ldots,m$ are not assumed to be normally distributed, are not correlated with one another, and have a common variance $\operatorname{Var}[Y_{i}]=\sigma^{2}$.

Gauss-Markov Theorem. Suppose the response variable $\boldsymbol{Y}=(Y_{1},\ldots,Y_{m})$ and the explanatory variables $\boldsymbol{X}$ satisfy a Gauss-Markov linear model as described above. Consider any linear combination of the responses

 $\displaystyle Y=\sum_{i=1}^{m}c_{i}Y_{i},$ (1)

where $c_{i}\in\mathbb{R}$. If each $\mu_{i}$ is an estimator for response $Y_{i}$, parameter $\theta$ of the form

 $\displaystyle\theta=\sum_{i=1}^{m}c_{i}\mu_{i},$ (2)

can be used as an estimator for $Y$. Then, among all unbiased estimators for $Y$ having form (2), the ordinary least square estimator (OLS)

 $\displaystyle\hat{\theta}=\sum_{i=1}^{m}c_{i}\hat{\mu_{i}},$ (3)

yields the smallest variance. In other words, the OLS estimator is the uniformly minimum variance unbiased estimator.

Remark. $\hat{\theta}$ in equation (3) above is more popularly known as the BLUE, or the best linear unbiased estimator for a linear combination of the responses in a Gauss-Markov linear model.

Title Gauss-Markov theorem GaussMarkovTheorem 2013-03-22 15:02:53 2013-03-22 15:02:53 CWoo (3771) CWoo (3771) 8 CWoo (3771) Theorem msc 62J05 BLUE LinearLeastSquaresFit Gauss-Markov linear model best linear unbiased estimator