Let $X_1,X_2,\ldots,X_n$ be samples (with observations $X_i=x_i$ ) from a distribution with probability density function $f(X\mid\theta)$ , where $\theta$ is a real-valued unknown parameter in $f$ . Consider $\theta$ as a random variable and let $\tau(\theta)$ be its realization.

An estimator for $\theta$ is a statistic $\eta_{\theta}=\eta_{\theta}(X_1,X_2,\ldots,X_n)$ that is used to, loosely speaking, estimate $\tau(\theta)$ . Any value $\eta_{\theta}(X_1=x_1,X_2=x_2,\ldots,X_n=x_n)$ of $\eta_{\theta}$ is called an estimate of $\tau(\theta)$ .

Example. Let $X_1,X_2,\ldots,X_n$ be iid from a normal distribution $N(\mu,\sigma^2)$ . Here the two parameters are the mean $\mu$ and the variance $\sigma^2$ . The sample mean $\overline{X}$ is an estimator of $\mu$ , while the sample variance $s^2$ is an estimator of $\sigma^2$ . In addition, sample median, sample mode, sample trimmed mean are all estimators of $\mu$ . The statistic defined by $$\frac{1}{n-1}\sum_{i=1}^{n}(X_i-m)^2,$$ where $m$ is a sample median, is another estimator of $\sigma^2$ .