Let be a statistical model with parameter . Let be a random vector of random variables representing observations. A statistic of for the parameter is called a sufficient statistic, or a sufficient estimator, if the conditional probability distribution of given is not a function of (equivalently, does not depend on ).
In other words, all the information about the unknown parameter is captured in the sufficient statistic . If, say, we are interested in finding out the percentage of defective light bulbs in a shipment of new ones, it is enough, or sufficient, to count the number of defective ones (sum of the ’s), rather than worrying about which individual light bulbs are the defective ones (the vector ). By taking the sum, a certain “reduction” of data has been achieved.
The numerator is if . So in this case, and is not a function of . Otherwise, the numerator is and becomes
where ’s are the rearrangements of the ’s in a non-decreasing order from to . For the denominator, we first note that
From the above equation, we find that there are ways to form non-decreasing finite sequences of positive integers such that the maximum of the sequence is . So
again is not a function of . Therefore, is a sufficient statistic for . Here, we see that a reduction of data has been achieved by taking only the largest member of set of observations, not the entire set.
If we set , then we see that is trivially a sufficient statistic for any parameter . The conditional probability distribution of given is 1. Even though this is a sufficient statistic by definition (of course, the individual observations provide as much information there is to know about as possible), and there is no loss of data in (which is simply a list of all observations), there is really no reduction of data to speak of here.
The sample mean
of independent observations from a normal distribution (both and unknown) is a sufficient statistic for . This is the result of the factorization criterion. Similarly, one sees that any partition of the sum of observations into subtotals is a sufficient statistic for . For instance,
is a sufficient statistic for .
A sufficient statistic for a parameter is called a minimal sufficient statistic if it can be expressed as a function of any sufficient statistic for .
Example. In example above, both the sample mean and the finite sum are minimal sufficient statistics for the mean . Since, by the factorization criterion, any sufficient statistic for is a vector whose coordinates form a partition of the finite sum, taking the sum of these coordinates is just the finite sum . So, we have just expressed as a function of . Therefore, is minimal. Similarly, is minimal.
Two sufficient statistics for a parameter are said to be equivalent provided that there is a bijection such that . and from the above example are two equivalent sufficient statistics. Two minimal sufficient statistics for the same parameter are equivalent.
|Date of creation||2013-03-22 15:02:42|
|Last modified on||2013-03-22 15:02:42|
|Last modified by||CWoo (3771)|
|Synonym||minimally sufficient statistic|
|Defines||minimal sufficient statistic|