|
|
|
|
Occam's razor
|
(Definition)
|
|
|
The following example in statistics illustrates the principle of Occam's razor. See Article on Occam's Razor on Wikipedia for a thorough discussion, mathematical or not, of this principle, which is also known as the principle of parsimony.
Example in statistics:
Given the following ten pairs of hypothetical observations of continuous response variable  and continuous explanatory variable  :
 |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
8 |
9 |
10 |
 |
11 |
13 |
15 |
15 |
16 |
18 |
21 |
22 |
22 |
24 |
be fitted using a linear regression model given by
Using the method of least squares, the regression coefficients are found to be
 and
 , so that the regression line is
The p-values for both regression coefficients are found to be less than 0.0001 and the p-value for the overall fit of the model is also less than 0.0001. The p-value analysis can be summarized by the following table:
| |
 |
 |
overall fit of model |
| p-value |
 |
 |
 |
This indicates that the linear regression equation fits the data very well.
Next, fit the data using a 2nd order polynomial regression model, given by
Least square estimation shows that the regression equation is given by
The following table shows the result of the p-value analysis of the 2nd order polynomial regression model:
| |
 |
 |
 |
overall fit of model |
| p-value |
 |
 |
0.9188 |
 |
The p-value for the fit of the overall model suggests that the polynomial regression equation also fits the data well. The same can be said about the estimates of the regression coefficients  and  . However, the coefficient
 shows that the additional term does not contribute too much to the overall fit of the model to the observations. Furthermore, its p-value is very high, indicating that the  term is not significant.
In light of the above analysis, we prefer the simpler model
 over the more complicated one
 . This example shows that, in statistical modeling, when a simpler, easier to interpret model exists in the presence of more complicated ones, the simpler one should be chosen. This is Occam's razor at work!
|
Anyone with an account can edit this entry. Please help improve it!
"Occam's razor" is owned by CWoo. [ full author list (4) | owner history (1) ]
|
|
(view preamble)
| Other names: |
law of parsimony |
|
|
Cross-references: term, coefficient, estimates, polynomial, polynomial regression model, order, equation, p-values, line, regression coefficients, least squares, linear regression model, explanatory variable, response variable, continuous, observations, statistics
There are 3 references to this entry.
This is version 6 of Occam's razor, born on 2004-10-14, modified 2007-12-18.
Object id is 6371, canonical name is OccamsRazor.
Accessed 5895 times total.
Classification:
| AMS MSC: | 00A20 (General :: General and miscellaneous specific topics :: Dictionaries and other general reference works) | | | 62-07 (Statistics :: Data analysis) | | | 62A01 (Statistics :: Foundational and philosophical topics) |
|
|
|
|
|
|
Pending Errata and Addenda
|
|
|
|
|
|
|
|
|
|
|
