## You are here

HomeOccam's razor

## Primary tabs

# Occam’s razor

The following example in statistics illustrates the principle of *Occam’s razor*. See Article on Occam’s Razor on Wikipedia for a thorough discussion, mathematical or not, of this principle, which is also known as the *principle of parsimony*.

Example in statistics:

Given the following ten pairs of hypothetical observations of continuous response variable $Y$ and continuous explanatory variable $X$:

$X$ | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|

$Y$ | 11 | 13 | 15 | 15 | 16 | 18 | 21 | 22 | 22 | 24 |

be fitted using a linear regression model given by

$Y=\alpha+\beta X.$ |

Using the method of least squares, the regression coefficients are found to be $\alpha=9.867$ and $\beta=1.424$, so that the regression line is

$Y=9.867+1.424X.$ |

The p-values for both regression coefficients are found to be less than 0.0001 and the p-value for the overall fit of the model is also less than 0.0001. The p-value analysis can be summarized by the following table:

$\alpha$ | $\beta$ | overall fit of model | |
---|---|---|---|

p-value | $<0.0001$ | $<0.0001$ | $<0.0001$ |

This indicates that the linear regression equation fits the data very well.

Next, fit the data using a 2nd order polynomial regression model, given by

$Y=\alpha+\beta X+\gamma X^{2}.$ |

Least square estimation shows that the regression equation is given by

$Y=9.783+1.466X-0.004X^{2}.$ |

The following table shows the result of the p-value analysis of the 2nd order polynomial regression model:

$\alpha$ | $\beta$ | $\gamma$ | overall fit of model | |
---|---|---|---|---|

p-value | $<0.0001$ | $0.0085$ | 0.9188 | $<0.0001$ |

The p-value for the fit of the overall model suggests that the polynomial regression equation also fits the data well. The same can be said about the estimates of the regression coefficients $\alpha$ and $\beta$. However, the coefficient $\gamma=-0.004$ shows that the additional term does not contribute too much to the overall fit of the model to the observations. Furthermore, its p-value is very high, indicating that the $X^{2}$ term is not significant.

In light of the above analysis, we prefer the simpler model $Y=\alpha+\beta X$ over the more complicated one $Y=\alpha+\beta X+\gamma X^{2}$. This example shows that, in statistical modeling, when a simpler, easier to interpret model exists in the presence of more complicated ones, the simpler one should be chosen. This is Occam’s razor at work!

## Mathematics Subject Classification

62-07*no label found*00A20

*no label found*62A01

*no label found*

- Forums
- Planetary Bugs
- HS/Secondary
- University/Tertiary
- Graduate/Advanced
- Industry/Practice
- Research Topics
- LaTeX help
- Math Comptetitions
- Math History
- Math Humor
- PlanetMath Comments
- PlanetMath System Updates and News
- PlanetMath help
- PlanetMath.ORG
- Strategic Communications Development
- The Math Pub
- Testing messages (ignore)

- Other useful stuff

## Recent Activity

new question: Prove that for any sets A, B, and C, An(BUC)=(AnB)U(AnC) by St_Louis

Apr 20

new image: information-theoretic-distributed-measurement-dds.png by rspuzio

new image: information-theoretic-distributed-measurement-4.2 by rspuzio

new image: information-theoretic-distributed-measurement-4.1 by rspuzio

new image: information-theoretic-distributed-measurement-3.2 by rspuzio

new image: information-theoretic-distributed-measurement-3.1 by rspuzio

new image: information-theoretic-distributed-measurement-2.1 by rspuzio

Apr 19

new collection: On the Information-Theoretic Structure of Distributed Measurements by rspuzio

Apr 15

new question: Prove a formula is part of the Gentzen System by LadyAnne

Mar 30

new question: A problem about Euler's totient function by mbhatia

## Comments

## Occam's Razor, PM vs. WIkipedia

How should we go about with entries that are already

in a very finished state at Wikipedia? For example,

in the case of this entry, it feels like a waste of

time composing a new entry, when there already exists

a fine article on Wikipedia. Also, I think this goes

against the whole principle of FDL.

As I understand things (and I might be wrong),

Wikipedia uses almost the same license as PM; so

we could copy material to PM from Wikipedia as long as we credit

the authors that have written the entry. Is this right?

How should such crediting be done? Is it for example

enough to mention that this entry is based on the

Wikipedia article xxx, version yyy?

I am not proposing a massive copy operation from W.

into PM as is. Clearly any material added to PM needs

to be scrutinized so that it fits into the existing PM

definitions.

- Quality\neq quantity.

- all material should be verified that it is actually

real mathematics. Are there any errors in the material.

- Wikipedia entries tend to be very long, and some

editing might be in place. For example, the

Occam's Razor entry contains much information that

has nothing to do with its use in mathematics.

- Should wiki-based entries be treated as "normal" entries

(editing policy/points)?

One approach would be to create "link" entries that are

essentially just links to an external resource. Much like

this entry is in its current state. For example, while

we eventually will want to have our own biography entries,

at the moment there are much better entries for example

on Wikipedia and on

http://www-gap.dcs.st-and.ac.uk/~history/Mathematicians/

With this in mind, it would make sense to create an

'Newton' entry that, say is world editable, and just contains

links to the aforementioned sites. This way, Newton will

be linked at PM, and over time we can build up a real biography

entry. This is the 'seed entry' idea that has been up on

discussion before.

What do you think?

Matte

## Re: Occam's Razor, PM vs. WIkipedia

I have looked at the explanation at the "Wiki" site. The explanation is thorough and very clear. However, I don't see any actual mathematical example regarding "Occam's Razor". Perhaps it would be a good idea, then, to add a concrete mathematical example in this PM entry. Any ideas on a concrete example?

Chi

## Re: Occam's Razor, PM vs. WIkipedia

Well, I suppose Occam's Razor applied to math insists that the

simplest proof should be prefered as a proof.

Thus, the below proof

http://planetmath.org/encyclopedia/NthRootOf2IsIrrationalForNge3ProofUsi...

should not be prefered as there are elementary ways to

prove that 2^(1/m) is irrational for m\ge 3.

Matte

## Re: Occam's Razor, PM vs. WIkipedia

As I see it, the two main issues here are convenience of the reader and not wasting resources.

At present, there are a bunch of websites which offer free mathematical information to the public --- in addition to to Planet Math and Wikipedia, there are, for instance, AGATHOS and the catalogue of algebraic systems.

Since there are several such sites, it can be inconvenient for the reader to have to look through a dozen websites to find a particular piece of information. Therefore, having some sort of "one-stop shopping" would be nice.

As for scarcity, there are two resources to be considered. On the one hand, there is storage space. In this age of DVDs and gigabyte laptops, there is no shortage of storage. On the other hand, there is the labour needed to keep these websites going and write articles. This resource is definitely in short supply, so it should not be wasted.

These considerations lead me to the following conclusions: 1) There should be a way for a reader to access the combined content of all these websites in a coherent form. 2) It is wasteful to spend time writing up stuff which is already available when there are so many mathematical topics which have not yet been entered.

The question is what to do. As I see it, the ideal solution would be to have a website which serves as a combined front-end for all these websites. If, for instance, one looks up "group" in such a front-end, one would see the Planet Math entry on "group", the Wikipedia entry on "group", the entry in the catalogue for algebraic systems, .... A benefit of this arrangement is that it takes some of the legal and ethical burden off of the websites which provide the information. The biggest drawback to this suggestion is that it takes a lot of work to set up and maintain such a website.

Another possible solution is for authors to have and unwritten agreement to cross-post their contributions. This way, there would be no ethical qualms about one website copying content from another since the author did the copying. The disadvantage I see to this proposal is the potential for the same article to evolve into different forms. What I mean is this: Suppose I post a theorem to both Planet Math and Wikipedia. As time goes on, the Planet Math entry will be revised in response to suggestions and corrections and the Wikipedia entry will be modified by other Wikipedia members. As I see it there is no feasible way of dealing with this which does not involve either a) having both versions of the same entry linked at the same website or b) having the author also agree to keep the two versions in agreement which would involve much work of the author and might might not sit well with the policies on some of the sites.

A third option, which I consider the most practical, is something like what Matte did with the "Ockham's razor" entry. That is to say, Planet Math could include links to entries on other websites providing that a) there were no legal issues and noone fconnected with the other website would be upset and b) it would be clear that these were simply links to definitions found elsewhere for the convenience of the reader so nobody would accuse anyone of trying to take credit for other people's work.

Personally, I am for the idea of providing links to all the Wikipedia articles on math. As long as it is clearly stated somewhere that these links are being provided for the convenience of the reader and that providing as linking to an article on another website is not to be construed as an endorsement or certification of correctness, I don't think there should be any problems. If the Wikipedia article is incorrect or incomplete, it seems that there would be two obvious courses of action --- either edit that article or write a more suitable article and post it here. In the latter case, both the link to the off-site article and the on-site article would appear next to each other in the listing, so the reader has a choice.

Ray

## Re: Occam's Razor, PM vs. WIkipedia

A few moments after pressing the "post" button, I had another thought whic might have been worth adding. When all an entry does is provide a link to a definition on another webpage, maybe the title could be something like "handwaving (Wikipedia)" or "handwaving (other websites)" so as not to confuse the unsuspecting reader.

## Re: Occam's Razor, PM vs. WIkipedia

> The question is what to do. As I see it, the ideal solution

> would be to have a website which serves as a combined

> front-end for all these websites. If, for instance, one

> looks up "group" in such a front-end, one would see the

> Planet Math entry on "group", the Wikipedia entry on

> "group", the entry in the catalogue for algebraic systems,

> .... A benefit of this arrangement is that it takes some of

> the legal and ethical burden off of the websites which

> provide the information. The biggest drawback to this

> suggestion is that it takes a lot of work to set up and

> maintain such a website.

Yes. However, a big problem with this is that

all entries (in the entire system) will no longer be interconnected

in a PM-way. There will also be multiple definitons for

some terms (T4 space, etc). This will be confusing for

the reader. Why could PM not be such a system?

> Another possible solution is for authors to have and

> unwritten agreement to cross-post their contributions. This

> way, there would be no ethical qualms about one website

Suppose we would obtain an agreement with Wikipedia that we

can copy articles from them provided that we give proper credit

to its authors. (How exactly this should be done, I have no idea.

Some edits are anonymous, and there does not seem to be any

version numbers for the entries.) Ideally, this would work

both ways. For example, Wikipedia (or we) could copy a very brief

entry from PM to Wikipedia. Over time it evolves, and PM can commit

(with suitable changes and verification of the facts)

the entry back into PM. This does involve some handiwork,

and the result is two entries with quite similar content. However,

it would spare a lot of unnecessary labor.

>

> A third option, which I consider the most practical, is

> something like what Matte did with the "Ockham's razor"

> entry. That is to say, Planet Math could include links to

> entries on other websites providing that a) there were no

> legal issues and noone fconnected with the other website

> would be upset and b) it would be clear that these were

> simply links to definitions found elsewhere for the

> convenience of the reader so nobody would accuse anyone of

> trying to take credit for other people's work.

I agree, but only as a temporary solution. In order for PM

to be truely interlinked, we need the content on PM.

Matte

## Re: Occam's Razor, PM vs. WIkipedia

I agree we can link for temporary purposes but this should not discourage someone of writting the same entry on his own and then removing the link. Since, if we link entries, well other people should be able to write a new entry on the same subject since it is useful to have many versions of the same subject. The more reference being given to the reader the easiest it will be to understand the subject.

Yann

## Re: Occam's Razor, PM vs. WIkipedia

I don't think quantity=quality, at least for the casual

user who simply wants to know what Occam's razor is.

One brief to-the-point entry I think is ideal. However,

before we arrive at such an entry, there might be many

PM-entries that over time are merged into this final entry.

## Re: Occam's Razor, PM vs. WIkipedia

There are a number of high-tech and low-tech considerations. "How

to do it?" should be considered _after_ "What to do?"

#

For now, I would advise being very careful *not* to copy text from

Wikipedia, because a person doing this by hand is very likely to do

it in a way that would make PlanetMath GNU FDL non-compliant. Just

because both sites use the same license does not mean that you can

arbitrarily copy text from one to the other. It is not clear to me

that PM or WP users are fully aware of this issue. So for now, I

think the only copying that people should do is stuff that falls

within "fair use", just like you would do from other non-FDL media

sources.

#

For the future (say, when we have a nice system that make it

possible to share content between PM and WP and preserving FDL

compliance for both sites, adequately giving credit to authors,

etc.), we have to ask what we want the sites to be like. I think PM

articles should reflect PM's collective goals. In my view, the main

goal is to be the premier web-based mathematical reference. WP

wants to be "The Free Encyclopedia" -- and I sometimes wonder

whether they don't mean "the" in an exclusive and singular sense.

But whatever! -- and more to the point, they want to be

comprehensive, and we should respect that. But this does not mean

that they will want to have all of the content we have here on PM on

their site. And vice versa.

#

However, there is absolutely no reason not to _link_ to WP. I would

encourage people to provide links to WP in their references. I

would also _discourage_ people from linking to non-free websites.

Books are of course almost universally non-free, and they are

another matter. But for websites, especially reference sites, it

is better not to link to a non-free site.

#

One other comment is about cross-posting. I think that is not

typically a good idea, just like cross-posting in usenet is not

typically a good idea. There are rare exceptions.

## Re: Occam's Razor, PM vs. WIkipedia

>

> For now, I would advise being very careful *not* to copy

> text from

> Wikipedia, because a person doing this by hand is very

> likely to do

> it in a way that would make PlanetMath GNU FDL

> non-compliant. Just

Indeed. In order to add material onto PM from Wikipedia

we should strengthen the PM-license so that derivatives should

reference PM. Right? See

http://en.wikipedia.org/wiki/Wikipedia%3ACopyrights

In principle, I have nothing against this. Also

legally, this would probably not require more than a consensus

among PM (active) users and changing the licence on the main page.

(We would not give users more rights, quite the contrary.)

Considering the benefits, I think this is something to consider.

In fact, since PM makes references to external resources in many

of its entries, I think that would be approriate. Now

someone can take just the text and leave all references out.

According to the above page, the main restrictions when copying

material from WP are:

* your materials in turn have to be licensed under GFDL,

* you must acknowledge the authorship of the article (section 4B), and

* you must provide access to the "transparent copy" of the material (section 4J). (The "transparent copy" of a Wikipedia article is its wiki text.)

[quoted from above page]

If we would strengthen the PM-license, the first

condition is met; the 2nd by adding a suitable

notice/link to the entry; and the 3rd by making

Version 1 of the entry the raw Wikipedia article. After this,

we still need a list of the WP writers behind the article. HOwever,

something like that can be probably be arranged in some way.

> etc.), we have to ask what we want the sites to be like. I

> think PM

> articles should reflect PM's collective goals. In my view,

> the main

> goal is to be the premier web-based mathematical reference.

PM doesn't have any goal ;-) But it would be nice to have some

mission plan.

> WP

> wants to be "The Free Encyclopedia" -- and I sometimes

> wonder

> whether they don't mean "the" in an exclusive and singular

> sense.

> But whatever! -- and more to the point, they want to be

> comprehensive, and we should respect that. But this does

> not mean

> that they will want to have all of the content we have here

> on PM on

> their site. And vice versa.

Competition is good.

> However, there is absolutely no reason not to _link_ to WP.

> I would

> encourage people to provide links to WP in their references.

.. but only if the WP entry contains additional information.

## Re: Occam's Razor, PM vs. WIkipedia

> > For now, I would advise being very careful *not* to copy text

> > from Wikipedia, because a person doing this by hand is very

> > likely to do it in a way that would make PlanetMath GNU FDL

> > non-compliant.

> Indeed. In order to add material onto PM from Wikipedia we should

> strengthen the PM-license so that derivatives should reference

> PM. Right?

No, the FDL already requires that derivative works acknowledge the

previous authors. The main problem I was referring to is that PM

users copying from WP by hand are not likely to be able to

incorporate the the WP history into the PM history.

> legally, this would probably not require more than a consensus

> among PM (active) users and changing the licence on the main page.

I don't think there is any reason to change the license.

> Now someone can take just the text and leave all references out.

Yes, and that is different from the rule of preserving history

sections that the FDL imposes. References sections aren't history

sections.

If Wikipedia requires people to do something that the GFDL does not

require, I'm not sure what to think. But none of the conditions you

listed were really seperate from those imposed by the GFDL.

> > etc.), we have to ask what we want the sites to be like. I

> > think PM articles should reflect PM's collective goals. In my

> > view, the main goal is to be the premier web-based mathematical

> > reference.

> PM doesn't have any goal ;-)

I disagree with you, but perhaps I should have been more clear. My

statement was essentially that *my* goal for PM is to help make it

the premier web-based mathematical reference.