Friday, July 25, 2008

Occam's razor

Occam's razor (sometimes spelled Ockham's razor) is a principle attributed to the 14th-century English logician and Franciscan friar William of Ockham. The principle states that the explanation of any phenomenon should make as few assumptions as possible, eliminating those that make no difference in the observable predictions of the explanatory hypothesis or theory. The principle is often expressed in Latin as the lex parsimoniae ("law of parsimony" or "law of succinctness"): "entia non sunt multiplicanda praeter necessitatem", roughly translated as "entities must not be multiplied beyond necessity".

This is often paraphrased as "All other things being equal, the simplest solution is the best." In other words, when multiple competing theories are equal in other respects, the principle recommends selecting the theory that introduces the fewest assumptions and postulates the fewest entities. It is in this sense that Occam's razor is usually understood.

-

Blogger BadRabbi gives the following example: (Blogger Bad Rabbi?? lol, love the sound of it)

Suppose you wake up one winter morning and notice that the ground is covered with snow. You wonder what happened at night when you were sleeping. The possibilities are as follows:

  1. It snowed last night and the ground was covered with snow.
  2. A group of professionals with snow making machinery came in when you were sleeping and worked all night to fill the streets with artificial snow.
  3. It started snowing last night, but the snow fall was brief. Following the brief period of snow, the professionals came and augmented the snow with powder.

If you had no other data than the information given to you above, which choice would you consider more likely?

You might say that choice #1 is more likely since it only assumes the occurrence of a common and natural process, namely the occurrence of snow in winter time. It is simpler to assume that the ground would be covered by natural snow than man made snow, given that we routinely see natural snow in the winter. You might say that choice #1 is more parsimonious, and thus more likely to be true.

But suppose now that you lived in Southern California. Here, the occurrence of snow is much less common. It is very uncommon to see snow in the Southern California. Perhaps in anticipation of an upcoming motion picture filming, a crew equipped with snow making machinery really made snow to cover your street. Given the information of your location makes possibility 1 and 2 somewhat equally parsimonious. Possibility #3 is not parsimonious since it assumes both snow in a warm climate and existence of machinery. All else being equal, possibility #3 is not parsimonious.

What we learn from the above example are:

  1. Hypotheses which make the least number of assumptions are more likely to be true, all else being the same.
  2. When further information is provided, the parsimony of a given hypothesis is altered.

Now let’s consider one more example:

Suppose I wish to know how many inches of snow are on the ground in place X in the winter. The choices are:

  1. 1 inch
  2. 2 inches
  3. 3 inches

Having no further information, and knowing nothing about place X, can we decide which is the most likely answer? Can we by parsimony argue that choice #1 must be the correct answer since it is the simplest? If not, why not?

Notice here, that the choosing “1 inch” of snow is not any more parsimonious than the other choices.

-

(thanx for the concept)

-

I just read on the linked wiki page (where the first paragraphs are from) that this is used "in some medical schools, "When you hear hoofbeats, think horses, not zebras"."

Ha. And in therapy we learn "When you hear hoof beats, think not horses but zebras". Awesome.

No comments: