Suppose you hear a vase break. You go to the living room and there's little Jimmy, next to the broken vase. He says that the cat did it.
But you didn't even have a cat in this household. So, Jimmy says, some cat must have sneaked in from the outside. You remember closing all the doors and windows, so he says it must have entered the house before you closed them.
Next you note that Jimmy is allergic to cats. You don't see him sneezing. So, he says, it was a Sphynx cat, which has no fur.
You go look for the cat, but you can't find it. Jimmy says it already run away by now. This conversation, in principle, could go on forever.
Every objection you would make, can be countered with an additional hypothesis, for example at some point in the conversation Jimmy could argue that the cat somehow unlocked the doors you remember locking or even that the cat is just invisible. Before that would happen, you would probably tell Jimmy to stop with these explanations and draw the much simpler conclusion that HE broke the vase. If you would do so, you would be acting in accordance with a philosophical rule of thumb, called "Occam's razor" - named after the sholastic philosopher William of Ockham.
The rule can be summed up as following: "always pick simplest solutions, since they are more likely to be correct than complex ones. " But, why do we think that? Why would we think that simple solutions are more likely to be correct?
In the history of philosophy, there were different attempts to justify this kind of reasoning. Prior to the 20th century, for example, it was commonly believed that nature itself is simple. So, of course, simpler hypotheses about nature are more likely to be true.
This view often drew from theology. Aquinas, for example, wrote that "nature does not employ two instruments where one suffices. " Perhaps the most well-known instance of such a justification can be found in Copernicus' rejection of the Ptolemaeus' geocentric model of the solar system - which was a quite complex model - in favor of a simpler model.
Copernicus adopted Plato's conception of celestial bodies as being perfect and thereby they would have to display a perfect and simple kind of motion, one that is not to be found in the somehow chaotic Ptolemaeus' geocentric model. One could therefore say that one of his reasons for developing an alternative model of the solar system was aesthetic. Although it was by no means its only reason.
Since then, philosophers presented justifications for Occam's razor that we might find more convincing - one being a direct result of basic probability theory. By definition, all assumptions introduce possibilities for error. If an assumption does not improve the accuracy of a theory, it's only effect is to increase the probability that the overall theory is wrong.
Let's put it in other terms - if we get back to this Jimmy and the broken vase: with every additional explanation that he offered, the cat theory increased in complexity - meaning, it rested on an increasing number of conditions that had to be met in order to the cat to break the vase. The more conditions had to be met for something to happen, the less likely the theory will seem. Another justification for Occam's razor rests on the observation that if we manage to simplify theoretical models in science, they get better at predicting future events.
Simplicity, in this view, is a counterweight or counterbalance for the problem of overfitting our theory to a particular set of data. If we manage to deduce a simpler version of our theory, we can get rid of all the statistical noise in our experiments. Arguably, simpler models capture the underlying structure of the world better.
Occam's razor continues to be an effective method of reasoning, both in science and everyday life.
