Was Occam’s razor ever wrong?

In brief, I am looking for an example where Occam’s razor favoured a theory A over another theory B, but theory B turned out to be a better description of reality later. But let me formulate some criteria of what I mean by that:

• First of all, as our modern perspective may be biased – e.g., due to didactical advances in the prevailing theory or new insights into historic experiments –, consider my criteria on qualities of theories to refer to historical scientific opinions and statements, as long as they can be considered to be based on reason (instead of, e.g., being strongly influenced by a religious bias).

• At a given time, two theories (A and B) were comparably good at describing the same aspect of reality as it was observable at that time. They need not have been perfect descriptions of available observations, but they should not have been so far-off that they were applicable only to special cases or none at all.

• Occam’s razor was reasonably invoked in a scientific dispute in favor of theory A. This invocation needs neither to have happened by name nor in a peer-reviewed publication (if such existed at that time at all). I am also interested in, but do not prefer, cases in which adherents of both theories invoked Occam’s razor (or similar) to argue against the respective other theory.

• At a later time, theory B or a reasonably small modification thereof turned out to be a better description of reality than theory A. Alternatively, theory B is still used today for some aspects, while theory A isn’t. Theory B does not need to be the prevailing theory today.

I am asking out of curiosity. I am very well aware that the existence of such an example does not invalidate Occam’s razor.

• Mathematical theory since the 1870s has recognized that there is more than one variety of infinite quantity; for example, that the infinite number that counts the integers is essentially different from the infinite number that counts the points on a line; in fact, the family of different infinite quantities is itself infinite. To a mathematician of the 18th century or earlier, this would have seemed to be a bizarre and unnecessary proliferation of entities, but it is now universally recognized as correct. Dec 10, 2014 at 21:55
• No, because it deals with probabilities so you could look at overall but not one specific isolated case. Dec 17, 2014 at 0:14
• I often find that after inspecting the results of an experiment and the available controls, simpler theory X appears more likely than the more complicated theory Y (where both appear to be possible explanations). But upon conducting additional control experiments that probe more variables, it turns out Y is in fact true and X is not. Does this count? Jun 10, 2015 at 3:36
• I'm not sure I understand. Are you asking whether there has ever been a theory that was more parsimonious than a contending theory but empirically falsified, after a period in which both theories were equally plausible empirically, while the contender survived falsification? I think this would describe many debates between an original and parsimonious theory and a less parsimonious but 'realistic' modification of the original theory. Oct 27, 2017 at 9:25
• @henning: I think this would describe many debates between an original and parsimonious theory and a less parsimonious but 'realistic' modification of the original theory. – I would be surprised if anybody argued with Occam’s razor in this case. Someone could argue that the modification is factually wrong or irrelevant, but you would not argue that the modification is unlikely to be correct just because the original is simpler – which would essentially be arguing that the modification is incorrect because it is a modification. Oct 27, 2017 at 11:07

By way of throat-clearing, what is Occam's razor? John Baez has a useful essay giving the history and some examples. William of Ockham's original formulation was

Entities should not be multiplied unnecessarily.

In other words, don't assume the existence of something unless there is good evidence for it. Again quoting Baez, "In physics we use the razor to shave away metaphysical concepts." The canonical example is discarding the ether. Newton's absolute time and space, mechanical explanations for gravity, and classical trajectories for particles all have felt the edge of the razor.

But Baez also mentions a famous failure of this version of Occam's razor:

Mach and his followers claimed that molecules were metaphysical because they were too small to detect directly.

Mach's point is that the molecular hypothesis is just unnecessary decoration layered on top of empirical regularities (laws of Dalton and Gay-Lussac in chemistry, Boyle's law) that work just fine without the added ornaments. We have (or so Mach would claim) an analogy:

ether : relativity = molecules : (chemistry+physics)

Occam's razor is often strengthened to the rule of simplicity: in one formulation (taken from Baez's essay),

The simplest explanation for some phenomenon is more likely to be accurate than more complicated explanations.

Often people say Occam's razor when they really mean the rule of simplicity. The obvious problem with the rule of simplicity is its subjectivity. A sterling example is the heliocentric hypothesis.

To Copernicans in the 16th C. (Galileo, Kepler, a few others) heliocentricity was clearly simpler. In this period, the contest was between true heliocentricity and so-called geoheliocentric hybrids: the planets revolve around the sun which revolves around the earth. (Tycho's geoheliocentric system was the most famous, but not the only one.)

To modern eyes, heliocentricity looks obviously simpler. But advocates of geoheliocentricity deployed two powerful arguments from simplicity.

• Heliocentricity was inconsistent with physics as then understood. Kepler responded by inventing his own celestial physics, with three different forces guiding each planet, plus the force of gravity, which had nothing to do with the planet's orbit.
• The lack of detectable stellar parallax implied enormously greater distances to the fixed stars than with a geoheliocentric theory. The apparent sizes of stellar disks (an artifact of optics, not understood at the time) then implies that every other star is way bigger than the sun. Tycho first put forward this argument, quite convincing to many of his contemporaries. (See this article by Chris Graney for more details.)

Simplicity is not simple.

• Also, Popper argued what makes a hypothesis "simple" is how easily it's falsified, e.g. it takes more points to refute an ellipse them a circle, hence the term "Popper's chopper". So people refine the razor to fit their newer ideas.
– J.G.
Nov 19, 2018 at 7:50
• Curiously, from the standpoint of predictive accuracy, Kepler’s elliptical orbits were much less important than some of his other innovations, among them technicalities never mentioned in brief accounts. As Curtis Wilson remarks in an article in the Encyclopedia of the History of Astronomy, for some time all observation could tell you is that the orbits were oval. Nov 20, 2018 at 5:25

In the 1960s the mathematical structure of Turing degrees was conjectured to be rather simple and homogeneous. This was consistent with what was known at the time. It later turned out that the opposite is true in a sense: the Turing degrees are as complicated as can be.

Details in Ambos-Spies and Fejer, History of degree theory.

• I have a few additional examples, but I wondered whether the phrasing of the question excluded mathematics. (Was there a theory built and favored by the community on the assumption of the homogeneity of the degrees? The closer I can think of would be intensive study of large cardinal axioms that turn out to be inconsistent. -- Reinhardt cardinals in ZFC do not qualify.) Nov 1, 2014 at 0:52
• I cannot fully evaluate your example for now, but I am not looking for something not turning out to be more complicated than it had been thought to be. Occam’s razor does not plainly favor the simplest solution, but the simplest of two solutions that are equally good at describing reality. Can you elaborate a little bit more how your answer fits into this, even if it does not fit perfectly. (BTW: Sorry for the late reply, I somehow totally forgot this.) Nov 1, 2014 at 21:27
• @AndresCaicedo: I would be very surprised about an answer coming from mathematics, because even as experimental mathematics does exist, I am not aware that it produces sufficiently general hypotheses or theories. Inconsistent axiom sets may be an interesting thing to look at though (after all, one could argue that the closest thing mathematics has to a scientific theory is that certain axioms are fulfilled by real life): Has it ever been argued that a set of axioms is preferable because it’s simpler and this set turned out to be inconsistent afterwards? Nov 1, 2014 at 21:36

This is too near a miss to my criteria not to mention it for completeness.

In the early years of molecular genetics, it was only known that the genetic code used an alphabet of four different bases and encoded twenty amino acids. This sprouted several hypotheses on the design of the code, which were all able to explain what was experimentally known at that time. However, some of these yielded the correct number of amino acids without further ado, i.e., they did not require this number as a parameter and thus were slightly favourable as per Occam’s razor.

For example, Crick et al. considered comma-less codes that featured an immunity against frameshift errors. They showed that, given a codon length of three, there exist codes that can encode twenty amino acids and that it was impossible to have a code encoding more amino acids.

Code designs that automatically yielded the correct number of amino acids met particular interest at that time, however, when the real genetic code was discovered it turned out to be of a different type: you could encode up to 63 amino acids with this general design.

Now, I can find no contemporary invocation of Occam’s razor. Crick even cautioned against it (due to natural selection not being bound to yield the most effective mechanism):

While Ockham's razor is a useful tool in the physical sciences, it can be a very dangerous implement in biology. It is thus very rash to use simplicity and elegance as a guide in biological research.

Still, the razor was mentioned in retrospect, e.g., by Woese:

The details of Gamow’s coding theories (there was more than one) are no longer of interest, for in their specifics his models were wrong. However, his Occam’s razor approach and the impact his thinking had on his contemporaries was a major factor in moulding how gene expression was perceived.

[…]

But without doubt, the most memorable and influential theory to emerge from this new chapter in the code’s history (in that it retained a biological semblance and theoretical panache) was Crick’s famous “comma-free code”—one of those wonderful, but ephemeral triumphs of intellect over reality (to which theoreticians are predisposed). The comma-free code remained based upon the hopeful presumption that the code might be inferred from first principles of some kind.

To respond to the requested example where Occam’s razor favoured a theory A over another theory B, but theory B turned out to be a better description of reality later I would mention the history of real analysis which has been based since the 1870s on theory A (for Archimedean), which involves the Archimedean complete ordered field. An older/newer approach involves a theory B (for Bernoullian), working with infinitesimals as Johann Bernoulli did. It turns out that while the background continuum is easier to describe in the A-track, the procedures are easier to work with in the B-track. For example, instead of defining continuity of a function by requiring that for every epsilon greater than zero there should be a delta greater than zero such that the students already are falling asleep or taking calming pills, you can just follow Cauchy (1821) in requiring that every infinitesimal change $\alpha$ in input must produce an infinitesimal change in output: $f(x+\alpha)-f(x)$ is infinitesimal.

The best example I can think of is steady-state versus big bang cosmology.

In steady-state models the universe is homogeneous in space and in time. In big bang models it's homogeneous in space but not in time. Big bang models have far more parameters than steady-state models because there are so many things that plausibly could have been different in earlier eras.

Models of both types were taken seriously until the early 1990s when COBE found anisotropies in the cosmic microwave background. ΛCDM, a big-bang model, has enough parameters to fit the CMB power spectrum (which, it must be noted, is vaguely elephant shaped). Steady-state models can't reproduce it, so they're wrong.

• Has Occam’s razor ever been invoked as an argument for steady-state models (vs. big bang)? Also, aren’t there far more arguments for the big bang than ΛCDM, which is just a specific theory featuring it? Finally, when your only argument for a theory is that it has enough parameters to be fit to anything, this seems like a big red flag to me. Aug 14, 2020 at 8:41