Was Occam’s razor ever wrong?

Question

In brief, I am looking for an example where Occam’s razor favoured a theory A over another theory B, but theory B turned out to be a better description of reality later. But let me formulate some criteria of what I mean by that:

• First of all, as our modern perspective may be biased – e.g., due to didactical advances in the prevailing theory or new insights into historic experiments –, consider my criteria on qualities of theories to refer to historical scientific opinions and statements, as long as they can be considered to be based on reason (instead of, e.g., being strongly influenced by a religious bias).

• At a given time, two theories (A and B) were comparably good at describing the same aspect of reality as it was observable at that time. They need not have been perfect descriptions of available observations, but they should not have been so far-off that they were applicable only to special cases or none at all.

• Occam’s razor was reasonably invoked in a scientific dispute in favor of theory A. This invocation needs neither to have happened by name nor in a peer-reviewed publication (if such existed at that time at all). I am also interested in, but do not prefer, cases in which adherents of both theories invoked Occam’s razor (or similar) to argue against the respective other theory.

• At a later time, theory B or a reasonably small modification thereof turned out to be a better description of reality than theory A. Alternatively, theory B is still used today for some aspects, while theory A isn’t. Theory B does not need to be the prevailing theory today.

I am asking out of curiosity. I am very well aware that the existence of such an example does not invalidate Occam’s razor.

Answer 1

By way of throat-clearing, what is Occam's razor? John Baez has a useful essay giving the history and some examples. William of Ockham's original formulation was

Entities should not be multiplied unnecessarily.

In other words, don't assume the existence of something unless there is good evidence for it. Again quoting Baez, "In physics we use the razor to shave away metaphysical concepts." The canonical example is discarding the ether. Newton's absolute time and space, mechanical explanations for gravity, and classical trajectories for particles all have felt the edge of the razor.

But Baez also mentions a famous failure of this version of Occam's razor:

Mach and his followers claimed that molecules were metaphysical because they were too small to detect directly.

Mach's point is that the molecular hypothesis is just unnecessary decoration layered on top of empirical regularities (laws of Dalton and Gay-Lussac in chemistry, Boyle's law) that work just fine without the added ornaments. We have (or so Mach would claim) an analogy:

ether : relativity = molecules : (chemistry+physics)

Occam's razor is often strengthened to the rule of simplicity: in one formulation (taken from Baez's essay),

The simplest explanation for some phenomenon is more likely to be accurate than more complicated explanations.

Often people say Occam's razor when they really mean the rule of simplicity. The obvious problem with the rule of simplicity is its subjectivity. A sterling example is the heliocentric hypothesis.

To Copernicans in the 16th C. (Galileo, Kepler, a few others) heliocentricity was clearly simpler. In this period, the contest was between true heliocentricity and so-called geoheliocentric hybrids: the planets revolve around the sun which revolves around the earth. (Tycho's geoheliocentric system was the most famous, but not the only one.)

To modern eyes, heliocentricity looks obviously simpler. But advocates of geoheliocentricity deployed two powerful arguments from simplicity.

• Heliocentricity was inconsistent with physics as then understood. Kepler responded by inventing his own celestial physics, with three different forces guiding each planet, plus the force of gravity, which had nothing to do with the planet's orbit.
• The lack of detectable stellar parallax implied enormously greater distances to the fixed stars than with a geoheliocentric theory. The apparent sizes of stellar disks (an artifact of optics, not understood at the time) then implies that every other star is way bigger than the sun. Tycho first put forward this argument, quite convincing to many of his contemporaries. (See this article by Chris Graney for more details.)

Simplicity is not simple.

Answer 2

In the 1960s the mathematical structure of Turing degrees was conjectured to be rather simple and homogeneous. This was consistent with what was known at the time. It later turned out that the opposite is true in a sense: the Turing degrees are as complicated as can be.

Details in Ambos-Spies and Fejer, History of degree theory.

Answer 3

This is too near a miss to my criteria not to mention it for completeness.

In the early years of molecular genetics, it was only known that the genetic code used an alphabet of four different bases and encoded twenty amino acids. This sprouted several hypotheses on the design of the code, which were all able to explain what was experimentally known at that time. However, some of these yielded the correct number of amino acids without further ado, i.e., they did not require this number as a parameter and thus were slightly favourable as per Occam’s razor.

For example, Crick et al. considered comma-less codes that featured an immunity against frameshift errors. They showed that, given a codon length of three, there exist codes that can encode twenty amino acids and that it was impossible to have a code encoding more amino acids.

Code designs that automatically yielded the correct number of amino acids met particular interest at that time, however, when the real genetic code was discovered it turned out to be of a different type: you could encode up to 63 amino acids with this general design.

Now, I can find no contemporary invocation of Occam’s razor. Crick even cautioned against it (due to natural selection not being bound to yield the most effective mechanism):

While Ockham's razor is a useful tool in the physical sciences, it can be a very dangerous implement in biology. It is thus very rash to use simplicity and elegance as a guide in biological research.

Still, the razor was mentioned in retrospect, e.g., by Woese:

The details of Gamow’s coding theories (there was more than one) are no longer of interest, for in their specifics his models were wrong. However, his Occam’s razor approach and the impact his thinking had on his contemporaries was a major factor in moulding how gene expression was perceived.

[…]

But without doubt, the most memorable and influential theory to emerge from this new chapter in the code’s history (in that it retained a biological semblance and theoretical panache) was Crick’s famous “comma-free code”—one of those wonderful, but ephemeral triumphs of intellect over reality (to which theoreticians are predisposed). The comma-free code remained based upon the hopeful presumption that the code might be inferred from first principles of some kind.

Answer 4

The best example I can think of is steady-state versus big bang cosmology.

In steady-state models the universe is homogeneous in space and in time. In big bang models it's homogeneous in space but not in time. Big bang models have far more parameters than steady-state models because there are so many things that plausibly could have been different in earlier eras.

Models of both types were taken seriously until the early 1990s when COBE found anisotropies in the cosmic microwave background. ΛCDM, a big-bang model, has enough parameters to fit the CMB power spectrum (which, it must be noted, is vaguely elephant shaped). Steady-state models can't reproduce it, so they're wrong.

Answer 5

To respond to the requested example where Occam’s razor favoured a theory A over another theory B, but theory B turned out to be a better description of reality later I would mention the history of real analysis which has been based since the 1870s on theory A (for Archimedean), which involves the Archimedean complete ordered field. An older/newer approach involves a theory B (for Bernoullian), working with infinitesimals as Johann Bernoulli did. It turns out that while the background continuum is easier to describe in the A-track, the procedures are easier to work with in the B-track. For example, instead of defining continuity of a function by requiring that for every epsilon greater than zero there should be a delta greater than zero such that the students already are falling asleep or taking calming pills, you can just follow Cauchy (1821) in requiring that every infinitesimal change $\alpha$ in input must produce an infinitesimal change in output: $f(x+\alpha)-f(x)$ is infinitesimal.