I've recently been reading Stephen Law's new book - Believing Bullshit (do check out the Amazon link if only for the comedy review Stephen mentions on his blog).
I'm only about a quarter of the way in, and finding it enjoyable, but I thought I'd pick up on one bit of the text I'm not completely sure I agree with. It's about what makes a scientific theory that is well supported by evidence.
Law says that a theory needs to be:
- clear and precise
- surprising and
- true
Now, I'm all in favour of clear and precise theories - if you make a waffly prediction it's hard to check how well the evidence fits your prediction. I've mentioned before how important it is from a Bayesian point of view, and even if you don't want to analyse your data that way I think people can see the benefit of a theory that makes precise predictions. A precise prediction is essentially easier to test, and testing things is pretty much what science is about.
I'm also all in favour of true theories. That's not terribly surprising.
What is surprising to me, is that Law says that a theory should be surprising in order for us to find strong confirmation for it. Law gives the example of the idea that fairies cause trees to grow more quickly in the summer months. This theory makes an unsurprising prediction, because we expect trees to grow more quickly in the summer months anyway. It's then argued that because we expect this to happen anyway, it's not a good theory.
This has problems. Suppose we actually had some complex idea for some phenomenon - a good example might be epicycles to describe the orbits of planets. Now, Copernicus comes along and has a simpler idea, a clearly better idea, but it's not a surprising idea because it predicts exactly the same apparent motions of the planets across the sky as we already had. An awful lot of theories have unsurprising predictions, but fortunately they frequently also have surprising ones - and these provide us with the best way to come up with an experiment to figure out whether they or another theory is best. However, given two theories that make the same predictions, labelling the first theory better because it made the surprising prediction and the later theory worse because the first one came up first and so now its predictions are unsurprising - well that's just not a good way to distinguish between the theories.
Instead, given theories that have the same predictions there's an old rule of thumb for deciding between them - Occam's Razor. The fairy theory isn't bad because it makes the same predictions as another theory - it's bad because it adds in fairies when we don't need them. A theory can be good while still being unsurprising, as long as it gives a better explanation.
That all said, the book's great. Maybe I'll review it properly when I've got to the end, and post occasional comments on the way to the end as well, but for now I think it's well worth picking up if critical thinking's your thing.
Leave a comment