Today let’s clean up some detritus from past columns. I have used the words “theory,” “hypothesis,” and “conjecture” several times in a scientific sense rather than the commonly used senses. In common use, these terms are often blurred and even considered to be synonyms. This blurring causes mis-understanding and unnecessarily argumentative discussions. My Merriam-Webster dictionary lists six definitions of “theory,” and they range from the generally accepted scientific meaning to something as weak as a conjecture or guess. That is not what I mean when discussing decision theory, but in common language we often hear comments like, “I have a theory about why people buy SUVs.” Such a statement is not backed up by experiments, rational presentation, predictions, etc. It merely represents one person’s guess to explain a given phenomena.

So what is a good definition of “theory?” Different sources list somewhat different definitions, but I have combined several to get this definition. To be clear, I will refer to “Scientific Theory.”

Scientific Theory: a set of statements or principles devised to explain a group of observations of natural phenomena, especially one that has been repeatedly tested and/or is widely accepted and has been used to make predictions about related natural phenomena. Theories are the highest level of science. They can occasionally lead to laws when the theory is so accurate that no exceptions or deviations are known. By its nature, science cannot establish truths absolutely in the mathematical sense (or in the sense that some religions consider revealed truths). Theories can often be the source of compact mathematical expressions such as e = mc2, which compress many observations and experimental results into an extremely compact form.

A feature of scientific theories is that they can be used to make predictions and explain things about the natural world. By definition, then, scientific theories cannot be used to prove or disprove anything that is supernatural. If something is thought to be supernatural, but proves to be amenable to scientific analysis, then the original attribution was incorrect.

Scientific theories include Quantum Theory, Relativity Theory, Atomic Theory, Genetic Theory, and Darwin’s Theory of Evolution.

Consider atomic theory. No one can see an atom. However, acceptance of the existence of unseen atoms is based on scientific evidence. You can say you believe in atoms in the colloquial sense, but in reality, belief is not necessary. And for educated people, belief in atoms does not constitute a religious statement.

Here’s an example of how we might build up a theory. We could start with the informal observation the crows are black (for some reason, considering that crows or ravens are black is a common example).

I say every crow I’ve seen is black. So I conjecture that all crows are black. This leads directly to the prediction that other observers who see crows will see black ones. I ask around and record the observations of many people. All observed crows are indeed black. So my conjecture is well on its way to progressing through hypothesis to theory. I publish the results in a reputable journal, and at next year’s meeting of the international crow society, another scientist reports she has independently repeated the observations with 1000 observers who on the average saw 10 crows each of the 10,000, 9,999 were definitely black, but one was uncertain. The theory stands.
However, the crow watchers are a stubborn, precise group. Another scientist is concerned by the one uncertain measurement. He works for several years and demonstrates that a genetic failure can produce albino crows. His observations are published. This does not invalidate the original theory, but the black crow theory must now be modified to include the new observations.

Of course, some scientists argue that albino crows should not be considered to be real crows. This leads to the kind of erudite exchanges that enliven scientific meetings.

So we had informal observations that resulted in a conjecture that led to a hypothesis, and through a process of prediction and experimentation, led to a theory, which was robust enough that, with mild modification, it was able to incorporate measurements which were much more accurate than the original data. However, we doubt that even with the modifications, the theory will achieve the level of a scientific law that has never been observed to give faulty results such as e = mc2.

In following columns, I will discuss “conjecture” and “hypothesis.”

For those who wish to delve further into decision theory without wading through a lot of equations, I have posted a tutorial on elementary decision theory. It shows examples of faulty physicians’ diagnoses (important for those considering surgery) and how to evaluate anti-terrorist activities (important for everyone). That tutorial can be found here.