29. The Bayesian Belief: 5. The Problem With Induction

I have been discussing Stephen Stigler’s suggestion that Thomas Bayes’ 1763 paper was connected to David Hume (see The Other Title of Bayes’ Essay).  Specifically, Stigler has argued that Bayes’ work may have been initially written as a response to Hume’s essay regarding religious miracles.  Evidence to support this position comes from the alternate title of Bayes’ 1763 publication as well as Richard Price’s subsequent challenge to Hume.

The alternate title of Bayes’ essay was A Method of Calculating the Exact Probability of All Conclusions founded on Induction.  To Stigler, this title suggests that Bayes was focused on induction and, by extension, the work of Hume.

We had previously discussed induction in an earlier post that reviewed differential diagnosis. (see Sick Versus Slick.  Testosterone, Differential Diagnosis, and Marketing).   Diagnosis is the process of measuring and assessing symptoms in order to arrive at ideally one underlying disease state or condition that best accounts for these symptoms.  Diagnosis is  inferring a cause (disease) from events (symptoms).  In a nutshell, this is induction.

A good diagnostician, however, is never absolutely confident in his or her diagnosis.  There will always remain other unknown factors at play that might also account for the symptoms observed.  By definition, unknown causes are just that — unknown.  We can never truly eliminate a cause of which we have no knowledge and we can never assume that we have knowledge of all causes.  And it is largely because of David Hume that doubt and uncertainty play an important role in the scientific process and the reason why this process is inherently skeptical in nature.

Hume argued that there are two kinds of human reasoning and inquiry:  (1) the relation of ideas and (2) matters of fact. The relation of ideas concern those concepts within logic and mathematics that can be proven without need for evidence.  Statements, such as “five times ten equals fifty” are demonstrably certain and can be established by reason alone.  Matters of fact, on the other hand, are empirical in nature (that is, gained through observation or through our senses).  Hume argued that matters of fact are always open to revision and we have no proper justification in assuming that events we observe presently can inform us about any event we have not yet observed.  Hume, of course, understood that we do make inferences and that we observe a causal connection between events — fire burns, food nourishes, and oxygen is required to sustain life.  But any fact that is gained through experience can be altered by some other, as of yet unseen, future possible experience.  For Hume, the presumed link between past and future is based merely on habit or custom as well as the strong assumption that conditions that existed in the past must continue into the future.  This, according to Hume, is simply a convenient device and cannot prove, or ever prove, that experiential events are causally linked.

What is curious about connecting the word induction to Hume is that Hume, himself, did not label the inference of cause from effects as induction and Hume did not use the word induction in his essays. Similarly, in Bayes’ publication, no reference is given to Hume nor is the term induction mentioned beyond the recently discovered alternate title. One wonders, if Bayes’ article was meant to be a direct challenge to Hume, then why be so coy about this within the essay?

Ultimately, the intent of Humes’ work was to establish a clear understanding of the limits of human reasoning.  Hume’s philosophy raised doubt to whether the underlying principles of nature or any empirical event could ever be known with confidence.  However, in contrast to Hume’s  skepticism, a contemporary of Bayes and Fellow of the Royal Society did offer a more positive and forward method of understanding natural phenomenon and discovering its causes.  That contemporary was Isaac Newton.  Unlike Hume, Newton did discuss induction to describe the process of empirical discovery and Bayes did reference Newton in Bayes’ 1763 essay.

In the next several posts I will suggest that it may have been Newton, and not Hume, that Bayes (or Price) was addressing with the alternative title of Bayes’ essay.

29. Map