 |
BAYES' THEOREM |
See also ...
( Home → Science → Probability → Bayes )
|
Bayes' theorem lies at the heart of the computation of conditional probabilities. Conditional probabilities take into account the fact that the occurrence of one event may depend on the occurrence of one or more other events. Bayes' theorem expresses the relationship between conditional probabilities when some events are contingent on others. |
Conditional Probability
In many situations, an experiment may consist of a number of components that can be considered to be independent. For example, consider an experiment in which two dice are rolled. If the two throws of the dice are random, then the probability that the second die lands as a six (or any other number) is completely independent of the outcomes of the throw of the first die, and is equal to 1/6.
However, consider now two events A and B. Suppose that event B occurs first. If A and B are independent, then the probability that A occurs does not depend on the probability that B occurs. However, there may be some circumstances in which the probability that A occurs is contingent on whether the event B has occurred or not. That is, if we know that the event B has occurred, then that knowledge influences the probability that A occurs.
In this case, we talk of the conditional probability that A occurs, given that event B has occurred. This conditional probability is written as follows:
and is usually termed the “conditional probability of A given B”.
The conditional probability of A given B is evaluated from the following expression:
(A)
The plausibility of this formula can be confirmed by rewriting it slightly as
This makes the not unreasonable assertion that the probability of A and B occurring is equal to the probability of B occurring, multiplied by the probability of A given that B has occurred. One can also write this in the form
(B)
This result follows by swapping A and B around in expression (A). Note that if A and B are independent then
Similarly, if A and B are mutually exclusive:
Example: Two dice are rolled and the sum of the two numbers is odd. What is the probability that the sum is less than 8? (This example is taken from the book Probability and Statistics, by Morris DeGroot, 1989).
We begin by letting A be the event that the sum is less than 8, and B be the event that the sum is odd. Now, in this case,
Now, there are two ways in which the sum can be 3 or 11, four ways in which the sum can be 5 or 9, and six ways in which the sum can be 7. Therefore,
Therefore the probability that we seek, namely the probability that the sum is less than 8 given that the sum is odd, is equal to
.
Bayes’ Theorem
Before we state Bayes’ theorem, we need an intermediate result that can be derived by considering the following diagram.
This figure shows a series of events A1, A2 … that are disjoint (i.e. there is no overlap between the events) and which span the entire sample space S. Now, it can be seen from this diagram that we can write the event B as
With this in mind, it can be shown (though it is fairly obvious from the figure) that
Substituting this into equation (A) and using (B) gives us Bayes’ theorem:
Example: Two coins are in a box. One coin is fair, one coin has two heads. A coin is selected at random and tossed, and it lands heads. What is the probability that the coin chosen is the fair coin? (Again, this example is taken from the book Probability and Statistics, by Morris DeGroot, 1989)
Let us define some events for use in Bayes’ theorem. We know that we need to use Bayes’ theorem because the probability we seek is a conditional one – the probability that the coin is fair, given that it landed heads up. So here goes …
Let A1 be the event that the coin is fair;
Let A2 be the event that the coin has two heads;
Let H be the event that a head is obtained when the coin is tossed.
Note here that the events A1 and A2 cover the whole of the sample space, since of the choices we can make, the chosen coin must be either the fair one or the double-headed coin.
With these definitions, the probability that we seek is
According to Bayes’ theorem, our required probability is given by
Now, the information in the specification of the problem tells us that
These arise because there is an equal probability of choosing either coin, and the probability of getting a head with the fair coin is 1/2. Therefore
Therefore the probability that the coin tossed is the fair one is 1/3.
Two further examples of the application of Bayes’ theorem are given in the next article. The reader will note that one of the keys to applying Bayes’ theorem is to define carefully the events A and B, so that the desired result is obtained.