Bayes Rule

Bayes rule states that

P(A|B)=P(B|A)P(A)/(P(B|A)P(A)+P(B|A')P(A'))

Given one set of conditional probabilities, it enables one to calculate conditional probabilities with the reverse conditioning. The two way table below is presented as a way to easily reconstruct Bayes Rule, which should be easier than memorizing it.

A Venn diagram for two events divides the sample space into four disjoint subsets: AB, A'B, AB', A'B'. The probabilities of these four events can be concisely represented with a square:

               A      A'                           
             _____________                                        
            |      |      |
         B  |  x   |  y   |  x+y=P(B)              
            |______|______|                                    
            |      |      |
         B' |  z   |  w   |  z+w=P(B')                              
            |______|______|___                                         
              x+z=   y+w= |
              P(A)   P(A')|  1
In accordance with the row and column labels, this square means that P(AB)=x, P(A'B)=y, P(AB')=z, and P(A'B')=w. P(A)=x+z and P(B)=x+y as indicated above. P(A|B) = P(AB)/P(B) = x/(x+y); and the other conditional probabilities can be represented in a similar manner.

There are four unknowns (x, y, z, and w) in the above square, in terms of which all the probabilities we are interested in can be calculated. One constraint is that x+y+z+w=1 (P(S)=1); hence it is reasonable that three further equations would enable us to solve for x, y, z, and w, hence all probabilities. In the case of Bayes rule, we will be given P(A)=(x+z), P(B|A)=x/(x+z), and P(B|A')=y/(y+w).

Example (Bayes rule): P(A)=.5, P(B|A)=.2, P(B|A')=.6; P(A|B)=?
From P(A')=1-P(A) (this is the fact x+y+z+w=1), the above square becomes:

               A      A'                           
             _____________                                        
            |      |      |
         B  |  x   |  y   |  x+y=P(B)              
            |______|______|                                    
            |      |      |
         B' |  z   |  w   |  z+w=P(B')                              
            |______|______|___                                         
                          |
               .5     .5  |  1
Then P(B|A)=.2 becomes .2 = x ÷ .5 which provides x=.1; x+z=.5 then provides z=.4, and we are now looking at:
               A      A'                           
             _____________                                        
            |      |      |
         B  |  .1  |  y   |  x+y=P(B)              
            |______|______|                                    
            |      |      |
         B' |  .4  |  w   |  z+w=P(B')                              
            |______|______|___                                         
                          |
               .5     .5  |  1
Finally, P(B|A')=.6 becomes .6 = y ÷ .5 which provides y=.3, and w=.2 follows. Summing x+y and z+w lets us complete the square:
               A      A'                           
             _____________                                        
            |      |      |
         B  |  .1  |  .3  |  .4              
            |______|______|                                    
            |      |      |
         B' |  .4  |  .2  |  .6                              
            |______|______|___                                         
                          |
               .5     .5  |  1
From this we can readily calculate P(A|B)=.1/.4=.25, or any other probability we wish.

return to index

Questions?