Statistics For Data Science Course

Updating Believes with Bayes’ Theorem


Where, P(A/D) is the probability of event A given event D has occurred. Let’s understand Bayes theorem in detail with the help of an example.

When does Bayes’ Theorem help?

Let’s consider this problem.

A, B, C are the rating that a bank gives to its
borrowers. Let’s the probability of getting rated A, B, and C are as follows.

P(A) = 30%

P(B)= 60 %

P(C)= 10%

Some of the customers defaulted on their borrowings. 1%
of the customers who were rated A, 10% of the customers who were rated B and
18% of the customers who were rated C became defaulters.

If a customer who is a defaulter. What is the probability
that he was rated A?

We can show all the customers of the bank by a rectangle and designate the portion of the customer’s who are rated A, B and C respectively by sections which are named A, B, and C as below. Also, the circle represents the customers who are defaulters and is denoted by D.

The question at hand is to determine what is the probability that the customer was rated A, if it is known that he belongs to defaulter category. Important properties of this kind of problem are as follows :-

  • No two customer can have same rating. When a customer is rated A, he cannot be rated B or C. In this case we can say that A, B, and C are mutually exclusive events.
  • Every customer is given a rating out of A, B, and C.  Here, A, B, and C are called collectively exhaustive events. It is sure that one of these events will occur when the customers are rated.  Mathematically, P(AUBUC) =1
  • From the diagram above, we can write that P(D)= P(A∩D) + P(B∩D) + P(C∩D)  =P(A)*P(D/A) + P(B)*P(D/B) + P(C)*P(D/C) ( from multiplication rule)

Applying Bayes Theorem to this problem

If any set of events has the properties as mentioned in the preceding section, we can apply the Bayes theorem. The previous problem comes down to calculating 𝑃(𝐴/𝐷), which can be calculated as below.

In above formula, we know each of the individual probabilities. 

P(A)= 0.30, P(B)= 0.60, P(C)= 0.10, P(D/A) = 0.01, P(D/B) = 0.10, P(D/C) = 0.18.

 By putting  these values in the above formula, we get that P(A/D) = 0.037

Bayes theorem – Updating believes

A very interesting thing about Bayes’ theorem is that is its nature to update the belief based on the evidence available.  Let’s take an example.
Ram and Shyam are two friends. Both have a belief about a particular restaurant in a city. we would call this as prior belief. Ram thinks that the restaurant is cost effective and serves very tasty food. While Shyam has  a different belief. He had heard from someone that the restaurant does not maintain proper hygiene. Anyway after much insistence from Ram, Shyam decides to have a dinner in the restaurant . He found the restaurant to be satisfactory in nature so he keeps on visiting the restaurant. After several hygienic and tasty dinner at the same restaurant ,
  • Ram’s belief has strengthen that the restaurant is good.
  • Shyam’s belief that the restaurant is not good has weakened
Hence, after exposure to evidence, both have reached to almost same conclusion that the restaurant is indeed good. This is called as posterior belief.

Updating believes example

It was found in US that 1.48 out of every 1000 people have breast cancer. Two sets of tests were done, and the patient was diagnosed with cancer if she tested positive in both of them. The following information was known about such tests.
  • Sensitivity of the test (93%) – true positive Rate- P(+/ cancer)
  • Specificity of the test (99%) – true negative Rate- P(-/ not cancer)
  • P(cancer) = 0.00148, P(not cancer) = 0.99852

Probability of having cancer given the patient is tested positive in the first test P(Cancer/+)

Here we see that we started with the data of all the people in US and from the first test it is found that if a person is tested positive, there is only 12% chance that she actually has cancer.


Probability of having cancer given the patient is tested positive in the second test P(Cancer/+)

In next step, we would update P(cancer) as 12% and probability of no cancer as 88%. Then we can use the existing data to calculate what is the probability that she has cancer given that she also tests positive in second test? 

As evident from the following diagram, the probability(Cancer/+) is 93%.

Updating Believes – Summary

 After first test, we updated the probability of having cancer to 12%. And over that we used evidence to calculate the posterior probability. This process is summarized in the following diagram.

One can refer the following video for detailed explanation of theses concepts.

Show Comments

No Responses Yet

Leave a Reply