Bayes' Theorem in Action: How to Use New Information to Update your Beliefs

Magic Coins, Spam Filtering and Cancer Detection

Jan 19, 2025

This is The Curious Mind, by Álvaro Muñiz: a newsletter where you will learn about technical topics in an easy way, from decision-making to personal finance.

Traducir al español

How should new information change our prior beliefs?

The world is full of uncertainty, and we often have to take decisions based on a guess of the probability of our assumptions being right. Unfortunately, our instincts often fail us when dealing with probabilities, leading to conclusions that feel right but aren’t. Math can teach us how to adjust our beliefs based on new information, helping us make better choices.

Today we explore Bayes’ theorem and how it flips around the question: assuming my beliefs, is this information likely?

Today in a Nutshell

When dealing with an uncertain statement, the best we can do is assign it a probability of being true. This is our prior belief.
If we receive new information then we should adjust this probability: a rainy weather tomorrow is not equally likely if it rained today or not. This updates guess is our posterior belief.
Sometimes, it is not easy to see how to adapt our belief based on new information.
With Bayes’ theorem we can turn the question of “Based on this information, is my belief likely?" to "If my prior belief were true, how likely is it that I observe this information?”.

Let’s see this in action.

Magic Coins

Here is a nice puzzle:

Makas Island is famous for having magic coins.
When you flip a magic coin, it always shows heads. Magic coins are not common in Makas Island—about 1 in 1,000 coins is magic. Except for their behaviour, magic coins are identical to regular coins: they look the same, weight the same and feel the same in your hand.
Magic coins are very valuable, so you travel to Makas Island to try to find one. Once in the island, you find a coin and flip it 10 times. Every single flip lands heads.
How sure are you that you have found a magic coin?

Your intuition is probably telling you that you found a magic coin. You flipped the coin 10 times and you saw all heads—this is very rare for a normal coin. Thus, it is most likely a magic coin.

This intuition is wrong. We saw a similar thing when talking about COVID tests and the probability of being infected. Today we will see how to update our beliefs based on observations.

Spoiler: the coin you flipped is equally likely to be a magic coin or a regular coin!

English mathematician and philosopher Thomas Bayes

Bayes’ Theorem

The mathematical result that helps us solve this type of problems is called Bayes’ theorem, in name of the English statistician and philosopher Thomas Bayes.

The set-up is as follows: you have a statement you are unsure about and some information that could help you find the statement’s likelihood. For example, in the magic coin puzzle the statement could be "The coin is magic" and the information could be "We flipped a coin 10 times and they all landed heads".

We are interested in estimating the likelihood of the statement given the information we have. It turns out that, in cases like the Magic Coin Puzzle, it is easier to turn the problem around:

I don’t know the likelihood of the coin being magic if I saw 10 heads.
But I do know the likelihood of seeing 10 heads if the coin is magic (this likelihood is 100%).

Bayes’ theorem lets us interchange this two problems. If we denote the statement we are studying by S, and the information we have by I, then Bayes’ theorem says:

This is how to read the equation above:

In the right we have three things:
- P(S): this is the probability of our statement without having received the information (e.g. “the probability that a coin I just found is magic, without having flipped it”). We call this our prior: what we believe to be true before any information
- P(I): this is the probability of our information (e.g. “the probability that we flip a random coin 10 times and see 10 heads).
- P(I | S): this is the probability of observing our information if the statement is true (e.g. “the probability that we flip a coin 10 times and see 10 heads, given that the coin is magic). It is the likelihood of the information.
In the left we have P(S | I): this is the probability of our statement being true given information I (e.g. “the probability that the coin is magic assuming that I saw 10 heads”). We call this our posterior: our updated belief after seeing the information. It is the thing we want to find.

The key is that in the left we have P(S | I) and in the right we have P(I | S)

Bayes’ theorem flips the problem from “How likely is my belief, given this information?” to “If my belief were true, how often would I have observed my information?”.

Example: Magic Coins

Let’s work out the Magic Coin puzzle to see how to use Bayes’ theorem.

We want the probability of the coin being magic (our statement S), given that we say 10 heads (our information I). That is, we want P(S | I). To apply Bayes’ theorem, we need:

P(I | S): the probability of seeing 10 heads if the coin is magic. Since magic coins always show heads, we have P(I | S) = 1.
P(S): the probability of the coin being magic, without having received the information. We were told that in Makas Island about 1 in 1,000 coins is magic, so we have P(S) = 1/1,000 = 0,001.
P(I): the probability of observing 10 heads. This is the trickiest thing to compute, since it depends on whether the coin is magic or not:
- If the coin is magic, the probability is 1 (magic coins always show heads).
- If the coin is a normal coin, then the probability of 1 flip being heads is 0.5, the probability of 2 flips being two heads is 0.5 x 0.5, and the probability of 10 flips being 10 heads is 0.5 x 0.5 x … x 0.5 for a total of 10 times, i.e. 0.5^10.
The coin is magic 1/1,000 times, and is normal the rest of the times, 999/1,000. Therefore, P(I) = 1/1,000 x 1 + 999/1,000 x 0,5^10 = 0.002.

We now have all the ingredients to apply Bayes’ theorem:

We have no reason to believe that it is a magic coin—nor that it’s not!

Real-life examples

Bayes’ theorem is not just an interesting mathematical fact to solve magic coin puzzles, but it is used on a daily basis in several industries.

Here are some examples:

Detecting fraude

Have you ever had your credit card blocked because of a 'suspicious transaction'?

Your bank needs a way to estimate the probability of a fraude, given some particular type of transaction. Using Bayes’ theorem, they can turn this question into "If this were a fraud, would they do a transaction similar to this one?". This is easier to estimate, since the bank only has to study characteristics of fraudulent transactions.

Thank Bayes’ theorem for saving your money!

Spam filtering

How does your e-mail app guess whether an e-mail should be sent to the spam folder?

The app needs a way to estimate the probability of an e-mail being spam given its content. Using Bayes’ theorem, it can turn this question into "If this e-mail were spam, how likely is it that it would look like this?". Now you only have to understand what spam e-mails look like, and forget about their similarities with non-spam e-mails.

Thank Bayes’ theorem for cleaning your inbox!

Detecting cancer

How can you estimate the likelihood of a person having cancer, given some symptoms and health markers?

Your doctor and the healthcare system need a way to know if a patient’s symptoms hint to the patient having cancer. Using Bayes’s theorem, they can instead ask the question "If this person had cancer, would I see these symptoms". It is easier to look for these symptoms in previous cancer cases than to look for cancer among all people with some symptom (e.g. you can cough for cancer but also for a flue, because you smoke or for a sore throat).

Thank Bayes’ theorem (and doctors!) for saving lives!

Bayes’ theorem is ubiquitous, finding applications over and over again. Whenever you want to update your beliefs based on information you receive, use Bayes’ theorem and ask yourself:

Assuming my belief, how likely is this information?

Thanks for reading The Curious Mind! If you like this post, hit the ❤️ button below and share with your friends.

The Curious Mind