Bayes' theorem?
What does Bayes' theorem really mean?
You’ll often see this formula in Bayesian statistics:
P(A ∣ B) = P(B ∣ A) × P(A) / P(B)
This might look scary, but it’s actually telling a simple story: You start with what you believe (the prior), you see new data (the evidence), and you update your belief depending on how well that data fits your original idea (the posterior).
How does this translate to more common situations?
Imagine:
A = a person has a rare disease.
B = their test comes back positive.
The prior:
The disease is rare, so before the test, you didn’t think they had it.
The likelihood:
The test is usually right when the person is sick.
The posterior:
Now that you’ve seen the positive test, you update your belief, but you don’t forget the disease was rare to begin with!
In other words: even with a positive test, there’s still a good chance the person doesn’t have the disease, because the disease itself is so unlikely. Sounds like the way people naturally think, doesn’t it?
Bottom line: Bayes’ theorem is all about clear thinking, don’t rely solely on the data, but combine it with what you already know!
If you’re familiar with chemometrics or statistics, you might notice similarities with fuzzy logic and Kalman filters. Fuzzy logic handles uncertainty by letting things be partly true instead of strictly yes/no. Just like Bayes lets you assign a probability to a belief, fuzzy logic assigns degrees of membership to different possibilities. Kalman filters are like a real-time version of Bayes. They take a prior estimate, combine it with noisy measurements, and give you a better estimate and knowledge about uncertainty, then repeat the process as more data arrives.
All in all, whether you’re using Bayes’ theorem, fuzzy logic, or a Kalman filter, you’re doing the same smart thing .... using imperfect data and clever tools to make better decisions and/or predictions.
Discussion
Interesting explanation….but which of the A, B and P is now the ‘prior’, ‘evidence’ or ‘posterior’?
That's a good question and is indeed an ommission in the article. Let me try to explain.
First the formula: P(A|B)=P(B|A)xP(A)/P(B)
In human words: The posterior equals the likelihood times the prior, divided by the evidence. Mmm, okay, still not very enlightening, but I'm on my way.
P(A∣B) is the posterior: the probability of A being true given that B is observed. This is what we want to know, an updated belief after seeing the data.
P(A) is the prior: our belief about A before seeing any data.
P(B∣A) is the likelihood: the probability of observing the data B assuming A is true.
P(B) is the evidence or marginal likelihood: the overall probability of observing B, regardless of whether A is true.
How does this translate to our real-life example.
First, recall the formula:
P(A|B) = P(B|A) x P(A) / P(B)
P(A): the prior >> the disease is rare, so this value is low,
P(B∣A): the likelihood >> the test is usually accurate,
P(B): the evidence >> the probability of a positive test result across the whole population,
P(A∣B): the posterior >> the updated probability that the person has the disease, given the test result.
So even if the test is positive, the posterior P(A∣B) might still be quite low because P(A) (the prior) is very small, and P(B) (the evidence) might be elevated by false positives.
Some more explanation about what P(...), exactly means and what 'posterior' and 'prior' say.
P(A) means: the probability that event A happens.
P(B) means: the probability that event B happens.
P(A|B) means: the probability of A, given that B is true.
In other words, it answers: “How likely is A, if we know B has occurred?”
In our example:
P(A): the chance that someone has a disease (before any test).
P(B | A): the chance the test is positive if they have the disease.
P(A | B): the updated chance that they have the disease, given a positive test result.
So, P is simply a way to express uncertainty, a numerical value in the range of 0 ... 1 and represents a way to measure belief using numbers. In math: P(x) ∈ [0, 1]
.. If P(x)=0.5 there is 50% change x is true and 50% change that x is not true.