📊 Many people use p-values every day. Few actually know what they mean.
How many times have you heard (or said) something like: “p < 0.05, so the result is true”?
That is wrong. Fundamentally wrong.
🤔 What is the p-value, really?#
The p-value measures how surprising your data would be if nothing real were happening.
In other words:
“If everything were just random… how weird is what I just saw?”
Cookie test example:
- Old cookie: 52% approved
- New cookie: 60% approved
- p-value = 0.2
This means: if both cookies were equally good, we’d see a difference this big 20% of the time. It’s not saying the new cookie is better.
❌ What the p-value does NOT say#
- ❌ “There’s a 5% chance I’m wrong”
- ❌ “The result is true with 95% confidence”
- ❌ “Lower p-value = more true result”
✅ What it DOES say#
“If nothing real were happening, I’d see something this extreme only X% of the time.”
The confusion comes from our brain wanting to go from data → truth. But p-values work the other way: assume a world with no effect → evaluate how weird your data is in that world.
🎯 0.05 isn’t magic#
The 5% threshold was proposed by Ronald Fisher as a practical value, not a mathematical truth. It balances:
- False positives: believing something is happening when it’s not
- False negatives: missing a real effect
💡 Explanation in a nutshell#
The p-value doesn’t measure the probability that your hypothesis is true. It measures how unlikely your data would be in a world where there is no effect. A low p-value doesn’t confirm anything — it just says what was observed would be very unusual by pure chance.
More information at the link 👇
