How sample size changes a verdict about a player
Every scouting conversation reaches the same point: is this player actually better, or does he just have a different record? This guide puts a number on that question. Not a model, not a rating — the sample size required before a difference between two players can be called a difference at all.
In this guide (6)
The question, stated properly
"Player A has a better rate than player B" is a claim about two underlying rates. The records are samples from those rates, and samples move. The useful question is therefore not "who is ahead in the table" but "how large a gap would I need to see before the gap is unlikely to be noise".
That question has an answer, and the answer is uncomfortable: for most football metrics the required sample is far larger than a season.
The arithmetic of telling two players apart
Treat each player's count of events as a Poisson process. To separate two rates with 95% confidence and roughly 80% power, each player needs on the order of 16 divided by the square of the relative difference. The table shows what that means in practice.
| Difference between the two rates | Events needed per player |
|---|---|
| +20% | ≈ 390 |
| +30% | ≈ 175 |
| +50% | ≈ 63 |
| +100% (twice the rate) | ≈ 16 |
Illustration using a Poisson approximation: equal exposure per player, independent events, constant rate. Real football breaks all three assumptions — which makes the true requirement larger, not smaller.
Now translate events into minutes. Take a metric that arrives twice per 90 minutes — shots, progressive passes, defensive actions all sit in that range.
| Minutes played | Expected events | Smallest difference that is distinguishable |
|---|---|---|
| 180 (two matches) | ≈ 4 | +198% |
| 450 (five matches) | ≈ 10 | +125% |
| 900 (ten matches) | ≈ 20 | +89% |
| 1 800 (twenty matches) | ≈ 40 | +63% |
| 3 000 (a season) | ≈ 67 | +49% |
| 5 000 (two seasons) | ≈ 111 | +38% |
Read the last row carefully. After two full seasons, at this event rate, the smallest difference you can defend is about 38%. A player who produces 2.0 shots per 90 and one who produces 2.8 are, on this arithmetic, not yet distinguishable. Most published comparisons are made on far less than two seasons.
Why rare events are worse than they look
Goals are the classic trap. A striker scoring 0.4 per 90 produces roughly 13 goals in a 3 000-minute season. To distinguish that from 0.5 per 90 — a 25% difference — you would need several hundred goals' worth of exposure, which no career provides. This is not an argument that goals do not matter; it is an argument that goal counts alone cannot settle a comparison between two good strikers.
The same logic explains why assists, red cards, penalty saves and error-led goals are almost useless as comparative metrics over a single season: their arrival rate is too low for the sample to carry the claim.
What to do when the sample cannot settle it
- Compare bundles, not single numbers. Shots, shot locations, carries into the box and role all speak to the same question; one of them alone does not.
- Shrink towards the positional average. With small samples, the honest estimate sits closer to the group mean than to the raw rate. This is what makes scouting shortlists stable instead of random.
- Treat a difference as a prompt, not a verdict. A gap in the table is a reason to watch the player, not a conclusion about him.
- Fix the confounds before the numbers. Minutes as a substitute, minutes in a leading team, minutes against strong opponents — all shift rates for reasons that have nothing to do with quality.
- Say what the sample was. "Twelve matches, 940 minutes, one competition" is a sentence that lets a reader weigh the claim. A rate without a sample cannot be weighed at all.
What this does to a "who is better" answer
It changes the shape of the answer rather than refusing to give one. For a specific, measurable behaviour — say, how often a full-back receives the ball in the final third — the difference is real, repeatable and visible inside a handful of matches. For outcomes like goals and assists, the sample required is beyond a career, so the honest answer is provisional: these two are close enough that the choice should be made on role, system and price rather than on the rate table.
The productive sentence is not "A is better than B". It is "A and B are not distinguishable on this evidence; here is the evidence that would separate them".
Where the arithmetic stops
This guide uses a Poisson approximation on counts and assumes independent, identically distributed events. Football does not cooperate: form is autocorrelated, opponents differ in strength, roles change within a season, and a player returning from injury is not a stationary process. Those effects widen the real uncertainty, so treat the numbers above as a floor rather than a precise requirement.
For the smaller version of the same problem — one player's rate on very few minutes — see why per-90 numbers lie on small samples. For definitions, use the glossary; to apply the reading rules to a real match, see the field tilt case study.
Read next
What is xG and PPDA? A football metrics primer
Expected goals and passes allowed per defensive action explained from first principles: what each number measures and how to read it.
Expected goals (xG): how the metric is built
Shot location, angle, body part and defenders: the ingredients of an xG model and why a single number never tells the whole story.
Expected assists (xA): valuing the final pass
How xA scores a pass by the quality of the chance it creates, and how to use it without overrating a creative midfielder.