AIONORA Research · Probability verification · 3 November 2026

Brier Score Explained — With a Calculator

What does it mean to get a probability forecast right? Move beyond a simple hit rate and examine the size of the model's errors.

Why not just count wins?

Suppose two models each correctly call eight of ten outcomes. One repeatedly predicts 51%, while the other makes confident 95% predictions. A simple hit rate may treat them similarly, even if the second model is dangerously overconfident when it misses. Brier score is one way to evaluate the probabilities themselves.

For a binary event — such as whether at least one match goal occurs — the Brier score is the mean of the squared difference between a forecast probability and what actually happened:

Brier = average[(forecast probability − outcome)²]

Outcome is 1 if the specified event happens and 0 if it does not. A perfect binary Brier score is 0; larger values indicate greater squared error. The score must be computed on the same exact market and time-frozen predictions across both models.

Try three hypothetical forecasts

Edit any probability or outcome. The score and equal-sample baseline update locally in your browser. No personal information or entries are sent to AIONORA.

ForecastProbability %Observed
Event A
Event B
Event C
0.1633Mean Brier score
0.2222Constant observed-frequency baseline

On these three illustrative outcomes, the model's squared error is below the in-sample constant-frequency baseline. Three events cannot establish predictive skill.

How to interpret the result

For the initial example, the three squared errors are (0.70−1)² = 0.09, (0.60−0)² = 0.36 and (0.80−1)² = 0.04. Their mean is 0.1633. Two events occurred, so a constant forecast of 2/3 produces an in-sample Brier score of about 0.2222.

Important: This is a three-event educational toy example. It is not an AIONORA backtest, a calibrated sports probability model, an official track record, a staking system or evidence of future performance. A baseline estimated from the same three outcomes benefits from hindsight; for serious evaluation, define the baseline prospectively.

Compare like with like

Compare forecasts on the identical events, at comparable information cutoffs. Brier score is sensitive to the frequency of the event, so comparing values across different seasons, sports, markets or event types can be misleading. A model can have a low score and still fail to outperform a simple baseline.

Do not hide rejected forecasts

Dropping a forecast because its result was unfavourable or because the model later revised it can manufacture apparent improvement. An honest audit keeps the original forecast time, exact market, source odds, later revisions and outcome in separate immutable records.

What about three-way match outcomes?

This calculator intentionally covers only binary events. A football home/draw/away distribution is a multiclass forecast and needs a multiclass scoring definition — not this binary formula applied once without specifying a particular event.

Sources and further reading

This formula and baseline-comparison approach follow standard probabilistic forecast verification methods. See the ECMWF probabilistic forecast user guide and the scikit-learn model evaluation documentation. For the relationship between hit rate and price, read Why a 90% Hit Rate Can Still Lose Money.

Published by AIONORA Research, with AI-assisted drafting and independently checkable formula examples. Editorial limitations and source references are included to make claims auditable.

Explore AIONORA research.

Football, basketball and tennis probability research, with transparent methodological limitations — no guaranteed outcomes.

Read Daily COREFree research newsletter