المنهجية

What is measured

Every time an event settles, we freeze what each venue was pricing at fixed horizons before the outcome was known: at resolution, and 1, 7, 30 and 90 days before it. Each frozen reading is one observation — a predicted probability paired with what actually happened.

The reading at a horizon is the last price recorded at or before that moment. If collection has a gap and the most recent reading is more than three days older than the horizon, the observation is dropped rather than stretched — an old price is not a prediction at that horizon.

Brier score

The mean squared difference between the predicted probability and the outcome coded as 1 or 0. It runs from 0 (every prediction perfect and confident) to 1 (every prediction confident and wrong). Always predicting 50% scores 0.25, which is the number to beat before a market has told you anything.

Log loss

The average negative log of the probability assigned to what actually happened. It punishes confident errors far more sharply than the Brier score. Probabilities are clipped to 0.000001at each end: an unclipped prediction of 0 or 1 that turned out wrong would produce infinite loss, letting a single bad reading swallow a venue’s entire score.

Calibration curve

Observations are grouped into ten buckets by predicted probability — 0–10%, 10–20%, and so on, with 100% falling in the last bucket. For each bucket we plot the mean prediction against the share that actually happened. A perfectly calibrated market sits on the diagonal: of everything it called 70% likely, 70% happened. Point size reflects how many observations sit in that bucket.

An empty bucket is shown as no data, not as zero. “Never happened” and “never measured” are different statements and the chart keeps them apart.

Sample size

Every figure carries the number of resolved observations behind it. Below 30 resolved events a slice is labelled as not yet reliable — but it is still shown. Hiding weak slices would quietly flatter whichever venue or category happens to have the least data.

What this does not measure

Only events that reached settlement are counted, so a venue that delists markets before they resolve is under-represented — its hardest calls never enter the sample. Events are included only from the day we began tracking them, which skews toward larger, longer-lived markets. And an accurate market is not a profitable one: fees, spread and the cost of capital are not in any of these numbers.

Recomputation

Snapshots are rebuilt daily from the full observation set — never incrementally — so a fix to the maths or a correction to an outcome propagates through the entire history rather than leaving old numbers standing.