How We Check a Poker Population Range Is Right

Every “population range” you have seen is a guess at a probability: given that this player limped, what hand is he holding? Most of them are published without any check of whether the guess is good. We wanted a number for that, because our engine plays against these ranges for real money, and a wrong range is worse than no range.

Step one: have the answer key

On Ignition the exported hand history shows every dealt seat’s hole cards, including the ones that folded. So for every villain who limped, completed, flatted, over-called, limp-called or cold-called a 3-bet in our 22,000 hands, we know the hand. That is the answer key. Sites that only reveal showdowns cannot run this test at all; their samples are biased toward the hands people choose to show.

Step two: ask the range how surprised it is

Read a range as a distribution over the 169 hand classes (weight × combos, normalised). For each real hand, record the log of the probability the range gave it. Average over the hands. That is the mean log-likelihood, in nats per hand: higher is better, and a range that assigns zero to a hand that then shows up is punished hard (we floor at 10⁻⁴, so a zero costs about −14.6 nats, and we report the zero rate separately).

Then compare, paired on the same hands, against three references:

• uniform: every combo alike. The no-information baseline. A range that cannot beat it is worse than nothing.
• group8: an 8-parameter model (the shares of premiums, pairs, suited aces, suited broadways, other suited, offsuit aces, offsuit broadways, offsuit junk), fitted leave-one-out. A 169-class table has to beat this to justify its extra 161 parameters.
• shipped, leave-one-out: the range we actually use, with the observation’s own hand refitted out wherever the table was built from it. Scoring a table on its own training hands is how you fool yourself.

Differences come with 95% bootstrap intervals from resampling hands.

What it looks like

For first-in limps by players with 60 bb or less (n = 719 held hands):

(The chart plots the magnitude; the actual values are negative.) The first row looks like a win: the shipped table scores −5.04 against uniform’s −5.17. But that table was built from these hands. Refit it leave-one-out and it drops to −5.21, below uniform. The table had memorised its sample. A simple 8-group model beats both.

That result is why that table was rebuilt. The replacement (v4) scores better than its predecessor by +0.195 nats [+0.144, +0.243] on this family and beats the 8-group baseline by +0.048 [+0.011, +0.086]. Across all six limp families it is better than uniform and better than the old table; on one family (over-limps) the lower bound is +0.014, which we read as “not shown worse”, not as a demonstrated gain.

The rules this leaves us with

1. Small cells are not ranges. At n = 40, the 95% interval on a 10% frequency is roughly 4% to 22%. We shrink small cells toward their parent (stack band → all stacks → all limp lines) rather than publish them raw, and on the blog any cell under 30 observations is hatched.
2. Shapes can come from a bigger pool, frequencies cannot. For the within-group shape (which suited connectors, which offsuit aces) a 9-million-hand sample from another site is better than 700 Ignition hands, because showdown bias cancels inside a hand group. The frequency of the action and the group shares must come from Ignition’s visible cards.
3. A gate, not a vibe. A new table replaces the old one only if its paired delta is above zero over hands it never saw. If the interval spans zero it may ship, but the register says “no worse”, not “better”.
4. Win-rate tests cannot do this job. At 6-max NL200 the standard deviation is about 100 bb/100, so 5,000 hands give a standard error of about 14 bb/100. The ranges touch 5–10% of hands. A +10 bb/100 effect on those hands is +0.5–1 bb/100 overall: invisible. Likelihood on held cards is the only test with enough power.

Why this is on a blog

Because the data posts here lean on these ranges, and you should know how hard they were tested. When a post says “the Ignition pool limp-calls 56% of the time”, that number came through this pipeline, with the n and the interval, and if it ever fails the gate we will say so in an update.