NIM paper · interactive edition Interactive edition · paper, explanations and Lean proofs.

Definition 5.6Deterministic rollout policies and the rollout outcome bit

Checked definition or construction Lean contains the definition or construction, with checked properties.

1 Paper text

Definition 5.6 (Rollout policy and rollout outcome) A deterministic rollout policy maps a nonterminal NBNB-bit NIM position to a legal move, encoded by a heap index and its new value; on the terminal position it outputs a halt symbol. It is AC0\AC^0-bounded when this map has polynomial-size, constant-depth circuits. A rollout has both players follow that policy until termination. Its outcome ROπ(P)\mathrm{RO}_{\pi}(P) is 1 if the first player wins and 0 otherwise, including at a terminal start. The interface returns only this terminal bit, not a trajectory or move count.

Generated from paper_jmlr.tex, lines 633–637.

2 Explain this

A rollout policy is a deterministic rule that, given a board, names one legal move. A rollout lets both players follow that same rule from a start position until the board is empty; the only thing reported back is one bit: did the first player win (make the last move)?

Example: the policy π0\pi_0 "subtract 1 from the leftmost nonzero heap" played from (2,1)(2,1) goes (2,1)→(1,1)→(0,1)→(0,0)(2,1)\to(1,1)\to(0,1)\to(0,0): three moves, so the first player wins and the outcome bit is 11.

Why only one bit: this isolates how much information repeated simulation can expose, without letting the interface pass whole trajectories or move counts to the planner.

Multi-head rollouts and the outer controller, tick by tick

Exact construction trace Every value is computed from the construction’s definitions for the example shown.

Policy πj\pi_j subtracts 2j2^j from the first heap of size at least 2j2^j, and otherwise subtracts 1 from the first nonzero heap. A rollout lets both simulated players follow πj\pi_j until no move is left; its outcome bit is the parity of the number of moves. Step through the outer controller one tick at a time: it resets, runs head 0's rollout move by move on a working copy of the board, records the outcome when the rollout ends, reloads the original board for the next head, and, in the all-head schedule, finally reads out a move. Every outcome bit here is produced by actually running the rollout; nothing is supplied from outside. The early-exit schedule (Figure 3) stops at the first outcome 1 and only classifies the position.

35 component evaluations (34 controller ticks and one readout). Colours: reset move terminal test store and reload store and stop readout.

  1. tick 1: reset
  2. tick 2: move, head 0
  3. tick 3: move, head 0
  4. tick 4: move, head 0
  5. tick 5: move, head 0
  6. tick 6: move, head 0
  7. tick 7: move, head 0
  8. tick 8: move, head 0
  9. tick 9: move, head 0
  10. tick 10: move, head 0
  11. tick 11: move, head 0
  12. tick 12: move, head 0
  13. tick 13: move, head 0
  14. tick 14: move, head 0
  15. tick 15: terminal test, head 0
  16. tick 16: store outcome and reload, head 0
  17. tick 17: move, head 1
  18. tick 18: move, head 1
  19. tick 19: move, head 1
  20. tick 20: move, head 1
  21. tick 21: move, head 1
  22. tick 22: move, head 1
  23. tick 23: move, head 1
  24. tick 24: terminal test, head 1
  25. tick 25: store outcome and reload, head 1
  26. tick 26: move, head 2
  27. tick 27: move, head 2
  28. tick 28: move, head 2
  29. tick 29: move, head 2
  30. tick 30: move, head 2
  31. tick 31: move, head 2
  32. tick 32: move, head 2
  33. tick 33: terminal test, head 2
  34. tick 34: store outcome and stop, head 2
  35. readout

35 / 35 Readout: one evaluation of the readout circuit on the original board and the 3 stored outcome bits.

Working board (0, 0, 0)
012
Original board (read-only input) (3, 4, 6)
012
head j
2
parity of moves
1
rollout finished
1
running
0
some outcome 1
1
stored outcomes
o0 = 1 o1 = 1 o2 = 1

Readout

σ0 = o0 = 1; σ1 = o1 ⊕ o0 = 0; σ2 = o2 ⊕ o0 = 0, so σ = 1. The readout tests every heap in parallel and takes the first that decreases:

  • heap 0: 3 ⊕ 1 = 2 smaller
  • heap 1: 4 ⊕ 1 = 5 not smaller
  • heap 2: 6 ⊕ 1 = 7 not smaller

Move: set heap 0 to 2 → (2, 4, 6), nim-sum 0.

Counting the work

  • Simulated moves M = 13 + 7 + 7 = 27 (3 heads run).
  • Evaluations M + 2B + 2 = 27 + 6 + 2 = 35: 27 move updates, 3 terminal tests, 3 outcome updates, 1 reset, 1 readout.
  • Bound on M: BN(2B − 1) = 63.
  • Each single evaluation is a shallow circuit: one policy step has Boolean depth ≤ 17 (≤ 36 threshold layers), the readout ≤ 19 (≤ 40), one controller tick ≤ 34 (≤ 70). Those bounds are per evaluation; the number of evaluations grows with the board. That no shallow family computes the complete board-to-move map is a separate statement, conditional on the named classical input that parity is not in AC0.
All 34 controller ticks as a table
TickKindWorking boardjparityo₀…What happens
1reset(3, 4, 6)00000Reset: load the board, select head 0, clear the outcomes.
2move(2, 4, 6)01000Head 0 moves: heap 0 from 3 to 2 (subtract 1 from the first heap of size at least 1).
3move(1, 4, 6)00000Head 0 moves: heap 0 from 2 to 1 (subtract 1 from the first heap of size at least 1).
4move(0, 4, 6)01000Head 0 moves: heap 0 from 1 to 0 (subtract 1 from the first heap of size at least 1).
5move(0, 3, 6)00000Head 0 moves: heap 1 from 4 to 3 (subtract 1 from the first heap of size at least 1).
6move(0, 2, 6)01000Head 0 moves: heap 1 from 3 to 2 (subtract 1 from the first heap of size at least 1).
7move(0, 1, 6)00000Head 0 moves: heap 1 from 2 to 1 (subtract 1 from the first heap of size at least 1).
8move(0, 0, 6)01000Head 0 moves: heap 1 from 1 to 0 (subtract 1 from the first heap of size at least 1).
9move(0, 0, 5)00000Head 0 moves: heap 2 from 6 to 5 (subtract 1 from the first heap of size at least 1).
10move(0, 0, 4)01000Head 0 moves: heap 2 from 5 to 4 (subtract 1 from the first heap of size at least 1).
11move(0, 0, 3)00000Head 0 moves: heap 2 from 4 to 3 (subtract 1 from the first heap of size at least 1).
12move(0, 0, 2)01000Head 0 moves: heap 2 from 3 to 2 (subtract 1 from the first heap of size at least 1).
13move(0, 0, 1)00000Head 0 moves: heap 2 from 2 to 1 (subtract 1 from the first heap of size at least 1).
14move(0, 0, 0)01000Head 0 moves: heap 2 from 1 to 0 (subtract 1 from the first heap of size at least 1).
15terminal test(0, 0, 0)01000Head 0: no legal move, so the rollout has ended after an odd number of moves; the outcome bit is 1.
16store outcome and reload(3, 4, 6)10100Store outcome o0 = 1, reload the original board and select head 1.
17move(1, 4, 6)11100Head 1 moves: heap 0 from 3 to 1 (subtract 2 from the first heap of size at least 2).
18move(1, 2, 6)10100Head 1 moves: heap 1 from 4 to 2 (subtract 2 from the first heap of size at least 2).
19move(1, 0, 6)11100Head 1 moves: heap 1 from 2 to 0 (subtract 2 from the first heap of size at least 2).
20move(1, 0, 4)10100Head 1 moves: heap 2 from 6 to 4 (subtract 2 from the first heap of size at least 2).
21move(1, 0, 2)11100Head 1 moves: heap 2 from 4 to 2 (subtract 2 from the first heap of size at least 2).
22move(1, 0, 0)10100Head 1 moves: heap 2 from 2 to 0 (subtract 2 from the first heap of size at least 2).
23move(0, 0, 0)11100Head 1 moves: heap 0 from 1 to 0 (no heap has 2 or more: subtract 1 from the first nonzero heap).
24terminal test(0, 0, 0)11100Head 1: no legal move, so the rollout has ended after an odd number of moves; the outcome bit is 1.
25store outcome and reload(3, 4, 6)20110Store outcome o1 = 1, reload the original board and select head 2.
26move(3, 0, 6)21110Head 2 moves: heap 1 from 4 to 0 (subtract 4 from the first heap of size at least 4).
27move(3, 0, 2)20110Head 2 moves: heap 2 from 6 to 2 (subtract 4 from the first heap of size at least 4).
28move(2, 0, 2)21110Head 2 moves: heap 0 from 3 to 2 (no heap has 4 or more: subtract 1 from the first nonzero heap).
29move(1, 0, 2)20110Head 2 moves: heap 0 from 2 to 1 (no heap has 4 or more: subtract 1 from the first nonzero heap).
30move(0, 0, 2)21110Head 2 moves: heap 0 from 1 to 0 (no heap has 4 or more: subtract 1 from the first nonzero heap).
31move(0, 0, 1)20110Head 2 moves: heap 2 from 2 to 1 (no heap has 4 or more: subtract 1 from the first nonzero heap).
32move(0, 0, 0)21110Head 2 moves: heap 2 from 1 to 0 (no heap has 4 or more: subtract 1 from the first nonzero heap).
33terminal test(0, 0, 0)21110Head 2: no legal move, so the rollout has ended after an odd number of moves; the outcome bit is 1.
34store outcome and stop(0, 0, 0)21111Store outcome o2 = 1 and stop (last head).

How this relates to the Lean proof. Each tick is one step of Shootout.ostep, with the policy step SelectCircuit.policyIdx inside the rollout registers RolloutExec.rsStep. Rollout.run_parity identifies the outcome bits with σ0\sigma_0 and σj⊕σ0\sigma_j\oplus\sigma_0, and Shootout.ah_counts proves that the all-head schedule takes M+2B+1M+2B+1 controller ticks, the reset included; with the single readout this gives M+2B+2M+2B+2 evaluations. The browser re-implements these steps; the agreement is checked against Lean's own evaluation of them.

What this does not show. Simulated moves and circuit evaluations are different counts: the MM moves are part of the M+2B+2M+2B+2 evaluations. The depth bounds (17 per policy step, 19 for the readout, 34 per controller tick) apply to one evaluation each. The complete board-to-move computation repeats them a number of times that grows with the board. That no shallow family computes the complete map is a separate statement (Proposition 5.14 and the text after it), conditional on the named classical input that parity is not in AC0\mathrm{AC}^0.

Depicted in Lean: SelectCircuit.policyIdx, RolloutExec.rsStep, Shootout.ostep, Shootout.initOS, Shootout.query_step, Rollout.run_parity, Game.recoverBit, Game.reconstruct_eq, Shootout.ahMove_eq, Shootout.ah_counts, Shootout.ahL_counts, Shootout.ee_spec, Shootout.ee_counts. Explains the paper’s Figure 3.

Computed in your browser by a short re-implementation of the definitions above (tools/visuals/nimvis-core.js). It is not itself checked by Lean. At build time it agreed with Lean's own evaluation of those definitions on all 313 reference cases.

3 How this is formalised

Lean formalises the structured policies πj\pi_j that the results use. A legal πj\pi_j step (Rollout.PolicyMove) subtracts 2j2^j from some heap of size at least 2j2^j, or, when every heap is below 2j2^j, subtracts 11 from some nonzero heap; the tie-breaking rule is left arbitrary. Rollout.policyStep is the concrete first-eligible-heap version, and Game.rolloutWinner executes the rollout and returns whether the first player made the last move. Prob.rollout_outcome_linear proves that this executed outcome equals the linear functional ⟨wj,σ(P)⟩\langle w_j,\sigma(P)\rangle of the nim-sum.

The general notion of an arbitrary AC0-bounded rollout policy is not a separate Lean object; no result depends on it.

Coverage record
Paper anchordef:rollout-policy (manuscript label)
Displayed asDefinition 5.6
ClaimDeterministic rollout policies and the rollout outcome bit
Coverage categoryChecked definition or construction
Statement sourcepaper_jmlr.tex lines 633–637; SHA-256 of the normalised block source 4edfadc2a395e224…
Lean declarations
Prose-to-formal notes

Lean defines the outcome of the structured policies πj\pi_j by executing the rollout (rolloutWinner). The general definition of an arbitrary AC0\mathrm{AC}^0-bounded rollout policy is not formalised as a separate object; the paper's results use only the structured family.

Audited correspondence rowsdef:rollout-policy
Explanationsexplain/def:rollout-policy, formal/def:rollout-policy
Build evidencefresh isolated rebuild, audit report

4 Lean statement and source

Principal declarations

def Game.rolloutWinner

Module NimGame (lean/phase2/NimGame.lean, lines 142–152) · file SHA-256 23df6bb87ecfd344…

Rollout outcome RO_{pi_j}(P) as a game: both players follow pi_j; the result is `true` iff the player to move at P (the first player) wins under normal play. At a terminal start it is `false`.

Statement as elaborated and printed by Lean 4.26.0

ℕ → List ℕ → Bool
Exact source text, lines 142–152
142/-- Rollout outcome RO_{pi_j}(P) as a game: both players follow pi_j; the result is143    `true` iff the player to move at P (the first player) wins under normal play.144    At a terminal start it is `false`. -/145def rolloutWinner (j : Nat) (p : List Nat) : Bool :=146  match h : policyStep j p with147  | none => false148  | some q => !rolloutWinner j q149termination_by rolloutCount j p150decreasing_by151  have := policyMove_count (policyStep_some h)152  omega

Project declarations in the statement: none

Project declarations used by the proof or definition body

Rollout.policyMove_count, Rollout.policyStep, Rollout.policyStep_some, rolloutCount

def Rollout.policyStep

Module NimRollout (lean/NimRollout.lean, lines 116–120) · file SHA-256 1a99428d71573d31…

One step of `pi_j` with first-eligible tie-breaking; `none` means no move.

Statement as elaborated and printed by Lean 4.26.0

ℕ → List ℕ → Option (List ℕ)
Exact source text, lines 116–120
116/-- One step of `pi_j` with first-eligible tie-breaking; `none` means no move. -/117def policyStep (j : Nat) (p : List Nat) : Option (List Nat) :=118  match subFirstGe (2 ^ j) p with119  | some q => some q120  | none => subFirstPos p

Project declarations in the statement: none

Project declarations used by the proof or definition body

Rollout.subFirstGe, Rollout.subFirstPos

def Rollout.PolicyMove

Module NimRollout (lean/NimRollout.lean, lines 28–32) · file SHA-256 1a99428d71573d31…

One legal move of the structured policy `pi_j` (tie-breaking unspecified).

Statement as elaborated and printed by Lean 4.26.0

ℕ → List ℕ → List ℕ → Prop
Exact source text, lines 28–32
28/-- One legal move of the structured policy `pi_j` (tie-breaking unspecified). -/29def PolicyMove (j : Nat) (p q : List Nat) : Prop :=30  (∃ pre x post, p = pre ++ x :: post ∧ 2 ^ j ≤ x ∧ q = pre ++ (x - 2 ^ j) :: post) ∨31  ((∀ x ∈ p, x < 2 ^ j) ∧32    ∃ pre x post, p = pre ++ x :: post ∧ 0 < x ∧ q = pre ++ (x - 1) :: post)

Project declarations in the statement: none

Project declarations used by the proof or definition body

none

theorem Prob.rollout_outcome_linear

Module NimProbability (lean/phase2/NimProbability.lean, lines 920–923) · file SHA-256 1d5d7977a43e854d… Axioms reported by the fresh build: propext, Quot.sound

RO_{pi_j}(P), the executed rollout outcome, is the linear functional <w_j, sigma(P)>.

Statement as elaborated and printed by Lean 4.26.0

∀ {B j : ℕ},
  j < B →
    ∀ (P : List ℕ),
      NimAudit.Game.rolloutWinner j P =
        NimAudit.Prob.dotB B (NimAudit.Prob.headVec j) (NimAudit.nimSum P)
Exact source text, lines 920–923
920/-- RO_{pi_j}(P), the executed rollout outcome, is the linear functional <w_j, sigma(P)>. -/921theorem rollout_outcome_linear {B j : Nat} (hj : j < B) (P : List Nat) :922    rolloutWinner j P = dotB B (headVec j) (nimSum P) := by923  rw [rolloutWinner_channel, head_linear hj]

Project declarations in the statement: Game.rolloutWinner, Prob.dotB, Prob.headVec, nimSum

Project declarations used by the proof or definition body

Game.rolloutWinner, Game.rolloutWinner_channel, Prob.dotB, Prob.headVec, Prob.head_linear, channel, nimSum

Definitions these statements use

def Prob.dotB

Module NimProbability (lean/phase2/NimProbability.lean, lines 259–262) · file SHA-256 1d5d7977a43e854d…

The F_2 inner product of the first B bits of u and v.

Statement as elaborated and printed by Lean 4.26.0

ℕ → ℕ → ℕ → Bool
Exact source text, lines 259–262
259/-- The F_2 inner product of the first B bits of u and v. -/260def dotB : Nat → Nat → Nat → Bool261  | 0, _, _ => false262  | B + 1, u, v => xor (dotB B u v) (u.testBit B && v.testBit B)

Project declarations in the statement: none

Project declarations used by the proof or definition body

none

def Prob.headVec

Module NimProbability (lean/phase2/NimProbability.lean, lines 875–876) · file SHA-256 1d5d7977a43e854d…

The vector w_j with RO_{pi_j}(P) = <w_j, sigma(P)>: w_0 = 1, w_j = 2^j XOR 1.

Statement as elaborated and printed by Lean 4.26.0

ℕ → ℕ
Exact source text, lines 875–876
875/-- The vector w_j with RO_{pi_j}(P) = <w_j, sigma(P)>: w_0 = 1, w_j = 2^j XOR 1. -/876def headVec (j : Nat) : Nat := if j = 0 then 1 else 2 ^ j ^^^ 1

Project declarations in the statement: none

Project declarations used by the proof or definition body

none

def nimSum

Module NimCore (lean/NimCore.lean, lines 7–9) · file SHA-256 b07db6c505210d5a…

Statement as elaborated and printed by Lean 4.26.0

List ℕ → ℕ
Exact source text, lines 7–9
7def nimSum : List Nat → Nat8  | [] => 09  | h :: t => h ^^^ nimSum t

Project declarations in the statement: none

Project declarations used by the proof or definition body

none

Full modules

ModuleFileLinesImports
NimCorelean/NimCore.lean139Std
NimGamelean/phase2/NimGame.lean634NimRollout
NimProbabilitylean/phase2/NimProbability.lean1074NimGame
NimRolloutlean/NimRollout.lean349NimCore

Build evidence: fresh isolated rebuild of all 38 bundled modules with Lean (version 4.26.0, started 2026-10-10T15:21:56Z, finished 2026-10-10T15:27:00Z: 38/38 modules compiled, 1588 axiom reports, no errors. See Build for the commands and the independent audit of 2026-10-10 (dated evidence).