[#34] Bench: deterministic roster — printTable must not mutate the shared brains slice #60

Merged

robertismo merged 1 commit from 60-bench-roster into master

2026-09-29 19:36:59 +00:00

Winnie commented

2026-09-29 19:36:33 +00:00

Owner

Bench harness bug found during the ship gate — 3-line fix, please review

What I found. printTable sorted the caller's brains slice in place. runBench passes its global roster slice, and benchPresets() is a Go map, so after the first preset printed, the roster order for EVERY remaining preset depended on random map iteration order:

duel (3 players) seated different triples run to run — this run it was adapt/mill/vault (adapt 83 wins), the baseline run had empire/mill/vault, another run had yet another triple;
in the 6-player presets the double seats (2 of the 4 brains get 2 seats) also shuffled between brains per run.

So the final merged bench numbers were not reproducible run to run. All four brain PRs' scenario numbers were measured under this nondeterminism; the wins on 6-player presets are order-influenced (double seats + seat-order tiebreaks) and duel's roster was arbitrary.

The fix (commit 9bb944f): printTable sorts a copy, the roster stays [empire, mill, vault, adapt] for every preset — duel seats empire/mill/vault (adapt does not play duel, as the preset intends), 6-player presets double-seat empire and mill (the first two roster entries, as the round-robin always intended).

Consequences for the four brains (verified by re-running the full bench on the fix — next comment has the table): adapt still wins base, patient-market, micro-economy without duel; vault still wins bond-economy + tight-cash; empire and mill hold their presets. No brain's win condition depended on the shuffled roster.

Gate decision: I am re-running the final 100-game merged bench on the fix before generating the PDF report, so the shipped numbers are reproducible. If you'd rather ship as-is and file this for v0.4.1, close this PR and I'll proceed that way.

## Bench harness bug found during the ship gate — 3-line fix, please review **What I found.** `printTable` sorted the caller's `brains` slice in place. `runBench` passes its global roster slice, and `benchPresets()` is a Go map, so after the first preset printed, the roster order for EVERY remaining preset depended on random map iteration order: - duel (3 players) seated different triples run to run — this run it was adapt/mill/vault (adapt 83 wins), the baseline run had empire/mill/vault, another run had yet another triple; - in the 6-player presets the double seats (2 of the 4 brains get 2 seats) also shuffled between brains per run. So the final merged bench numbers were not reproducible run to run. All four brain PRs' scenario numbers were measured under this nondeterminism; the wins on 6-player presets are order-influenced (double seats + seat-order tiebreaks) and duel's roster was arbitrary. **The fix** (commit `9bb944f`): `printTable` sorts a copy, the roster stays `[empire, mill, vault, adapt]` for every preset — duel seats empire/mill/vault (adapt does not play duel, as the preset intends), 6-player presets double-seat empire and mill (the first two roster entries, as the round-robin always intended). **Consequences for the four brains** (verified by re-running the full bench on the fix — next comment has the table): adapt still wins base, patient-market, micro-economy without duel; vault still wins bond-economy + tight-cash; empire and mill hold their presets. No brain's win condition depended on the shuffled roster. **Gate decision**: I am re-running the final 100-game merged bench on the fix before generating the PDF report, so the shipped numbers are reproducible. If you'd rather ship as-is and file this for v0.4.1, close this PR and I'll proceed that way.

Winnie added 1 commit

2026-09-29 19:36:33 +00:00

[#34 ] bench: printTable must not mutate the shared roster (sort a copy); benchPresets map order made rosters run-dependent

build / build (push) Successful in 24s

Details

9bb944f313

Winnie requested review from robertismo

2026-09-29 19:36:33 +00:00