Winnie/bankwars
Watch
1
0
Fork
You've already forked bankwars
 0
Code Issues 9 Pull requests 2 Projects Releases 6 Packages Wiki Activity Actions

[v0.4.0] Structured research log: every iteration machine-readable #46

New issue
Closed
opened 2026-09-29 18:25:00 +00:00 by Winnie · 5 comments
Winnie commented 2026-09-29 18:25:00 +00:00
Owner
Copy link

Prerequisite for the researcher-grade PDF report Robert requested on #39 (pre-release deliverable). Every hypothesis, brain edit, bench run, and keep/revert decision must land in a machine-readable log (iterations.json or similar) in the repo, committed alongside bench-results.md. Bench runs must emit per-preset per-brain rows (wins, win%, avg equity, maxAbs) plus per-specialist gauntlet rows, tagged with git SHA, timestamp, and the hypothesis text. The PDF pipeline (#47) consumes this log; nothing in the report may be hand-typed from memory. Acceptance: a fresh clone can rebuild the full iteration history table from the log alone.

Prerequisite for the researcher-grade PDF report Robert requested on #39 (pre-release deliverable). Every hypothesis, brain edit, bench run, and keep/revert decision must land in a machine-readable log (iterations.json or similar) in the repo, committed alongside bench-results.md. Bench runs must emit per-preset per-brain rows (wins, win%, avg equity, maxAbs) plus per-specialist gauntlet rows, tagged with git SHA, timestamp, and the hypothesis text. The PDF pipeline (#47) consumes this log; nothing in the report may be hand-typed from memory. Acceptance: a fresh clone can rebuild the full iteration history table from the log alone.
Winnie self-assigned this 2026-09-29 18:25:00 +00:00
Winnie commented 2026-09-29 18:25:39 +00:00
Author
Owner
Copy link

Starting work on this issue. Plan:

  1. Worktree /root/bw-46-iter-log on branch 46-iter-log off origin/master (e939cb1).
  2. Add iterlog.go with IterationRecord (Hypothesis, GitSHA, Timestamp, Presets map, Gauntlet map, BenchVS map) and append-only iterations.json handling.
  3. Wire --bench hypothesis flag + git rev-parse HEAD capture; gauntlet per-preset numbers captured into rows.
  4. go build / go vet / go test; end-to-end test of append-preserves-N-records without a full bench.
  5. Push branch, open PR against master, request review from robertismo.
    Will comment again at each milestone.
Starting work on this issue. Plan: 1. Worktree /root/bw-46-iter-log on branch 46-iter-log off origin/master (e939cb1). 2. Add iterlog.go with IterationRecord (Hypothesis, GitSHA, Timestamp, Presets map, Gauntlet map, BenchVS map) and append-only iterations.json handling. 3. Wire --bench hypothesis flag + git rev-parse HEAD capture; gauntlet per-preset numbers captured into rows. 4. go build / go vet / go test; end-to-end test of append-preserves-N-records without a full bench. 5. Push branch, open PR against master, request review from robertismo. Will comment again at each milestone.
Winnie commented 2026-09-29 18:29:40 +00:00
Author
Owner
Copy link

Milestone: implementation done, green in bankdev.

  • New iterlog.go: IterationRecord{Hypothesis, GitSHA, Timestamp, Presets map[string]PresetRow, Gauntlet map[string]GauntletRow, BenchVS map[string]VSRow}; appendIteration loads existing iterations.json, appends, rewrites (corrupt file -> error, never clobbered).
  • bench.go: runBench(hypothesis) captures per-preset rows via presetRowFrom (avgEq = sum/insts) and gauntlet rows as runGauntlet computes them; bench-vs logs VSRows. --bench accepts an optional hypothesis argument, --hypothesis flag also works; absent -> "untagged run".
  • git SHA via exec.Command("git", "rev-parse", "HEAD") inside the harness.
  • go build / go vet ./... / go test . all pass (10 tests, incl. 5 new iterlog tests).
  • Added BANKWARS_BENCH_GAMES / BANKWARS_GAUNTLET_GAMES env knobs for tiny smoke benches of the real binary.
    Next: real-binary smoke run (1 game) to verify iterations.json is written + appends, then commit/push/PR.
Milestone: implementation done, green in bankdev. - New iterlog.go: IterationRecord{Hypothesis, GitSHA, Timestamp, Presets map[string]PresetRow, Gauntlet map[string]GauntletRow, BenchVS map[string]VSRow}; appendIteration loads existing iterations.json, appends, rewrites (corrupt file -> error, never clobbered). - bench.go: runBench(hypothesis) captures per-preset rows via presetRowFrom (avgEq = sum/insts) and gauntlet rows as runGauntlet computes them; bench-vs logs VSRows. --bench accepts an optional hypothesis argument, --hypothesis flag also works; absent -> "untagged run". - git SHA via exec.Command("git", "rev-parse", "HEAD") inside the harness. - go build / go vet ./... / go test . all pass (10 tests, incl. 5 new iterlog tests). - Added BANKWARS_BENCH_GAMES / BANKWARS_GAUNTLET_GAMES env knobs for tiny smoke benches of the real binary. Next: real-binary smoke run (1 game) to verify iterations.json is written + appends, then commit/push/PR.
Winnie commented 2026-09-29 18:31:40 +00:00
Author
Owner
Copy link

Verification done end to end (no full bench, tiny real-binary runs):

  • --bench "text" inline hypothesis -> record written with hypothesis/git_sha/timestamp, 13 preset rows (wins/plays/avg_eq/max_abs per brain), 10 gauntlet rows.
  • --bench --hypothesis "text" separate-flag form also works; bare --bench logs "untagged run".
  • --bench-vs v0.3.0 -> BenchVS section with 4 rows {current_wins,current_plays,old_wins,old_plays}.
  • Append semantics: 4 consecutive runs -> 4 records in order, none lost; corrupt iterations.json -> error, file NOT clobbered.
  • Added BANKWARS_BENCH_GAMES / BANKWARS_GAUNTLET_GAMES env knobs so tiny smoke benches of the real binary are possible (production defaults unchanged: 100 / 40).
  • go build / go vet ./... / go test -count=1 . green (10 tests, 5 new).
    Smoke artifacts reset (iterations.json seeded as [], bench-results.md restored). Committing + pushing branch 46-iter-log next, then PR.
Verification done end to end (no full bench, tiny real-binary runs): - --bench "text" inline hypothesis -> record written with hypothesis/git_sha/timestamp, 13 preset rows (wins/plays/avg_eq/max_abs per brain), 10 gauntlet rows. - --bench --hypothesis "text" separate-flag form also works; bare --bench logs "untagged run". - --bench-vs v0.3.0 -> BenchVS section with 4 rows {current_wins,current_plays,old_wins,old_plays}. - Append semantics: 4 consecutive runs -> 4 records in order, none lost; corrupt iterations.json -> error, file NOT clobbered. - Added BANKWARS_BENCH_GAMES / BANKWARS_GAUNTLET_GAMES env knobs so tiny smoke benches of the real binary are possible (production defaults unchanged: 100 / 40). - go build / go vet ./... / go test -count=1 . green (10 tests, 5 new). Smoke artifacts reset (iterations.json seeded as [], bench-results.md restored). Committing + pushing branch 46-iter-log next, then PR.
Winnie commented 2026-09-29 18:32:11 +00:00
Author
Owner
Copy link

Done — PR #48: #48 (branch 46-iter-log -> master, review requested from robertismo).

Shipped:

  • iterlog.go: IterationRecord{Hypothesis, GitSHA, Timestamp, Presets, Gauntlet, BenchVS}; appendIteration = load existing iterations.json + append + rewrite (never loses old records; corrupt file errors instead of clobbering).
  • --bench "hypothesis text" inline argument + --hypothesis flag; absent -> "untagged run". git SHA via git rev-parse HEAD inside the harness.
  • Gauntlet per-preset numbers captured into GauntletRow{wins, plays, presets_won} as runGauntlet computes them; bench-vs logs VSRow{current_wins, current_plays, old_wins, old_plays}. bench-results.md output unchanged.
  • Seeded iterations.json as [] in the repo root; first real bench run appends record 1.
  • BANKWARS_BENCH_GAMES / BANKWARS_GAUNTLET_GAMES env knobs for tiny smoke benches (defaults 100/40).

Schema (array of records): {"hypothesis", "git_sha", "timestamp", "presets": {preset: {wins, plays, avg_eq, max_abs per brain}}, "gauntlet": {spec: {wins, plays, presets_won}}, "bench_vs": {brain: {current_wins, current_plays, old_wins, old_plays}}}.

Verified without a full bench: tiny real-binary runs (1 game) for inline/flag/untagged/bench-vs forms, 4 consecutive runs -> 4 preserved records; go build / go vet ./... / go test -count=1 . green (10 tests, 5 new). Ready for review.

Done — PR #48: https://git.machinetele.com/Winnie/bankwars/pulls/48 (branch 46-iter-log -> master, review requested from robertismo). Shipped: - iterlog.go: IterationRecord{Hypothesis, GitSHA, Timestamp, Presets, Gauntlet, BenchVS}; appendIteration = load existing iterations.json + append + rewrite (never loses old records; corrupt file errors instead of clobbering). - --bench "hypothesis text" inline argument + --hypothesis flag; absent -> "untagged run". git SHA via git rev-parse HEAD inside the harness. - Gauntlet per-preset numbers captured into GauntletRow{wins, plays, presets_won} as runGauntlet computes them; bench-vs logs VSRow{current_wins, current_plays, old_wins, old_plays}. bench-results.md output unchanged. - Seeded iterations.json as [] in the repo root; first real bench run appends record 1. - BANKWARS_BENCH_GAMES / BANKWARS_GAUNTLET_GAMES env knobs for tiny smoke benches (defaults 100/40). Schema (array of records): {"hypothesis", "git_sha", "timestamp", "presets": {preset: {wins, plays, avg_eq, max_abs per brain}}, "gauntlet": {spec: {wins, plays, presets_won}}, "bench_vs": {brain: {current_wins, current_plays, old_wins, old_plays}}}. Verified without a full bench: tiny real-binary runs (1 game) for inline/flag/untagged/bench-vs forms, 4 consecutive runs -> 4 preserved records; go build / go vet ./... / go test -count=1 . green (10 tests, 5 new). Ready for review.
Winnie commented 2026-09-29 18:43:23 +00:00
Author
Owner
Copy link

Merged: PR #48 (structured iteration log) is on master. Every --bench and --bench-vs run now appends a machine-readable record (hypothesis, git SHA, timestamp, per-preset rows, per-specialist gauntlet rows, bench-vs rows) to iterations.json, preserving old records and erroring on corrupt files instead of clobbering. Closing; the #47 PDF pipeline consumes this log.

Merged: PR #48 (structured iteration log) is on master. Every --bench and --bench-vs run now appends a machine-readable record (hypothesis, git SHA, timestamp, per-preset rows, per-specialist gauntlet rows, bench-vs rows) to iterations.json, preserving old records and erroring on corrupt files instead of clobbering. Closing; the #47 PDF pipeline consumes this log.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
158-market
157-control
156-records
155-stdlib
151-gauntlet
149-eligibility
147-report
145-vickrey
143-pauses
141-defaults
139-summary
137-version
127-minors
125-replay
126-negbuiltin
123-writeoff
124-revoke
121-panic
122-logf
120-finite
119-shoreup
116-acq-basis
88-failure-calc
113-player-display
merge-super
v050
canonical
105-bond-volume
106-bid-all
100-shelf-reset
100-nondet
exp13
98-tfix
94-amort
80-super-blend
89-t
91-nan
80-super-b
80-super-d
80-super-x4
80-super-x3
80-super-x1
87-let-if
80-super-x2
80-super-r1
80-super-ai2
80-super-ai2b
80-super-c
84-simult
80-super-ai
78-gaap
79-api-clean
75-reserve
73-acq-books
71-liquidity
69-bond-units
64-summary
67-loan-summary
63-churn-display
52-empire
53-mill
60-bench-roster
54-vault
55-adapt
39-main-overhaul
47-report
45-acct
46-iter-log
37-hybrids
40-version-harness
38/punishing-presets
41/fix-ci-checkout
v0.6.1
v0.6.0
v0.5.0
promotion-517
promotion-515
promotion-512
promotion-471
promotion-460
promotion-458
v0.4.0
v0.3.0
v0.2.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
Winnie
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Winnie/bankwars#46
No description provided.