Balance: distributions shift check vs 2026-05-30 sweep #98

Status, 2026-09-27

Done: none of the follow-ups. PR #206 updated only the timing figures in docs/balance-harness.md; its sample sweep is still the 2026-05-30 table.

The numbers below were measured on let-go v1.11.1; main now pins v1.13.0 (PR #195). Per the 2026-09-20 comment on issue #187, the let-go version alone moved mean survival from 132.2 to 163.4 turns on an identical tree, because process-creatures takes acting order from map iteration order.

To close this issue:

• Issue Creature turn order follows map iteration order in process-creatures #187 (stable creature turn order), so sweeps compare across let-go versions.
• Re-run the sweep on the current pin, then work the five follow-ups below: confirm the heavy-melee reversal, check the depth-2 troll, decide whether the :damage rune is meant to be a lever, refresh the docs/balance-harness.md baseline, and investigate the :ensouled/:passive-rest stalls.

Findings: distributions have shifted since the 2026-05-30 sweep (release prep)

Re-ran the survivability harness (xsofy/balance.lg) against current main to refresh the balance picture before release. The harness itself is unchanged since the original sweep, so the differences below are gameplay drift (combat / items / bestiary / world), not harness changes.

How it was run

Norman's exact reproduce from docs/balance-harness.md, same seed for a clean comparison:

(b/explore {:n-runs 40
            :loadouts [:starter :starter-with-damage-rune :heavy-melee :ranged]
            :policies [:greedy-melee :greedy-descend :random :passive-rest :ensouled]
            :max-turns 250 :target-depth 5 :seed 7})

• xsofy: main @ 160a09d
• let-go: v1.11.1
• Wall time: ~41s. A minimal smoke run (2 runs, target-depth 2) passed clean first — no crashes, distributions populate.

Then vs now (same seed 7)

reach-rate is 0.0 across the board in both runs — no heuristic bot reaches target-depth 5. Read the comparative numbers (loadout vs loadout), not the absolute reach rate.

To separate signal from single-seed noise, I re-ran the same sweep across three seeds (7, 17, 42 — 120 runs total), aggregating per-loadout:

What changed

heavy-melee flipped from worst to best. The 2026-05-30 finding flagged it as the worst performer (63.5 mean-turns, dying faster than starter), with the long-sword's speed penalty as the suspected cause. It now leads on both survival metrics — 119.4 mean-turns and the fewest deaths (12/30) across the three-seed run, so this is robust rather than seed-7 luck. A combat or weapon-speed change since 2026-05-30 is the likely cause, and it's the first thing to confirm — it inverts a documented finding.

Overall survivability is up. Fewer deaths and more runs surviving to the turn cap (stuck) across every loadout.

Killer mix moved. The old sweep had slimes dominant across all loadouts. Now kobolds lead (3 of ranged's deaths), spiders punish heavy-melee, and a troll showed up as early as depth 2, much shallower than its usual spawn range.

What held

• reach-rate 0.0 is stable. Mid-run, greedy-descend did reach depth 4 (stuck at turn 247), so descent works; the heuristic bots just don't finish a 5-deep run. Absolute reach rate reflects bot quality plus early-game lethality, not just balance.

What didn't survive the multi-seed check

• The :damage rune's survival edge. On seed 7 alone, starter-with-damage-rune looked clearly better than plain starter (101.0 vs 67.9 mean-turns), echoing the 2026-05-30 finding. Across three seeds the two are even — 100.1 vs 102.0 mean-turns, with the rune loadout dying slightly less often (14 vs 16/30). The single-seed gap was mostly noise, so the rune is not currently a reliable survival lever. Worth a closer look if it's meant to be one.

Follow-ups for the release

• Confirm the heavy-melee reversal against the combat / weapon-speed diff since 2026-05-30, and decide whether the new behavior is intended.
• Check the troll-at-depth-2 spawn — variance, or a spawn-curve change leaking it early?
• Decide whether :damage-rune survivability is meant to be a real lever; if so, it currently isn't pulling its weight.
• Refresh the recorded baseline in docs/balance-harness.md to the current numbers.
• :ensouled / :passive-rest still stall (many non-advancing actions → stuck at the cap). Pre-existing policy-quality issue, not new, but it caps how deep those policies probe.

The seed-7 column maps directly to the 2026-05-30 baseline; the three-seed aggregate is what the comparative claims rest on. Wider seed coverage would tighten them further before any retuning.