Phase EL — Evolution Lines

[aberson]

Build doc: documentation/plans/evolution-lines-plan.md
Track: 9 (Capability research / evolve substrate)
Blocked by: Phase 9 (improve-bot-evolve, operational) — this phase generalizes its single-lineage loop.

Goal

Build the population-genetics substrate for improve-bot-evolve. Today evolve runs a single lineage: one bots/current/current.txt pointer, one parent chain (v0 → … → v13), one evolve_pool.json, fitness-tested only against the immediate parent. This phase adds:

1. Lines of evolution — N parallel lineages, each with its own head, pool, and parent chain, scheduled round-robin (sequential, single-flight — respects the SC2 2-client cap).
2. Baseline database — a curated corpus of frozen opponent snapshots candidates are fitness-tested against (a gauntlet), instead of only the immediate parent.
3. Diversity metric + diversity-driven extinction — a behavioral fingerprint per lineage head + a population manager that culls lineages that are both low-fitness and low-diversity (redundant with a stronger sibling), under a population cap.

Motivation: the single-lineage loop converges and re-proposes already-promoted ideas (memory feedback_evolve_priors_diminishing_returns). Parallel diverse lineages with extinction pressure is the standard escape from that local optimum.

Design spine: the current.txt pointer shape is unchanged (40 consumers of current_version() keep working); lineages are an overlay registry and the scheduler flips the existing pointer between heads.

Work (7 steps)

• Phase EL Step 1: Lineage registry + round-robin scheduler #273 Step 1: Lineage registry + round-robin scheduler
• Phase EL Step 2: Baseline opponent database + fitness gauntlet #274 Step 2: Baseline opponent database + fitness gauntlet
• Phase EL Step 3: Behavioral diversity fingerprint #275 Step 3: Behavioral diversity fingerprint
• Phase EL Step 4: Population manager — diversity-driven extinction #276 Step 4: Population manager — diversity-driven extinction
• Phase EL Step 5: Dashboard lineage + extinction surfacing #277 Step 5: Dashboard lineage + extinction surfacing
• Phase EL Step 6: Multi-lineage smoke gate #278 Step 6: Multi-lineage smoke gate
• Phase EL Step 7: Multi-hour observation soak #279 Step 7: Multi-hour observation soak

Build order: largely sequential. Steps 1 + 2 both touch scripts/evolve.py (loop-wrap vs --fitness-mode argparse) — low conflict risk but not file-disjoint, so 1 → 2 recommended. 3 depends on 2; 4 depends on 1+2+3; 5 depends on 1+4; 6 (smoke) needs 1–5; 7 (soak) needs 6.

Acceptance

• Smoke gate (Phase EL Step 6: Multi-lineage smoke gate #278): one real multi-lineage round completes — both lineages schedule, baseline gauntlet runs, fingerprints compute, population manager evaluates extinction, no orphan SC2/python on port 8765, sandbox hook intact.
• Observation soak (Phase EL Step 7: Multi-hour observation soak #279): 2–3 lineages over a multi-hour budget — lineages diverge on fingerprint, ≥1 extinction event fires (or is justified as not-needed under cap), baseline-fitness trends sane, zero orphan processes, clean [evo-auto] commit hygiene.
• --lineages 1 --fitness-mode parent reproduces today's single-lineage behavior byte-identically (existing test_evolve_cli.py unchanged).

Effort

~4–6 days code (Steps 1–5) + ~1 h smoke (Step 6) + multi-hour soak (Step 7).

Kill criterion

If across the Step 7 soak the parallel lineages converge to near-identical fingerprints and extinction never meaningfully fires (no exploration gain over single-lineage), pause the phase — the population machinery isn't earning its wall-clock cost. Fall back to single-lineage evolve + the already-backlogged PFSP-lineage regression gate.

Deferred (not in this phase)

• Species knobs — task ordering, time-in-task. Blocked on Track 7 mini-games (Phases H/I/J, investigation-blocked). The monolith species is "differentiated by which mini-games it practiced" (Hydra investigation §1) — needs that substrate first.
• Build-order/army-composition fingerprint enrichment (v1 = baseline-result vector).
• Truly concurrent lineage execution (bounded by the SC2 2-client cap).

---

Synced from documentation/plans/evolution-lines-plan.md by /repo-sync at 154f7b0