feat: builder-only seed + mixed lv1/lv2 calibration for outer loop evolution - #1275

Draft

akashgit wants to merge 21 commits into

mainfrom

factory/run-17122260

Commits on Aug 15, 2026

feat: outer loop v1 phase 4 — merge code, CLI, checkpoints, observability, adaptive timeout

akashgit
and
claude
committed
90b17f8
fix: resolve lint errors and update workflow count assertion

akashgit
and
claude
committed
86b5a17
fix: correct featurebench path and instruction path for DirectEvaluator

akashgit
and
claude
committed
a575c92
fix: remove invalid --disallowedTools flag and resolve patch paths

akashgit
and
claude
committed
a414a99
feat: add outer loop runner script for FeatureBench evolution

akashgit
and
claude
committed
ccabd4f
fix: cap agent timeout and reduce seed builder timeout to 600s

akashgit
and
claude
committed
7c52a33
feat: add evolution runner and report generator scripts

akashgit
and
claude
committed
f1f83b4
feat: interim calibration report — 4/10 instances resolved (all PASS)

akashgit
and
claude
committed
9d732c3
feat: update report with calibration results (4/10 resolved, all PASS)

akashgit
and
claude
committed
e2dc6a8
feat: complete calibration — 10/10 instances resolved (seed score 1.0)

akashgit
and
claude
committed
9186eb2

Commits on Aug 16, 2026

feat: add builder-only seed workflow for outer loop evolution

akashgit
and
claude
committed
46cd018
feat: add lv2 support to DirectFeatureBenchEvaluator

akashgit
and
claude
committed
2e23a3e
fix: lv2 eval — wipe testbed for agent, create backup tar for verification

akashgit
and
claude
committed
4cf1ad1
feat: add lv2 calibration, evolution, and report scripts

akashgit
and
claude
committed
c1c0726
data: builder-only seed scores 100% on lv1 — confirms lv1 is too easy

akashgit
and
claude
committed
0ed74f8
feat: complete lv2 calibration — 0/10 instances resolved (seed score 0.0)

akashgit
and
claude
committed
032467f
data: lv2 calibration 0/10 resolved — builder-only fails on hard tasks

akashgit
and
claude
committed
6b44e86
feat: add mixed lv1+lv2 evolution runner for meaningful variance

akashgit
and
claude
committed
811ea13
report: interim outer_loop_v2 — calibration results, evolution in progress

akashgit
and
claude
committed
745f231
report: outer loop v2 — lv1 ceiling, lv2 floor, prompt > structure

akashgit
and
claude
committed
f2b1b57
data: final session artifacts — detailed report, checkpoint, progress events

akashgit
and
claude
committed
197ac10