MUSE 106-case sweep runner (DeepSeek-V4.1-Flash + kernelCAD)

[w1ne]

Five workstreams on one branch (run → fix → experiment → adopt winners).

1. Sweep runner

Resumable parallel MUSE batch runner, OpenAI-compatible client, generateCase/scoreCase split, judge wrapper, reports, spawn timeouts, all 106 tasks imported.

2. Model quality gates

No-shape gate, interference repair hints, truncation continuations, compound-solid interference.

3. Prompt compaction — opt-in, not adopted

Presets + builder + retrieval; paired A/B showed compaction hurts (16/12/10 vs 16/58) — default stays full.

4. Tool-loop agent arm — adopted default

evaluate_script({code}) iteration with provenance; paired A/B 39/58 vs 16/58 sandbox. --repair-loop keeps the one-shot baseline.

5. Best-verified artifact + design-intent protocol — adopted

• Best-verified selection (d4e2fee9): submit fence → last clean-evaluated → last evaluated, with a one-shot fence-failure fallback.
• Design-intent protocol (c5ab5530): verify the spec's must-have features before finishing (diagnostics-clean ≠ good design).

Full-106 progression, same driver (DeepSeek-V4.1-Flash):

Arm	Sandbox	Overlap	Judged	Final
repair loop	24.53%	15.09%	16	11.79
tool loop, last-evaluated	54.72%	41.51%	44	16.19
tool loop, best-verified	97.17%	75.47%	80	19.97
+ design-intent protocol	96.23%	75.47%	80	25.00
bare DeepSeek (official CadQuery, one-shot)	54.72%	n/a	54	27.83

Final improved 2.1× over the repair loop; sandbox 96% vs bare's 55%; judged 80 vs 54. Remaining gap is per-case judge-rated design quality (paired mean 0.396 vs 0.562 on shared cases).

Verification

lint 0 errors, typecheck clean, targeted suites green, per-task reviews with fix cycles. Full tables + honest reads: kernelCAD-private/docs/process/running-muse-benchmark.md §7. Artifacts gitignored.