fix: eliminate no-op knob mutations

[lambdabaa]

Summary

Two fixes to mutate_knob() that waste evaluation slots:

1. Guided no-op detection: when the reflector suggests a value that matches the current value (theory -> theory, False -> False), skip it and fall through to random selection instead of evaluating an identical config.

2. Exclude _prompt_* synthetic knobs: these are created by PROMPT_MUTATE (fix: PROMPT_MUTATE persists in knob_values for compile() round-trips #1411) to persist prompts through compile() round-trips. KNOB_MUTATE should not randomly select them since they contain full prompt text, not tunable parameter values.

Discovered in the chess demo: ~15% of knob mutations were no-ops (theory -> theory, 2.0 -> 2.0, False -> False), wasting evaluation budget on configs identical to the parent.

Test plan

• All 35 mutation tests pass
• ruff check + mypy clean

🤖 Generated with Claude Code

[lambdabaa]

@ceo-review

[github-actions]

✅ Factory Review: KEEP

Verdict: KEEP
Reason: QA: CLEAN — 2670 tests pass, 35/35 mutation tests pass, ruff+mypy clean, 5/5 adversarial tests verified with evidence (no-op detection, prompt* exclusion, mixed knobs, guided path, regression)

QA Analysis

Adversarial QA — PR #1414: fix: eliminate no-op knob mutations

Detected project type: Library (Python CLI with importable modules)
File changed: factory/outer_loop/mutations.py (+10, -2)

Smoke Test

Not applicable — no smoke test defined in factory.md. Proceeded to feature-specific tests.

Test Plan (derived from PR scope)

The PR makes two changes to mutate_knob():

1. Guided no-op detection: When the reflector suggests a value that equals the current value, skip it and fall through to random mutation.
2. _prompt_* exclusion: Exclude synthetic _prompt_* knobs from KNOB_MUTATE random selection (they are handled by PROMPT_MUTATE).

Five tests verify these behaviors and check for regressions.

---

Test Results

Test 1 — Guided no-op detection

Criterion: When reflector suggests style=theory and current value is already theory, mutate_knob() must never produce theory as the new value.
Status: VERIFIED

Command:

uv run python test_adversarial_knob.py

Evidence:

Ran 200 seeds with guided suggestion style=theory (current=theory)
Total mutations produced: 200
No-op mutations (value stayed 'theory'): 0
Sample values: ['broad', 'broad', 'broad', 'focused', 'focused', 'focused', 'focused', 'broad', 'broad', 'broad']

The function correctly detected the no-op guided suggestion and fell through to random selection from bounds, producing only broad or focused — never the current value theory.

---

Test 2 — _prompt_* exclusion (all knobs are _prompt_*)

Criterion: When ALL knobs start with _prompt_, mutate_knob() must return None (not crash with IndexError).
Status: VERIFIED

Command:

uv run python test_adversarial_knob.py

Evidence:

Ran 50 seeds with only _prompt_* knobs
None results (correct): 50
Non-None results: 0
Exceptions (e.g. IndexError): 0

No crashes, no mutations attempted on synthetic knobs. Returns None cleanly.

---

Test 3 — Mixed knobs with _prompt_*

Criterion: When knobs include both _prompt_builder and style, mutate_knob() must NEVER select _prompt_builder as the target.
Status: VERIFIED

Command:

uv run python test_adversarial_knob.py

Evidence:

Ran 200 seeds with knobs: _prompt_builder, style
Total mutations: 200
Mutations targeting _prompt_*: 0
Mutations targeting 'style': 200

All 200 mutations correctly targeted style, never _prompt_builder.

---

Test 4 — Guided mutation with different value still works

Criterion: When reflector suggests style=focused and current value is broad, the guided path should produce focused approximately 70% of the time.
Status: VERIFIED

Command:

uv run python test_adversarial_knob.py

Evidence:

Ran 200 seeds with guided suggestion style=focused (current=broad)
Times guided value 'focused' was selected: 176
Expected: ~70% of mutations should be guided → 'focused'
Ratio: 88.0%

The guided path works correctly. The 88% rate (vs expected ~70%) is within normal variance due to the two-stage random gating (70% guided chance, then choice among valid suggestions).

---

Test 5 — Regression: basic random mutation still works

Criterion: Without a reflection report, mutate_knob() should still mutate knobs within their bounds normally.
Status: VERIFIED

Command:

uv run python test_adversarial_knob.py

Evidence:

Ran 100 seeds with no reflection report
Total mutations: 100
Unique knobs mutated: {'style', 'depth'}
Unique values produced: {'4.0', '1.0', '2.0', 'theory', 'focused'}

Both knobs were mutated, and all produced values are within the declared bounds. No regression.

---

Acceptance Criteria Verification

#	Criterion	Status
1	Guided no-op (suggested value == current) falls through to random	VERIFIED
2	All-_prompt_* knobs returns None, no IndexError	VERIFIED
3	Mixed knobs never selects _prompt_* for random mutation	VERIFIED
4	Guided mutation with different value still applies correctly	VERIFIED
5	Random mutation without reflection is unaffected (regression)	VERIFIED

Adversarial Verdict: PASS

All five acceptance criteria verified with evidence. Both fixes work correctly and no regressions detected.

---

Posted by Factory CEO