You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Single-pass Qwen3.5-4B generation in Proven-Commons/mcproof PR 3 hit the pre-registered Phase 5 Honest Kill Boundary in siliconcordat/mde issue 4, scoring 0/96 on sealed private_holdout.jsonl across bare B0 (0/96), weftBetaEditP2-B (0/96), and 32,216-token Plane 1 prompt injection (B1, 0/96, 2.05x slower).
Because the full in-context Plane 1 ceiling also scores 0/96 in a single pass on 4B, deep Lean 4 tactic closure and dual-oracle code synthesis need iterative Tier 2 compiler-in-the-loop repair rather than single-shot generation.
Scope
Build a deterministic multi-turn Plane 1 retrieval and Tier 2 diagnostic repair harness in src/mcproof/ without weakening any existing Tier 2 oracle (lake comparator --paranoid, >= 80% mutation kill floor under python3 -O -B -W error, f0 gate, check_verdicts.py, and mde guard --code-audit):
Plane 1 section retriever: Given a Family A, B, C, or D prompt and any structured Tier 2 oracle findings from turn t, retrieve only the matching sections of spec/plane1_formal_spec.md (<= 4,096 tokens per turn instead of 32,216 tokens).
Bounded Tier 2 repair loop (k <= 4 rounds): Feed deterministic mcproof --json grade diagnostics (Lean kernel line/column errors, surviving planted mutations, missing What is not proved / What you must trust sections, or data-eido-skip omissions) back into Qwen3.5-4B (B0 vs P2-B mounted through warp-modeld) with temperature = 0.0, top_p = 1.0, and per-turn slot erase.
Evaluation protocol: Measure pass@1..4, total tokens consumed, wall-clock latency, and 8/8 safety-canary preservation on datasets/validation.jsonl (48 items), followed by a single blind verifier run against sealed private_holdout.jsonl (96 items).
Continuity update: Record measured multi-turn pass rates and token-efficiency trade-offs (B0 + retrieval + Tier 2 loop vs P2-B + retrieval + Tier 2 loop) in docs/CONTINUITY.md.
Automated posting by agentic team with human oversight.
Wave 1 single-pass baselines without retrieval or repair scored 4/48 (8.33%) on public validation.jsonl (B0) and 0/96 across B0, B1 (32,216-token full-spec prompt), and P2-B on sealed private_holdout.jsonl.
Findings and Companion Adapter Fix
Targeted retrieval + Tier 2 diagnostics: Public validation pass rate doubles (4/48 -> 8/48) and sealed holdout climbs off zero (1/96 on both B0 and P2-B) while cutting per-turn Plane 1 prompt context from 32,216 tokens (B1) to ~1,200 tokens (26.8x smaller).
Strict label-free inference stopping: Repair turns use only Tier 2 syntax, Lean v4.35.0--trust=0Meta.isDefEq statement checks, AST sandbox checks, and prose gates (0 hidden holdout fixtures or expected verdicts exposed).
Single-layer P2-B (blk.31) 0/48 root cause: Applying the full edit norm at blk.31.ffn_up incurs finite-perturbation RMSNorm radial base-logit shrinkage (s = 0.00268, c = +0.01088), attenuating </think> under enable_thinking: false and knocking greedy decoding into chain-of-thought rambling.
Four-layer TangentBetaEdit fix: Splitting SwiGLU/RMSNorm tangent-projected BetaEdit across blk.28..31 in siliconcordat/weft PR 5 cuts per-layer radial shrinkage 8.58x, preserves clean </think> emission, and lifts live llama-server triple-LoRA T-C1 step-0 retention from 0.00% to 90.73% (24/24 anchors preserved).
What Is Not Proved
Qwen3.5-4B remains capacity-bound on multi-step Lean 4 tactic synthesis (0/24 Family A on sealed holdout) and adversarial mutation-hardened Python checkers (0/24 Family B on sealed holdout) within k <= 4 greedy turns.
Re-running the full k = 1..4 repair sweep with the 4-layer blk.28..31TangentBetaEdit formal-verification adapter is left to follow-up benchmarking.
Activity
ahrazzle commentedon Nov 19, 2026
Bounded multi-turn Plane 1 retrieval and label-free Tier 2 compiler repair (
k <= 4turns,temperature = 0.0,top_p = 1.0,seed = 42,enable_thinking = false,<= 1,250retrieved Plane 1 tokens per turn) are implemented in Proven-Commons/mcproof PR 6 and evaluated on bothdatasets/validation.jsonl(48items) and sealedprivate_holdout.jsonl(96items).Measured
pass@1..4Results (Qwen3.5-4B-Q8_0.gguf,10cc391b...)items)k = 1k = 2k = 3k = 4k = 4(replies)validation.jsonl(48)B0+ Plane 1 retrieval + Tier 2 loop5/48(10.42%)6/48(12.50%)7/48(14.58%)8/48(16.67%)297,817(169)validation.jsonl(48)P2-B(blk.31) + retrieval + loop0/48(0.00%)0/48(0.00%)0/48(0.00%)0/48(0.00%)366,515(192)private_holdout.jsonl(96)B0+ Plane 1 retrieval + Tier 2 loop0/96(0.00%)1/96(1.04%)1/96(1.04%)1/96(1.04%,HOLD-C-04)779,756(339)private_holdout.jsonl(96)P2-B(blk.31) + retrieval + loop0/96(0.00%)0/96(0.00%)1/96(1.04%)1/96(1.04%,HOLD-D-10)818,160(336)Wave 1 single-pass baselines without retrieval or repair scored
4/48(8.33%) on publicvalidation.jsonl(B0) and0/96acrossB0,B1(32,216-token full-spec prompt), andP2-Bon sealedprivate_holdout.jsonl.Findings and Companion Adapter Fix
4/48 -> 8/48) and sealed holdout climbs off zero (1/96on bothB0andP2-B) while cutting per-turn Plane 1 prompt context from32,216tokens (B1) to~1,200tokens (26.8xsmaller).v4.35.0--trust=0Meta.isDefEqstatement checks, AST sandbox checks, and prose gates (0hidden holdout fixtures or expected verdicts exposed).P2-B(blk.31)0/48root cause: Applying the full edit norm atblk.31.ffn_upincurs finite-perturbationRMSNormradial base-logit shrinkage (s = 0.00268,c = +0.01088), attenuating</think>underenable_thinking: falseand knocking greedy decoding into chain-of-thought rambling.TangentBetaEditfix: Splitting SwiGLU/RMSNormtangent-projectedBetaEditacrossblk.28..31in siliconcordat/weft PR 5 cuts per-layer radial shrinkage8.58x, preserves clean</think>emission, and lifts livellama-servertriple-LoRAT-C1step-0 retention from0.00%to90.73%(24/24anchors preserved).What Is Not Proved
Qwen3.5-4Bremains capacity-bound on multi-step Lean 4 tactic synthesis (0/24Family A on sealed holdout) and adversarial mutation-hardened Python checkers (0/24Family B on sealed holdout) withink <= 4greedy turns.k = 1..4repair sweep with the 4-layerblk.28..31TangentBetaEditformal-verification adapter is left to follow-up benchmarking.