Repository navigation
docs(adr): ADR-0024 —— 第 8 条对照实验的判决量与结论边界(判决量改 last_candidate;P1 表述为「本形态下无发生机制」) - #141
Merged
Merged
Conversation
…1 表述为「本形态下无发生机制」)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概要
/dfs-grill三透镜(产品体验 / 后端协议 / SRE)拷问「§4.3 第 8 条对照实验」后的结论归档。正式件在 vault(a1a74fe),本 PR 落仓内镜像 + work.log 裁决事件。用户拍板:Q1 按推荐(B+C);Q2 知悉推荐为「不跑 N=20」,但另一 agent 已在执行。为什么需要这个 ADR:判决量在两臂上恒等于 0
pilot 实测(N=5,15 局 / 145s / 2,944 token):
A1/A2 的
violations_of恒返回同一条固定违规 ⇒ 任何候选v = v_0⇒v >= prev_v必命中 ⇒no_improvement⇒ argmin 回退带错原文。所以R_uncovered(A1) ≡ R_uncovered(A2) ≡ 0是算术后果,不是测量结果(tests/test_bench_judge_gaming.py:895自身即钉住a2["R_uncovered"] == 0.0)。后果:①「A1→A2 轮数效应」= 0 vs 0,零信息;② P2 的「A3 优于 A2」退化为「A3 > 0」;③ P1 宣称的「A2 是关键分辨器」分支不可达;④ A1 不等于「今日一次性润色」(它返回带错原文,而一次性润色返回模型候选)。
决策(ADR-0024)
R_uncovered(last_candidate)(同一口径函数、只换记录面),并预注册阴性对照不变量last_candidate(A2) ≈ last_candidate(A3)(两臂 prompt 逐位相同、裁判都不进 messages ⇒ 有差异只能来自采样噪声)。机制事实(代码实测,ADR 的立论基础)
polish_loop七个messages.append站点(工具列表 / Leitpunkte / 裁判异常 / 格式不合法 / 工具结果)没有一处回灌裁判 violations;会话无状态、权重冻结;模型唯一自查通道writing_check不带 nlp(delector/tools/writing_check.py:14)⇒ 语法族对它不可见。⇒ §4.3 被引用以禁言的那句(「模型会学会『通过检查』」)在本架构里没有发生机制。给正在跑 N=20 的 agent(已写进 work.log 事件)
报告 MUST 按 ADR-0024 口径书写,并如实登记已知未决项:无
seed且temperature未设未记(「同输入同输出」对--real是假陈述)· Token 成本走ceil(len/3.0)且只计 completion、不计 prompt(系数登记 UNVERIFIED)· 无逐篇落盘/续跑/重试/连通预检/成本硬闸 · rater 零执行入口却写进交付物清单。下一子计划(回灌通道)的硬依赖
① 裁判 FP 必须先修完(3 类误报全在语法族,而 prose 分流下只有语法族被计数 ⇒ 误报回灌即变成「必须修」的指令;现成标定工具
tools/bench_judge_fp.py);② 回灌内容 MUST 过同一道适用性闸与双限截断(否则散文被喂a1_*格式族违规、诱导模型加称呼语,撞tests/test_server.py:6833契约);③ 三角一致性(循环裁判带 nlp /writing_check不带 nlp / 回灌源)先于回灌;④ 阴性对照 MUST 在动产品之前先跑成基线。本 PR 改动
docs/adr/2026-10-16-adr-0024-gaming-control-experiment-verdict-metric.md(新增,vault 正式件镜像)WORKMEMORY/work.log(追加 DECISION 事件)