Skip to content

[EPIC] Budgets, model checking, fault injection and registered acceptance #8

Open

Description

@jamespud

Parent roadmap: #2 · 设计修订 4:https://github.com/jamespud/MAGI-system/blob/6fb34f052c33386bffd0e123ab5b36556a4c5415/docs/magi-final-personality-acceptance.md · 静态核对代码基线:98c69a14b274bc1d74561b01bd4de2f1e81f4c33

目标与设计追溯

适用章节:§15–20(尤其 §18 ROLLOUT-LAB-1.0)。主要不变量:VOTE-01..03、STAT-01..03、OPS-01 以及 §19 全量测试目录。本 Epic 组织阶段性实现与测试,不直接等同最终验收通过;具体任务须由 Implementation 模板 单独拆解并说明非目标。

已知实现差异(历史静态核对;现状见末尾对账)

现有 BenchmarkDataset/GoldenCase/Judge/CounterfactualStability/CI 是质量回归基础,但不是冻结的四组配对比较、独立盲评、23 项确认性声明和独立风险界。当前默认超时及 token 配额非分阶段活动期限/角色总预算。现有 LLM Judge 可辅助诊断但不能单独签发最终效用标签。

交付切片(进入 Ready 前各自拆为可单独验证的子 Issue)

  1. 实现角色和任务级工具/输入/输出预算、候选提交/异议/修订/冻结窗口、300s 活动期限与 WAITING/ARBITRATION 单独时钟;逐类独立指标报告。
  2. 构建与生产裁决器分离的独立单候选真值表及 k=1..4 有界/符号模型验证,提交范围与等价覆盖证明;真实注入 §13 所有提交切点和参考域故障模型。
  3. 冻结 AcceptanceManifest:模型、PersonaSpec、DomainProfile、数据、场景族、种子、标签、预算、工具与声明族;四组 SINGLE / SINGLE_THREE_LENSES / THREE_UNPERSONALIZED / MAGI 等外部权限保护。
  4. 实施 4,800 独立重置配对回合,独立场景规则及外部结果标签,开放文本双盲评+第三人裁决;23 项确认性声明的 alpha、LCB、零事故上界按 §18 原文不事后修改。
  5. 将硬不变量、人格行为、主效用、严重风险及 OPS/SLA 分开发布证据;未满足或未测试项显式标记,不凭样本数或全绿 CI 宣称最终通过。

阶段退出所需证据

  • 预算耗尽、等待期间时钟暂停、活动期限 300 秒和执行另设最长 990 秒不混淆。
  • 投票真值表/符号覆盖、并发取消/撤销/见证及旧代次 fault injection。
  • 四组严格配对、标签独立、23 项预登记 alpha 分配、风险上界及不可判定样本处理;清楚区分开发回归与封存验收。
  • 每个交付切片标注规范条款/不变量、代码基线、依赖、可运行的正反例和 RED→GREEN/模型检查/故障注入证据。
  • 更新 README/API/迁移、观测和能力边界(如适用);保留未覆盖项,不把模拟通过误报成完整最终验收。

依赖、范围及新发现分流

主要依赖:须等待核心协议、身份/出口和状态投影有可运行版本;可并行开发独立参考裁决器和验收基础设施,但封存实验不得提前用开发样本冒充。

明确非目标:本 Epic 不批准更改最终规范,不承诺特定生产环境的写操作,不在接口契约未明确时要求一次性大 PR 重写所有子系统。

待拆解事项(在当前阶段准备实施前补充)

  • 复核最新 main 与当前未合并 PR,移除已修复或过时的旧基线差异。
  • 标出交付切片的真实阻塞依赖、可并行范围及迁移顺序。
  • 对实施中新增、非阻塞的问题记录不变量/证据/原因;仅在排期和 DoD 明确后新建 Issue。

管理限制:同一不变量的新反例先补当前 Issue;独立阻塞 Bug 拆分并建立阻塞关系;需要降低或修改规范时先走 Design Change。父级路线图的 WIP 上限适用于本 Epic 所有在制任务。


2026-10-23 · Implementation progress reconciliation (main 37ec45997a62e4e2829f69e7384ed6f1a6c4447e)

已有开发回归证据: #17/T1–T6 的真实 MySQL Ownership/ABA/切换门禁;PR #33 C0 跨语言 Golden;#34 C1-A 结构负例;#35 ReadSet 类型身份;#36 DomainProfile Golden/语义负例;#37 三人格机器码基线/跨语言 Golden/预算组合。#37 终版 CI 39952872733 6/6 success,两项真实 MySQL Job 记录 143+47 个 PASS 事件、0 SKIP/FAIL(含子测试;不代表新协议端到端 MySQL 验收)。
不得提前宣称: Persona 实际最低调查、PER-01..03/§17.5、27 单候选真实投票、k=1..4 独立裁决器、分布式撤销与出口故障、四组 4,800 配对回合/23 项统计声明仍未运行或未集成。将 CI/静态检查、MySQL 旧路径、行为实验、最终统计验收分列。
下一步: 保持 Open;CAS-10 未来建立独立证据矩阵并登记 NOT RUN/BLOCKED;不得以此次进度同步替代封存验收。


2026-10-24 · PR #39 merged and C1-C1 scoped

  • New static/offline evidence from PR #39, final PR-head CI #40049521968: 6/6 jobs, 7 positive/88 named negative v2 fixtures, 9 full artifact + 1 risk-input Python/Go Goldens and 28 fixed-version C1-B1 Go/Python/Schema parity cases. F39-01/F39-02 regression tests were reviewed fixed. Identity/structure contracts are verified at the fixture level only.
  • Existing MySQL CI covers legacy generation/rollout behavior, not new v2 EVIDENCE_CLOSED transaction, real controlled sessions, C2/C3 races or Rev4 acceptance. Main merge commit f0277e15c356b426ee25be995d9f49c929534ec8 has separate push workflow #40050376341 (status to check; not equated with PR-head CI).
  • Future #40 must add ContextManifest independent canonical Golden and semantic negative matrix; this is planned, NOT RUN until a real PR. Continue to mark 27 ballot patterns, k=1..4, provider/revocation/Egress faults, PER-01–03 and 4,800-round experiment as NOT RUN/BLOCKED; [EPIC] Budgets, model checking, fault injection and registered acceptance #8 remains Open.

2026-10-24 · PR #41 merged; #40 completed; C1-C2A #42 scoped

New development-only evidence: #40/merged PR #41 at main 737ee9381ec0fdc57d26e605d557a844bdefd3d1; final PR Head CI #40055630479 6/6. Two independent ctxm_ full-body Python/Go Goldens (12-descriptor / 6-descriptor), 127 Context PASS events/2 explicit runtime/authority SKIPs, C0/Domain/Persona/v2 legacy Golden and mirror Schema parity/drift gates. F41-01/02 source-reviewed fixed. Two old-path MySQL jobs 143+47 PASS events, 0 SKIP/FAIL; not C1-C2 MySQL semantics. A separate merge-push CI #40056497201 must be checked before reporting merge-head success.

#42 C1-C2A is merely a newly scoped future ReadSet v2 offline consumer: status PLANNED/NOT RUN, no evidence yet. Retain OPEN for all Rev4 registered acceptance still absent: trusted actual reads, provider delivery and transaction failure cuts, true multi-candidate vote cases/independent checker, C2/C3 send races, personas and §18 experiments / 4,800 paired rounds / 23 claims.


2026-10-24 · Joint Stage #43 activated as tracked execution programme

Stage tracking only: #43 coordinates seven independently testable children #42, #44–#49, with first #42 and #45 eligible for separate parallel Draft PRs and maximum two active Implementation/Bug PRs. Final release must separately show real MySQL transaction fault cuts, trust-chain/epoch races, provider delivery completeness, independent witness/Egress and unknown outcomes; all currently NOT RUN/BLOCKED until their implementation. Existing offline PR #39/#41 Golden and old-generation MySQL tests do not satisfy this work. Stage pass is not §17–20 persona/vote/4,800-round acceptance. Epic #8 remains Open.


2026-10-24 · Stage #43 first-wave #51/#50 merged; #42/#45 complete

New development/offline source-backed evidence: merged PR #50 (c2c56912) ReadSet v2 exact rset_, graph and cutoff negative tests, F50-01 expiry; merged PR #51 (c818d297) C2/C3 design-only typed interfaces, 108 static traces, F51-01 independent immutable artifact/row versions. Final PR Head CI each 6/6; merge-main #40077327760 and #40077747132 both 6/6 success. Existing MySQL checks still cover legacy Generation only.

NOT RUN/BLOCKED: authentic Provider/session read completeness, real Journal MySQL and C2/C3 conditional transaction with revocation, independently deployed witness, sole Egress, atomic EVIDENCE_CLOSED, independent 27 vote patterns/k=1..4, personas and 4,800/23 study claims. This is stage milestone evidence, not full Rev4 acceptance; Epic #8 remains Open.


2026-10-24 · Stage #43 second wave #52/#53 merged; gated slices only

New development tests, not Rev4 acceptance: PR #52 final head 79f01c2 CI #40095754907 6/6, including real MySQL root/proof/credential rotation, owner consent, revocation races, command replay and P0 credential-scope closure regressions. PR #53 head bb189996 CI #40100073962 6/6, including 21 top-level MySQL ReadSetJournal tests, owner/Generation ABA, lock-wait stale lease/freshness F53-01, duplicate/UNKNOWN/replay and append-before-exposure fault cuts.
Not proven: Real provider session read/application, complete source interception, production root bootstrap, at-use same-order CommitIfCurrent, independent witness, protected sole Egress, three-role EVIDENCE_CLOSED, real fault model/network partitions, full Rev4 voting/persona/4,800 pairings and 23 claims. Merge-main CI #40101137176 was in_progress when synced; don't borrow successful PR-head results. CAS-10 remains Backlog.


2026-10-24 · M1/M2/M3 outcome-gated ROLLOUT-LAB-E2E-01

Roadmap #2 now defines a fixed illustrative and inspectable ROLLOUT-LAB-E2E-01 integration case and M1/M2/M3 gates. This is planning only; no actual complete end-to-end run, independent provider ReadSet, authorized SEND or formal Rev4 acceptance is claimed.

  • M1 test oracle: actual 3 controlled provider/persona sessions, real shared delivery verification, exact Context/ReadSet/EVIDENCE_CLOSED and roles, real immutable ballots + independent decision/selection, report source/authority/dissent reconciliation, no protected SEND; injected missing receipt/revoke/late material/invalid candidate cannot be falsely accepted.
  • M2 test oracle: isolated freshly authorized R1 C2 5% canary under Rev4 duration/error/recovery safeguards, verified CURRENT C2/C3 common order, independent witness + unique Egress, remote FENCED_IDEMPOTENT response reconciliation/UNKNOWN. No real production rollout or legacy self-issued permissions.
  • M3 oracle: independent §19 formal/implementation/per-persona evidence and §18 four model baselines with 24 scenario families, 4,800 independent paired rounds and 23 frozen multiplicity-controlled claims. AcceptanceManifest and statistical preregistration are not presently frozen; do not run/peek at confirmatory evaluation before independent readiness.
  • Engineering now: CAS-10 should prepare invariant→consumer→fixture/test→evidence→gate/status trace matrix, deterministic scenario runner specs and failure taxonomy; independent final verdict remains NOT RUN/BLOCKED. Existing CI/go/MySQL successes from PR feat: validate offline ReadSetSnapshot v2 semantics #50–feat: add durable ReadSetJournal registration #53 are only component evidence.
  • Project pressure rule: complete meaningful R0/R1 E2E assertions before increasing number of generic abstractions. The M1/R0 trace must never be marked PASS using untrusted fake receipt, fixture-only ballot or unresolved qualification. Keep Epic [EPIC] Budgets, model checking, fault injection and registered acceptance #8 Open.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions