Repository navigation
Archive MiniMax-H3 Ref2VA Ascend 950PR historical evidence - #527
JingjingWu6 wants to merge 1 commit into
Conversation
Preserve a compact searchable evidence core and register the versioned archive while retaining the INCONCLUSIVE verdict. Agent-assisted-by: Codex (implementation and validation) Signed-off-by: JingjingWu6 <JingjingWu6@users.noreply.github.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
首轮静态检视 493416520f537590eb8f2a4e74f276f82e488307 → da8c9f49c6295484caf28ed42da92f91fb674fe1,对照 #520 与归档契约。
- 声明的 v2 replay 与比较器不兼容。
artifacts.json把results/trials.jsonl指定为 replay 输出,但 448 条索引使用trial_id/name/source;tools/evidence_archives.py:466–486读取id,还要求conclusion/artifact_id/source_record/configuration。按现在的格式会得到trial_mismatch,不是 manifest 通过就能验证。请生成符合共用契约的投影,或显式采用受支持的历史索引契约。 - recompute 不能独立重建输出。
validator/recompute.py:5–10读取已有 summary 并核对两个算式,只打印两个标量,不从已认证的原始记录生成所声明的 trials/summary。请补可比对的重建输出;若只交付算术核对,应明确缩小范围和声明。 - 完整归档仍需可取回。 已核对 448 个索引 ID 和汇总计数,但未取得声明的 release 资产来独立核对完整覆盖和摘要。若保留完整归档交付,请发布后回取验证,或提供获授权的可核查存储路径。无需重跑 NPU 实验,历史结论仍是 INCONCLUSIVE。
非阻塞建议:将 7 条 FAILED 等计数标为源标签统计,避免把含非零退出码的 RECORDED 行误解为成功。已静态核对九个改动文件、完整索引及共用比较器;未运行验证器、下载完整资产或执行实验。
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
Reviewed base 493416520f537590eb8f2a4e74f276f82e488307 and head da8c9f49c6295484caf28ed42da92f91fb674fe1 (full diff, first review). pr/prior-reviews.md records no formal APPROVED review on this head and no outstanding CHANGES_REQUESTED, so this is the first formal pass. I also read Issue #520 and its two existing comments.
The framing is honest and the verdict is stated consistently: INCONCLUSIVE appears in README.md, plan/effective.json, results/summary.json, results/external_optimization_record.json and results/artifacts.json → availability.verdict; historical PASS strings are demoted to source observations; 4.2242× (50-step engine stack) and 7.3747× (10-step + Cache-DiT) are kept as separate, non-cumulative comparisons, matching both prior comments on #520. No gain-ranking or acceptance language anywhere in the bundle. The blockers below are pointer and classification integrity, plus a diff I could not finish reading.
Verified in the seeded diff
summary.jsonarithmetic holds:701.954369 / 166.172687 = 4.2242463648…, matchingexternal_optimization_record.jsonaggregate.- Three-seed Cache-DiT mean holds:
(7.71175565987307 + 7.2540855539449725 + 7.158391805120384) / 3 = 7.374744339646142, anddeepseek_rangeis the correct min/max of the same three values. status_countsinresults/summary.jsonsums to 448 (7 FAILED + 404 RECORDED + 4 RECORDED_VALUE + 27 COMPLETED + 6 INCONCLUSIVE), matchingtrial_rows: 448and theresults/trials.jsonlhunk header@@ -0,0 +1,448 @@.- File count and uncompressed bytes in
artifacts.json(2,684 files / 40,740,727 bytes) match Issue #520 exactly.
Blockers
-
Incomplete diff —
diffComplete: false. The seed truncates insideresults/trials.jsonl(hunk header claims 448 rows; I read well under half). Two files the bundle depends on are not visible:validator/recompute.py(named inREADME.md"Retained core",artifacts.json → key_files.validatorandverification.command) andresults/redactions.json(named askey_files.redaction_inventory).results/redactions.jsonshould have sorted betweenexternal_optimization_record.jsonandsummary.jsonand does not appear, so it is either absent from the change or dropped by the seed. Either way I cannot check the arithmetic-replay claim or the privacy-review claim the archive rests on. Please confirm both files are in the head. -
Archive identity changed from v1 to v2 without explanation. Issue #520 pins
h3-ref2va-ascend-history-20261118-v1.tar.gz, SHA-25606b95a85…a2627d,5,537,276bytes.results/artifacts.jsonnow declares-v2.tar.gz, SHA-256778f5c7c…85a0cd,5,454,990bytes, against a release-download URL. Compressed bytes and digest changed while file count and uncompressed bytes did not, which reads as a re-compression of the same tree rather than a re-collection. State which it is: if v2 is a re-gzip of the same 2,684 files, say so and retire the v1 digest in #520 so two competing hashes are not circulating; if the tree changed, say what changed. I cannot confirm the v2 release asset exists or is published, so the "results/artifacts.jsonis the authority for the external archive" sentence currently points at an unverified URL, andverify_evidence_archive.py --downloadis unproven. -
key_files.recordpoints at a path the bundle does not contain.results/artifacts.jsonnamesh3-ref2va-ascend-history-20261118/records/external_optimization_record.json, but the file added by this PR isresults/external_optimization_record.json, andREADME.md"Retained core" also locates it underresults/. Every otherkey_filesentry uses the same relative layout as the repo bundle. Fix the pointer or relocate the file so the key-file inventory can be resolved bytools/verify_evidence_archive.py. -
duplicates: 0is unsupported as written. Many distinctly named attempts share byte-identical retained source files, e.g.sha256 cef20a21…6286onhccl_a,mix8s_a,mix8_aandtp4_03;a7634062…e3dconb0_tp4_s50_d5,rsi_base_trial-001,sweep_b0sub,vbench_b0,vbench_b0sub_rep,vbench_fs8_s6,vbench_s10,vbench_s8,vbench_s9;bf4a9131…83edonres_probe,sci_fi_dense,vbench_b0_s2102/2103,vbench_s10_s2102/2103and allvbench_s1*. Issue #520 already concedes that related documents describe the same named attempt, which is exactly what these shared digests show.duplicates: 0either means something narrower than it reads (duplicate index rows) or is wrong. Define the field, or record the duplicate groups, so a downstream consumer does not read 448 rows as 448 independent observations. -
statusconflates run outcome with historical gate verdict, so failed launches are counted as recorded. Visible rows with non-zero return codes carry"status": "RECORDED":b1_gate0p9_seed2202andl0_tp2usp2_bf16(rc: 52),u0_usp4_bf16andphase0_c03_v032(rc: 22),lossless_tp4_ref_c47(rc: 7,wall_s: 0.0074). Separately, thelocal-summaryrow issource_status: "FAIL"/status: "FAILED"while carrying pooled speedups of 4.33× — a summary record classified as a failed run. AndDETERMINISM_FINDINGputs prose in the status field:source_status: "QUALITY_BAR.JSON INTENTIONALLY NOT FROZEN; RSI_VALIDATE_USP.SH KEEPS FAILING CLOSED."WithFAILED: 7andinterrupted_or_incomplete: 0, the index under-reports failed launches. Enumerate the field (run outcome) and keep the historical gate label in a separate field, or state the classification rule explicitly and note thatmetrics.rcis the authoritative failure signal. -
plan/effective.jsondrops the environment identity that #520 retains. The evidence contract puts the RunSpec and environment underplan/. The 14-line file records backend, hardware, model, scenario, window, source revision and verdict — but not Python 3.12.16, torch-npu 2.10.0.post7, vLLM-Omni 0.32.0+npu, vLLM-Ascend 0.19.1rc2.dev3247+ga26cc45a2, or the missing CANN/driver/firmware identity, all of which #520 states. It also flattens "attempts used 2, 4, or 8 logical Ascend 950PR devices" to"4-8 logical Ascend 950PR devices". For a bundle whose purpose is preservation, dropping the recorded versions and device-layout spread loses the specific thing being preserved. Absence of frozen gates is fine and already explained; absence of the known versions is not.
Not verified here
- No tests or validators run:
validator/recompute.py,tools/verify_evidence_archive.pyandtools/validate_evidence.pywere not executed, and the latter two are not part of this diff, so I did not confirm they exist on base or accept thisartifacts.jsonschema. - Archive bytes not downloaded or hashed; nothing in this review confirms
778f5c7c…against real content. - The visible
trials.jsonlrows do not expose the701.954369 / 166.172687pair used for the 4.2242× claim, andexternal_optimization_record.json → source_refsdigests the truncated file, so that pairing is unchecked beyond the division itself. - CI status on this head was not inspected.
Non-blocking
interrupted_or_incomplete: 0alongsideINCONCLUSIVE: 6needs a one-line definition of what that field counts.base_tp4_c03_v032_profrecordsforeign_requests: 3withrequests_served: 6forrequests_sent: 3; if that run feeds a retained ratio, add it tolimitationsas a contaminated-session observation.- Line endings are mixed:
effective.json,summary.jsonandexternal_optimization_record.jsonare CRLF,artifacts.jsonandtrials.jsonlare LF. Harmless, but inconsistent for a bundle whose integrity is digest-checked.
I have not approved anything and this review does not alter the scope of #520. Push the fixes and I will re-read the new head, including validator/recompute.py and results/redactions.json.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
david6666666
left a comment
There was a problem hiding this comment.
One measurement-boundary detail in the standalone evidence record.
| "aggregation": "recorded_ratio" | ||
| }, | ||
| { | ||
| "name": "10-step-cachedit-three-seed-recorded-speedup", |
There was a problem hiding this comment.
Please label this as diffusion-stage speedup in the standalone record. The indexed timings observation contains both diffuse and e2e columns, and 7.3747× is the mean of the three diffusion ratios, not the end-to-end request ratios. The unit x and recorded-speedup name alone don't preserve that distinction for downstream readers.
Dong1017
left a comment
There was a problem hiding this comment.
The three-seed Cache-DiT check has a reproducible false positive. With the indexed timings unchanged, I could replace the reported 7.3747× result with 1.0× and the checker still exited successfully. The inline comment gives the reproducer and a regression check.
| p = summary['performance'] | ||
| assert abs(p['opus_50_step_recorded_speedup'] - 701.954369 / 166.172687) < 1e-12 | ||
| vals = p['deepseek_seed_group_speedups'] | ||
| assert abs(p['deepseek_10step_cachedit_recorded_speedup'] - sum(vals) / len(vals)) < 1e-12 |
There was a problem hiding this comment.
In a temporary copy of results/summary.json, I set deepseek_seed_group_speedups to [1.0] and deepseek_10step_cachedit_recorded_speedup to 1.0. Running python validator/recompute.py exited 0 and printed deepseek_speedup: 1.0. This assertion takes both sides from the edited summary, so dropping two seeds and changing the remaining value passes. Please derive each expected ratio from the timings row in results/trials.jsonl (b0.diffuse / cache.diffuse for seeds 2101, 2102, and 2103), then compare all three values and their mean with the summary. The single-seed edit above should exit nonzero.
|
#516 has been merged into |
Governing Issue and delivery slice
Refs #520
Knowledge authorship declaration
Not applicable: this change does not modify protected knowledge paths.
Changed artifacts and evidence
h3-ref2va-ascend-history-20261118-v2.tar.gz778f5c7c73cea6ee0f545a59fa91f9a40c79818545f86d49f001d2b36585a0cdevidence/h3-ref2va-ascend-history-20261118/tools/evidence_inventory.jsontools/validate_evidence.py: no errors on the prepared snapshotChecks and dependencies
tools/validate_evidence.py; arithmetic recomputation; JSON syntax checks; targeted privacy scan;git diff --checkafter LF normalization.Remaining work and Issue closure
wujingjing6INCONCLUSIVE.