Repository navigation
Conversation
Skill evaluationPLANNED: not executed; there are no task outcomes yet.
Author verification steps (nonpaid; not yet run)
python tools/build_skill_index.py --source-revision 55cb1c5a945bae7e9dbd211b9f6662af143472dd --checkThe declared source pin matches this head's canonical library bytes. The command has not run: retain its exit code and full drift/success output. It checks the generated projection, not claim truth or skill efficacy. Evidence and logs · Retention: 30 days. Plan bindings, budget, frozen exam and advisory detailsAdvisory: obtain missing evidenceRetained evidence availability: planned / unexecuted. No approval or merge authorization is issued.
Model/settings:
Automatic execution / cumulative PR budget: {"reason":"Trusted configuration or pricing missing, invalid or expired","remaining_usd":"unknown","reserved_or_unresolved_usd":"unknown","spent_or_legacy_usd":"unknown","state":"blocked"}Projection check selection (not an executed result): {"declared_in":"src/inferrsi/knowledge/data/skill_library/_compiled/_provenance.json","source_revision":"55cb1c5a945bae7e9dbd211b9f6662af143472dd","status":"ready"}Provider/settings: Uncovered / unsupported paths (10)
|
zengchuang-hw
left a comment
There was a problem hiding this comment.
Independent first review of base 965ab5cf83399a99530949b04bb934a892ee417c → head 63d4e3af83c34dbc54a704af67fc8dfe6957d7e2 (all seven changed files). Prior scope review: #505 (comment) . This is a COMMENT, not approval or campaign acceptance.
Approval blockers
-
P2: refresh the generated Knowledge projection. The new owner-local case files and changed
methods/inference-optimization-casebook/SKILL.mdare not reflected indata/repo_packs/inferrsi/knowledge/common/skills/inference-optimization-casebook.json: itssource_filesomits both EPD resources and still records the previous SKILL hash.tools/build_skill_index.py::source_inputsincludes every file beneath each registered owner;tests/test_skill_index.py::test_generated_index_matches_current_sources_without_historical_git_objectshashes those current inputs, renders the projection, and asserts it equals the checked-in records. The missing resources therefore make that assertion unequal by inspection. This is not an allowed historical projection. Commit the source inputs, regenerate through the supported generator from that source revision (including the shared provenance manifest), and retain passing index-freshness evidence. I inspected this path; I did not execute the test. -
Admission remains explicitly noncompliant. The exact-head declarations log reports
Material AI author 1: model is unapproved or suspended; the declared model/effort are unknown andknowledge-model-policyis failure. Keep that attribution truthful. Before approval, resolve admission through an actually supported, truthful policy path, including an authorized exact-head exception if applicable. A content review does not itself grant that exception. -
Retain independently inspectable validation for the corrected head. The reported 122 targeted passes and off-checkout wheel read are author reports, not independently reproduced results here; the targeted list also omits the index-freshness test above. Current Skill Eval plan/publish jobs succeeded but execute was skipped, which is not a Knowledge test-suite or installed-package pass. Provide revision-bound Knowledge/index and installed-resource checks; do not relabel the reported incomplete full-suite/lint attempt as success. Maintainer scope disposition remains separate.
Independently checked / resolved since the Issue review
- Read the complete patch, AGENTS, Knowledge field/resource/source contract, citation guidance, associated #505 requirements and actual reader/generator/test definitions.
- Retrieved the seven declared files at external snapshot
3ed184e3a83229806a9fa1adb2bc75f1744c6518and independently SHA-256 checked all seven against the source record. - The positive-rate integer min-capacity argument, floor/ceiling selection and incumbent tie behavior match the pinned allocator. The four-GPU crossover condition is correct.
- The six-wave source JSON supports the table, TTFT/queue and cold-work counters. Independently calculated improvements are 11.79983% and 12.73764%, consistent with the rounded 11.8% and 12.7%. No definite numerical transcription defect found.
- The source snapshot, paths and hashes missing at the earlier Issue stage are now supplied. The case declares its JSON in
required_resources; the reader automatically includes its entry body, so the local link closure is present. One case/two findings agree with the inventory changes. - Author-reported / INCONCLUSIVE, preprocessing, fixed output length, finite arrival window, failed first attempt, limited repeats and absent independently verified model/runtime/raw-trace identities remain explicit. These observations do not establish answer quality, SLO qualification or a new hardware result.
Non-blocking: a focused EPD free-text discovery regression asserting the case ID and openable reference would improve future coverage.
Not performed: repository code execution, tests/build/install, GPU rerun, raw worker-trace replay, model-weight/container verification or an answer-quality experiment. Source arithmetic/hash inspection is not experiment reproduction. No merge or branch changes.
|
Updated in 9f89294. The case now specifies per-instance requests/s, the measured integer allocation constraints and unmodeled deployment costs. Added a case-specific reader control for resource bytes, normalized sources, missing required resources and stale revisions. Targeted Knowledge tests report 123 passed in 50.80s. A final-head wheel was installed outside the checkout and its body/source-resource bytes matched the wheel entries. PR body and issue #505 carry the final-head outputs and explicit claim dispositions. The prior automated Skill Eval plan was unexecuted with zero selected comparisons. Agent efficacy and empirical generalization remain unverified. Independent review and maintainer acceptance remain pending. |
zengchuang-hw
left a comment
There was a problem hiding this comment.
Independent incremental review: 63d4e3af83c34dbc54a704af67fc8dfe6957d7e2 → 9f89294c9898a90125d4f848fcaad2a29faa722f (commits 2c4befd1, 9f89294c; both changed files read in full). Previous review: #506 (review) . Also checked author update #506 (comment) .
Improved in this round
- Case text now states per-instance requests/s, feasible four-GPU integer splits and the omitted deployment-cost terms. This is consistent with the bounded capacity model; no new numerical or generalization claim was introduced.
- The new case-specific regression uses the actual KnowledgeReader: source normalization, self-record byte identity, content-revision drift and missing required resource rejection are checked. Reading the implementation confirms these are relevant controls. I did not execute them.
- The external pinned readset and its seven hashes are unchanged, so the prior independent source-number/hash checks still apply.
Still required before approval
- The generated-projection P2 is not fixed. This increment changes only case prose and
tests/test_knowledge_contract.py. I re-fetched the generated casebook card at this head; it is byte-identical to the stale prior card, still missing both EPD resources and retaining the old SKILL hash. The existingtest_generated_index_matches_current_sources_without_historical_git_objectscompares the current resource-derived projection against that card, so this remains a deterministic static mismatch. Describing this slice as “no generated projections” does not remove the existing projection contract. Regenerate through the supported workflow from a commit containing the source files and update shared provenance. Direct case reads can work while the package skill's recorded provenance is stale; I am not claiming the case is undiscoverable. - Exact-head admission is still failure. The latest declarations job explicitly evaluated
9f89294c9898a90125d4f848fcaad2a29faa722fand rejected the unknown material-AI-author model. Keep truthful attribution and resolve via the documented authorized admission path; this review grants no exception. - The reported 123 targeted passes and wheel SHA/output remain author-reported observations without independently inspected run artifacts here. Retain validation for the corrected head, including
tests/test_skill_index.pyand installed-resource checks. Current automated plan/publish succeeded, but execute was skipped; this is not an executed efficacy evaluation or test-suite pass.
Non-blocking suggestion remains an EPD free-text discovery regression.
Performed: complete two-commit diff review, actual reader/test/generator-contract inspection, current generated-card comparison, own-review/head checks and exact-head declaration log inspection. Not performed: repository code execution, local tests/build/install, GPU experiments or raw model/runtime/trace replay. Campaign acceptance remains INCONCLUSIVE; no approval or merge.
zengchuang-hw
left a comment
There was a problem hiding this comment.
Independent incremental review of 9f89294c9898a90125d4f848fcaad2a29faa722f → 455f1c8afd2b796c2a3a40d259b3ded246e98d06. Previous review: #506 (review) . All three changed files inspected.
Resolved: the generated-projection P2. The card now includes both EPD resources and the updated SKILL digest. Independent SHA-256 calculations over retrieved source bytes match all three recorded hashes (SKILL d9b125e0…, case JSON cc427795…, case Markdown 05e64f28…). The shared manifest pins 9f89294c9898a90125d4f848fcaad2a29faa722f, which contains these sources; this increment changes no library source files. No remaining definite projection mismatch was found in the reviewed changes. This is a source/hash verification, not a claim that I executed the freshness suite.
The non-blocking discovery suggestion is also addressed in code: the new test searches EPD cold/MMMU allocation text, selects the case ID and follows its actual read_args to assert body content. I checked that the argument builder requests material text; the test is not executed by me. Prior source-number/math checks and the INCONCLUSIVE experiment boundary are unchanged.
Still required before approval:
- Exact-head admission remains failure. The declarations log evaluates this head and reports
Missing, invalid or stale knowledge author declaration; at inspection the body still names9f89294c. Update declaration coverage truthfully, then resolve the separate unknown model/effort admission issue through the supported policy process. Merely updating the SHA does not make unknown authorship compliant. - Retain independently inspectable validation covering this corrected head, particularly index freshness, the new query regression and installed-resource access. Existing body outputs name the previous head, and the inspected automated plan/publish/skip-execute path is not that test evidence. I have not reproduced the reported targeted or wheel checks.
- Maintainer acceptance and campaign qualification remain separate. No experiment PASS or efficacy claim follows from this projection repair.
Performed: complete incremental diff, current generated metadata, three source-byte hashes, source-pin coverage, query/reader path and current check/log inspection. Not performed: repository code execution, tests/build/install, GPU or raw trace/model/runtime verification. No new blocking code defect found; COMMENT pending admission and validation evidence, not approval or merge.
|
Corrected head is 455f1c8. The P2 generated-projection defect is addressed by tools/build_skill_index.py from committed source 9f89294, refreshing the casebook card and _provenance.json. Both EPD resources and the current SKILL hash are included. Final-head Knowledge/index tests report 133 passed in 60.26s, including all nine tests/test_skill_index.py cases and an EPD free-text hit opened through its returned read_args. Explicit generator --check reports 80 current metadata records. A fresh off-checkout wheel install matches body/self-record bytes and checks generated-card resource membership and source pin. Retained author-executed JUnit, commands, installation script/output and wheel hash. Independent reproduction remains pending. Prior incomplete full-suite/lint results remain disclosed. @zengchuang-hw please re-check the corrected projection and current-head evidence when available. @Dong1017 the PR's truthful unknown model/effort declaration now includes the documented exact-head exception request and all seven protected changed paths. That request requires your separate authorized policy disposition. No content review or local check is treated as granting it. Draft status, maintainer scope acceptance and INCONCLUSIVE campaign qualification are retained. |
|
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5). Decision: exception not granted at Dong-bot (authorized delegate) is recording this so the PR isn't left waiting. Why not now
Content, from the diff (not a verdict)
What would let me revisit
When those are in place, @-mention me for a formal re-review of the final head. |
zengchuang-hw
left a comment
There was a problem hiding this comment.
Evidence-only incremental review prompted by the direct request in #506 (comment) . Head remains 455f1c8afd2b796c2a3a40d259b3ded246e98d06; no code delta since my previous review.
I retrieved and inspected the complete retained validation JSON at a927eaa8, including commands, sanitized JUnit, installation script and outputs. Its JUnit contains 133 test cases, including all nine skill-index tests and the EPD query/closure controls, with zero reported errors/failures/skips. The retained generator output reports 80 current records. These are now inspectable, pinned author-executed evidence rather than only a PR-body summary; I did not execute or independently reproduce them.
The installation script uses a fresh target, isolated Python and a temporary working directory, verifies imported package location, resource bytes against wheel entries, generated-card membership and source pin. Its boundary still matters: it chooses a locally available wheel and prints the tested head as a literal. The published wheel hash and JSON head declaration are not, by themselves, independent proof that the wheel was built from this exact clean checkout. A retained exact-head build/checkout record or independently reproduced installed check would close that provenance gap; I have not downloaded or rebuilt the wheel.
The projection P2 remains resolved by the independent source/hash checks already recorded. The stale declaration SHA is now corrected, and the exact-head exception request lists the protected paths. Admission is still unresolved: the newer declarations log reports both Exception requires explicit independent owner approval and Material AI author 1: model is unapproved or suspended. An exception request is not an exception grant; this COMMENT does not provide policy-owner approval.
Before approval: obtain the authorized policy disposition and complete the remaining verification/provenance evidence. Earlier incomplete full-suite/lint outcomes remain disclosed; no efficacy or H800 campaign acceptance is inferred. No additional source defect found in this no-code-delta follow-up. Not performed: local tests/build/install, repository-code execution, GPU/raw-trace replay, merge or permission changes.
|
Current-state reconciliation for the latest incremental review, checked against GitHub after that inspection:
Thank you for confirming the projection P2 is resolved and finding no new blocking code defect. No additional source change is proposed for this reconciliation. Maintainer scope acceptance, independent evidence review and authorized authorship-policy exception remain pending. Draft and INCONCLUSIVE campaign scope are retained. |
|
Attribution correction from the requester: GPT-6.2 Sol with xhigh reasoning effort. The requester clarified the earlier settings report in this conversation. The PR's knowledge-model-policy block now declares model_id gpt-6.2-sol, effort xhigh, at unchanged head 455f1c8. This records user-confirmed settings, with no independent execution-provenance verification claimed. Current policy admits this model/effort combination. The previous exception request concerning unknown attribution is withdrawn. Please evaluate the refreshed exact-head declaration on its current contents. The declined exception is not treated as approval. Independent review, maintainer scope acceptance and INCONCLUSIVE campaign scope remain separate. The 133-test/index and installed-resource validation artifact remains bound to the unchanged code head: https://github.com/Kelvin-Al/InferRSI/blob/a927eaa88e043312e6692a28c93b9cb0f5f6995c/validation/epd-case-checks.json |
zengchuang-hw
left a comment
There was a problem hiding this comment.
Declaration-evidence follow-up to #506 (comment) at unchanged head 455f1c8afd2b796c2a3a40d259b3ded246e98d06. Previous review: #506 (review) . Code delta: none.
The admission-check blocker is now cleared. I reread the current declaration (exact head, gpt-6.2-sol / xhigh, no exception request), current commit status and the new raw declarations log. The trusted checker evaluated this exact head against policy 965ab5cf83399a99530949b04bb934a892ee417c and returned compliant, no errors, Declaration compliance only. The earlier declined/pending exception is not being treated as granted. This verifies the repository declaration result, not the author's actual model execution provenance; no private session evidence is requested.
The generated-card P2 also remains resolved. There is no new source defect in this metadata-only follow-up. The existing 133-test/index/wheel artifact is unchanged, so the previously documented verification limit remains: the installation script compares resources to its selected wheel and prints a literal tested head, rather than retaining an independently checkable clean-checkout-to-wheel build record. Before I can approve, retain that exact-head build/checkout linkage (and wheel identity) or equivalent independently reproduced installed validation. I have not executed tests/build/install or independently downloaded and inspected the built wheel.
Maintainer scope disposition and formal H800 campaign acceptance remain separate; no new GPU, answer-quality or efficacy evidence is supplied. COMMENT records the meaningful admission improvement while keeping the remaining validation gap explicit. No merge or permission changes.
|
@Dong1017 Could you confirm the Knowledge-only scope in #505 and check whether the review CI needs maintainer approval to run? The authorship check has passed, and the clean-checkout build transcript, wheel and installation results are linked in the PR description. @zengchuang-hw The build evidence you requested is now available for |
|
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5). Scope: confirmed from the diff, as a Dong-bot (authorized delegate) record. Head
There are no runtime or interface changes and no benchmark execution. The case keeps its external, author-reported, INCONCLUSIVE framing.
CI: it most likely needs a maintainer to approve the run, and I can't do that from here.
State of the review on this head
Re-review: @zengchuang-hw is the right person to re-check the build evidence once CI is unblocked. I'm not posting a formal verdict on this mention. The ask was scope and CI confirmation, and the PR is Draft with maintainer scope acceptance and an independent approval still outstanding. @-mention me for a formal re-review once the PR is ready and CI has run. |
zengchuang-hw
left a comment
There was a problem hiding this comment.
Evidence-only incremental review of unchanged head 455f1c8afd2b796c2a3a40d259b3ded246e98d06, prompted by #506 (comment) . Previous review: #506 (review) . Code delta: none.
Resolved: the previously requested clean-checkout-to-wheel evidence gap. I read the complete immutable report.json and verification script. Unlike the earlier literal-head/selected-wheel check, this record captures exact-SHA fetch and detached checkout, measured HEAD, empty pre-build status including ignored/untracked files, committed-blob comparisons, fresh pinned build dependencies, the wheel build command, fresh isolated installation outside the checkout, imported-package location, and comparisons of five installed/wheel resources against committed hashes. The report records wheel inferrsi-0.1.dev1+g455f1c8af-py3-none-any.whl, SHA-256 8506c97a5d3bdb34cf9a80e7d07e3725ca55677c4efff47ddd0e771eb720e2e2, and successful actual discovery/read paths. This supplies the inspectable author-executed provenance record requested in my earlier review; I am not keeping that missing-record blocker open.
Evidence boundary: I inspected the retained report and code, but did not execute it, download/hash the binary wheel, independently reproduce installation, or independently verify the claimed second build. The earlier independently inspected 133-case JUnit artifact, source/hash/numeric review, generated-projection repair and declaration-compliance disposition remain applicable. Current knowledge-model-policy is success. No additional definite source defect was found in this no-code-delta follow-up.
Before approval: the exact-head ready/review CI run remains action_required; its jobs endpoint returns no jobs. That is neither a passing test run nor evidence of test failures. Maintainers should resolve that workflow gate and provide the resulting review-lane checks; I have not approved or rerun the workflow. #505 still explicitly reserves the Knowledge-only scope acceptance to maintainers, so this evidence review does not substitute for that decision.
Non-blocking documentation cleanup: label the earlier wheel hash in #505 as superseded by this clean-build artifact, to avoid presenting two wheel identities without context. The Knowledge-only contribution still records external author-reported observations and INCONCLUSIVE campaign acceptance, not new GPU efficacy, answer-quality, or service qualification. No repository-code execution, branch modification, merge, or permission change performed. COMMENT pending the remaining CI/scope disposition.
david6666666
left a comment
There was a problem hiding this comment.
One limitation of the reversed run order.
|
|
||
| The comparison establishes ranking at this workload and offered rate. | ||
| Two reversed rounds provide an order check. They provide limited statistical | ||
| coverage and no sustained saturation plateau or SLO qualification. |
There was a problem hiding this comment.
The frozen orders are [1, 2, 3] and [3, 2, 1], so 2E:2PD always runs second. Reversal swaps the outer splits but cannot distinguish a middle-position effect from the 2E result. Could we name that remaining limitation here and recommend rotating or counterbalancing all three positions in the next campaign rather than reversal alone?
There was a problem hiding this comment.
Thanks for pointing out that reversal kept 2E:2PD in the middle. We ran a nine-wave cold MMMU follow-up in one four-H800 allocation at the same target arrival rate of 34.27 requests/s. We rotated the encoder counts through [1, 2, 3], [2, 3, 1], and [3, 1, 2], so each split ran once in each position.
2E:2PD ranked first in all three rounds, reaching 26.04, 26.44, and 26.67 PD completions/s when run first, second, and third, respectively. All nine waves completed 720 requests with 720 visual forwards and zero EC or resident-cache hits. The ranking held across the three tested orders for this workload. Each split-position pair was measured once, and predecessor effects remain open.
We've added the original order limitation and the follow-up to the case. Updated case at 255887c · Frozen results
zengchuang-hw
left a comment
There was a problem hiding this comment.
Independent incremental review: 455f1c8afd2b796c2a3a40d259b3ded246e98d06 → 255887cd3633a7e3eb59bd9bfca461641aa76f50 (commits 2dabbfe5, 255887cd; all five changed files inspected). Previous review: #506 (review) . PR base remains 965ab5cf83399a99530949b04bb934a892ee417c.
Conclusion: no new definite blocking source defect found; COMMENT pending current-head validation/CI, not approval.
Independently checked
- Retrieved all five newly declared external files at
a0037f7ddc4b1af5f59fbef0a86b2a5627b69feb; independently computed SHA-256 values match the source record. Read the sweep, launcher, plan, nine-wave results and separately retained six-wave interrupted results. - The complete-result rows match the new table and the cyclic orders: each split occupies each position once. Recalculated 2E:2PD versus 1E:3PD improvements are 12.0695%, 10.6813%, 13.7707%, consistent with 12.1%, 10.7%, 13.8%. All nine published rows report 720 completions, zero failures, 720 visual forwards/images and zero EC/resident hits.
- The new wording correctly identifies the original middle-position confound and does not claim that cyclic position balance establishes predecessor balance, repeatability, answer quality, SLOs or general hardware efficacy. The interrupted record is separate, not silently pooled into the complete rerun.
- Both changed case-resource hashes match the regenerated card. Shared provenance points to
2dabbfe5, containing the revised source files; the final commit changes only generated metadata. The earlier stale-projection finding remains resolved. - The test change preserves ordered source-normalization comparison for both snapshots rather than assuming a single source. Existing byte-identity, revision-drift and missing-resource controls remain intact by inspection; I did not run them.
- The exact-head declaration log evaluated this head against policy
965ab5cfand returnedcompliant, no errors, “Declaration compliance only.” The admission blocker remains cleared; this does not independently verify model execution provenance.
Still required before approval
- The current-head ready/review run is action_required, and its jobs endpoint returns no jobs. Resolve that gate and retain the resulting checks. This is not a test failure, but it is not passing review-lane evidence either.
- Retain inspectable validation for the revised resources/test at this head. The PR reports 133 targeted passes and says clean-checkout wheel verification is in progress; the previously retained JUnit and clean-wheel artifacts establish the older
455f1c8contents, not these changed bytes. The old provenance gap was genuinely closed for that head and is not being reopened; the new head needs corresponding validation coverage.
Non-blocking: update #505's contribution summary, which still describes only the original six-wave/two-round snapshot, to distinguish the new follow-up and retain the Knowledge-only scope boundary. Maintainer scope disposition and formal campaign acceptance remain separate.
Performed: direct incremental patch and affected test/data review, repository guidance and associated Issue scope inspection, five external-file and two generated-resource hash checks, published-row arithmetic, own-review deduplication, exact-head CI/log inspection. No other reviewers' opinions were consulted. Not performed: repository-code execution, tests/build/install, wheel reproduction, GPU rerun, raw trace/model/container verification, or independent proof that the published plan was frozen before measurement. These are author-reported observations with independently checked transcription/hashes, not independently reproduced experiments. No merge, branch change or workflow approval performed.
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
Code/content approval for draft head 255887cd3633a7e3eb59bd9bfca461641aa76f50; this does not mark the PR ready. There is no code delta since my previous review. I independently checked the pinned sources, generated-resource hashes, and reported nine-wave calculations. The exact-head retained evidence covers the recorded clean checkout-to-wheel installation and 133 test cases. This approves the Knowledge-only content change, not campaign or hardware acceptance. Supersedes the CI-based hold: CI is assessed at merge, not during code review. No CI lookup in this pass, no local execution, and no merge.
|
#516 has been merged into |
Agent-assisted-by: Codex / gpt-6.2-sol [effort=xhigh] (author) Signed-off-by: Kelvin-Al <170944733+Kelvin-Al@users.noreply.github.com>
Agent-assisted-by: Codex / gpt-6.2-sol [effort=xhigh] (author) Signed-off-by: Kelvin-Al <170944733+Kelvin-Al@users.noreply.github.com>
255887c to
3c0dc17
Compare
|
@david6666666 Thanks for the guidance. PR #506 is now rebased onto I verified that |
Refs #505
This adds an owner-local Knowledge case for choosing an integer E:PD GPU split from independently estimated stage capacities. It preserves the completed cold MMMU observations as external author-reported evidence with INCONCLUSIVE formal campaign acceptance.
The text distinguishes the integer maximizer of min(e * mu_E, (N - e) * mu_PD), conditional on constant scalable stage rates, from empirical applicability. In the cited four-H800 cold comparison, 2E:2PD ranked first in both reversed-order rounds. The case retains preprocessing, model/runtime identity gaps, fixed generation length, finite measurement window, queueing, first-attempt failure and corrected rerun boundaries.
Rebase onto the #516 layout
Rebased onto
mainat1b826b5f9666a8b734000e1eb2990e582cdef876after #516. The existing inference optimization casebook declarescases/epd-cold-provisioning.mdin itsSKILL.mdand links it under recorded diagnostic options. Both case files retain their previous bytes and source pins. The compiled catalog inputs and coverage page were regenerated from source commit55cb1c5a945bae7e9dbd211b9f6662af143472dd. Review request · Casebook registrationOn head
3c0dc17adfad7e04819025372f4cee74c7e55ff9,inferrsi find "EPD cold MMMU GPU allocation stage capacity" --limit 5 --jsonreturnscase.epd-cold-provisioningfirst. Itsread_argsopens the case, including the position-balanced follow-up. The catalog and coverage freshness checks and targeted Ruff passed. Knowledge and CLI tests reported 203 passed, 2 skipped because the Windows host could not create symlinks. The wheel built from a clean detached checkout of this head was installed outside the source tree. Five resource hashes matched Git. Installedfindopened the case and its source record, and the published wheel was downloaded and rehashed. JUnit, build report, script and wheelPosition-balanced cold MMMU follow-up
The original orders
[1, 2, 3]and[3, 2, 1]kept 2E:2PD in second place in the schedule. The follow-up rotated encoder counts as[1, 2, 3],[2, 3, 1]and[3, 1, 2]. All nine waves ran in one four-H800 allocation on the original node, using the same cold 720-image MMMU cohort, seed, 64-token output and 34.27 requests/s target. Inline review · Frozen plan2E:2PD ranked first in all three rounds, at 26.04, 26.44 and 26.67 PD completions/s from first, second and third position. Its lead over each round's runner-up was 10.7%–13.8%. All waves completed 720 requests with 720 visual forwards and zero EC or resident-cache hits. Each split-position cell has one observation, and the cyclic schedule has limited predecessor coverage. Formal campaign acceptance remains INCONCLUSIVE. Nine-wave results · Interrupted attempt
Earlier validation at
255887cHead
255887cd3633a7e3eb59bd9bfca461641aa76f50uses generated projection source2dabbfe50ddddc872c030ba962b570fa2488e770. Author checks completed for 133 targeted Knowledge/index tests, generator freshness and targeted Ruff. The five follow-up source files were downloaded from their immutable public revision and rehashed. A clean detached checkout of this head built a wheel and installed it outside the checkout. Five installed resources matched the committed blobs, and the EPD discovery result opened the case. The published wheel was downloaded and rehashed. Build report, script, wheel and JUnitScope and delegation
User-authorized preparation of the Knowledge contribution. The diff covers the case and source record, its casebook entry, two navigation matrices, inventory/discovery tests, compiled catalog inputs and the generated coverage page. Runtime interfaces and benchmark execution are outside this slice.
Source snapshot epd-provisioning@3ed184e3.
External experiment issue,
frozen cold plan,
cold results.
The source record owns seven retrieved-file SHA-256 values. A case-specific test on the final head checks normalized source metadata, required-resource byte identity, missing-resource failure and invalidation of a stale content revision. This import is frozen. Updating the external repository leaves this historical case unchanged. New claims require a new pinned readset and affected evidence checks.
Prior verification at
455f1c8and earlier revisionspython -m pip install -e ".[dev]"completed.python -m pytest tests/test_knowledge_contract.py tests/test_knowledge_operations.py tests/test_knowledge_find.py tests/test_skill_library.py tests/test_skill_index.py --tb=shortreported:Built a wheel and installed it with pip into a fresh target directory. Executed Python in isolated mode outside the checkout, confirmed the imported package was in that target, then loaded the installed corpus and read the case and declared source resource:
Recomputed seven published file hashes at the source snapshot. Changed Python test files pass Ruff.
git diff --checkpasses.The full suite attempt with
--maxfail=5stopped with 3 failed, 131 passed, 4 skipped and 2 errors. The reported failures/errors concern CRLF byte expectations, Windows symlink privileges, unavailable os.mkfifo and oversized environment variables in unchanged Anthropic tests. Fullruff check src testsreports 12 F811 findings in four files verified byte-equivalent to this branch's base. Full-suite and full-lint success are not claimed. No new GPU run or answer-quality evaluation was performed for this import.Static contract feedback and automated Skill Eval
Both capacities are requests/s per effective one-GPU instance for the matched workload. The case explicitly gives the measured feasible counts e in {1, 2, 3} and PD = 4 - e. It lists preprocessing, embedding transfer, shared resource contention, routing and batching as effects absent from separate terms of the formula.
The case has one required local link, its source record basename. That file is declared in required_resources. External original evidence remains at its pinned owner URL. The final-head tests exercise the actual KnowledgeReader closure and the installed wheel independently of checkout imports.
The automated plan at the old head 63d4e3a selected zero comparisons and did not execute. It reported no verified ancestor pin for automatic selection and unavailable trusted pricing/configuration. The generated casebook projection and shared _provenance.json are now refreshed using the supported generator from source revision 9f89294, which contained the canonical inputs at that head. The explicit generator --check and all nine tests in tests/test_skill_index.py passed. An EPD free-text query regression also opens the returned read_args. The source pin and Skill Eval observations in this section describe the earlier review head.
Claim dispositions offered for independent review:
Acceptance and remaining work
Execution completion is separate from campaign acceptance. The published scope lacks a prospectively frozen acceptance gate, independently checked model-weight/runtime identity and a complete independently checked raw-artifact set. Two rounds provide limited repeatability evidence. A future qualification must freeze gates and sample protocol first, bind identities, retain verifiable artifacts and obtain independent review.
Maintainers own scope acceptance and issue closure. The earlier code/content approval covered
255887c. The rebased catalog and discovery changes are available for review. Earlier approvalKnowledge authorship declaration
The requester corrected and confirmed the reported authoring configuration as GPT-6.2 Sol with xhigh effort. The declaration below records that user-provided configuration. This is user-confirmed attribution and has not been independently verified from execution provenance. The earlier unknown-attribution exception request is withdrawn. Current policy lists gpt-6.2-sol at xhigh as admissible. The exact-head policy check and independent content review remain separate.
Earlier retained checks at
455f1c8Retained validation JSON binds head
455f1c8afd2b796c2a3a40d259b3ded246e98d06. It contains the 133-test JUnit, exact commands/exit statuses, index freshness output, installed verification script/output and wheel hash. Hostname/timestamp XML attributes are omitted. These are inspectable author-executed artifacts, with independent reproduction pending. The validation branch is separate from the PR diff.The source pin is
9f89294c9898a90125d4f848fcaad2a29faa722f. Generation and check commands werepython tools/build_skill_index.py --source-revision 9f89294c9898a90125d4f848fcaad2a29faa722f [--check]. The corrected card contains both EPD resources and the current SKILL digest. Installed package checks also verify the card's resources and source pin. Earlier incomplete full-suite/lint outcomes remain disclosed and are not superseded by these scoped results.The earlier unknown-attribution exception request is withdrawn following the requester's corrected GPT-6.2 Sol / xhigh report. That verification covered head 455f1c8. Maintainer scope acceptance, independent review and the refreshed exact-head declaration result remain pending. This metadata update does not claim independent model provenance or grant merge approval.
Earlier clean-checkout wheel evidence at
455f1c8The earlier install check used
inferrsi-0.1.dev5+g455f1c8af-py3-none-any.whl(SHA-2567678eefe7ea9b00cbac0864b56bed20c3e4b6e0f48864b7307c65720bd3152ad), built from the existing local checkout. The evidence below usesinferrsi-0.1.dev1+g455f1c8af-py3-none-any.whl(SHA-2568506c97a5d3bdb34cf9a80e7d07e3725ca55677c4efff47ddd0e771eb720e2e2), built from a fresh depth-one checkout with recorded build-tool versions. Their archived package metadata reports different versions. Use the retaineddev1wheel and build transcript below to recheck the link from the Git commit to the installed package.The build follow-up for the incremental review is available in the validation directory. The report records the actual fetched commit and Git tree, clean checkout status before the build, build tool versions, and the commands and exit codes. Two builds from fresh checkouts of
455f1c8afd2b796c2a3a40d259b3ded246e98d06yielded the same retained wheel SHA-2568506c97a5d3bdb34cf9a80e7d07e3725ca55677c4efff47ddd0e771eb720e2e2. An isolated install outside the checkout matched five case and projection resource hashes to the corresponding Git blobs, found the case through free-text search and opened the returned reference. The public wheel was downloaded again and matched the recorded hash.The Knowledge/index suite passed 133 tests again. The standalone EPD package passed its 14 tests, and an additional author check matched 36,414 integer solver calls against exhaustive enumeration for 2–64 GPUs and positive integer stage rates from 1 to 17. The seven published source hashes and both cold-round rankings were checked against the pinned external snapshot.
These are author-run checks. The ready/review CI run shows
action_requiredand has zero jobs. Independent review and maintainer scope acceptance remain pending.