Repository navigation
Make the v3 return supplement usable without our code: exporter defaults, reference predict, sealed roles, test-opening policy; CI timeout 90 - #45
Merged
Conversation
Encoder reference script (docs/data/return-v3/reference/export_hf_return.py): - export: --max-tokens defaults to 512 (XLM-R base limit); over the encoder's own limit it refuses and names the limit; a missing pooler points to the recipe; prints the installed runtime and whether it matches the product's pinned versions. - predict (new): loads a return exactly as RoleCraft's loader does (seed, fast tokenizer, safetensors, CPU float32, after_weights re-check) and writes v3 rows with the same abstentions (input_problem codes, no candidates, token limit, pack spans), mean pooling, pair_aggregate and evidence threshold; --labels lists rows our format check rejects (evidence_missing); never writes into a sealed return. - seal: writes artifact_roles, so validate-handoff --checkpoint-manifest accepts encoder returns (it required the roles for every non-linear return). Linear reference script: export_linear_return writes the training-notes template; write_predictions writes test rows in the dev format, outside the return directory. Specs: encoder-return (512 and the 391-474 token measurement, abstention table 5.1, runtime 11.1, pooler and XLMRobertaForSequenceClassification recipe, artifact_roles, predict); dataset-split-sealed section 7 (pending -> accept, interface draft); test-opening (one opening per candidate per freeze directory, one campaign for all, fusion declaration manifest); predictions-v3 (v2/v3 rows, v2 validator limits); README section 10 (three tightenings relative to v2); W07/W08 manual lines. SHA256SUMS regenerated (13 entries). Tests: PLUG-01 (unit), PLUG-06/07 (integration), PLUG-02..05 (optional encoder); ENC-01/03/05 pass the tiny BERT's 256 limit explicitly now that the default is 512.
…G-01, PLUG-06, PLUG-07 x2; 4 IDs) Appended only: backend 2628 -> 2632 IDs, frontend unchanged, source_commit unchanged; compared with 5a529ea's baseline: 0 missing, 0 changed outcomes. Sources: tests/unit/test_return_v3_package.py (1) and tests/integration/test_return_v3_plugin.py (3).
….05, 11 items) docs/HANDOVER.md: the optional encoder command now lists five files, 34 items with the XLM-R smoke (17 + 2 + 1 + 11 + 3); docs/data/expansion-v3-w08.md: the same file in the CPU encoder environment command and one line on what it checks.
…ictions checks) - test-opening-v1.md: the code guarantees one opening per freeze directory; the campaign is not bound to a freeze, so another freeze directory for the same candidate could open again (measured: exit 0). Written as a process rule: one freeze directory per candidate, created by us after the user confirms and committed to git with the campaign and the opening record. The opening paragraph now says "the second opening of the same freeze directory is refused". W08 manual aligned (HANDOVER has no such statement). - export_sklearn_linear.write_predictions: before writing any row, every array (coef, intercept, evidence_coef, evidence_intercept, idf, evidence_idf) and both vectorizers (main, and the evidence head: shared or its own) must equal the export (linear_objects_not_exported: <field>); prediction_output_inside_return and prediction_output_exists name their refusals. - export_linear_return: the training notes path is checked before anything is written (training_notes_path_invalid): relative, inside the return, not a member or under linear/. - README: max_tokens defaults to 512 and may not exceed the tokenizer's limit; dataset-split-sealed section 6 and encoder-return 5.1: validate-handoff records lawful abstention rows as pending (exit 2), the existing rule, confirmed by us at the joint check. - Tests: PLUG-07 tamper cases (8, one per array or vectorizer, match on the field) and bad notes paths (5); the existing raises now match its code. SHA256SUMS regenerated (13).
…nd notes-path cases; 13 IDs) Appended only: backend 2632 -> 2645, frontend unchanged, source_commit unchanged. Source: tests/integration/test_return_v3_plugin.py (8 tamper cases, 5 notes-path cases).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the gaps a from-scratch read of the v3 return supplement (as the data/model owner's agent) still hit after #44, and raises the CI timeout so the regression gate on
maincan finish again.What changes
max_tokensis 512 (the spec records that one learner sentence measured 391–474 tokens, so the earlier 256 sample would abstain on everything);encoder-return.md§5.1 lists every abstention condition the product applies (input_problem, over-length, missing evidence) andreference/export_hf_return.py predictreproduces the product loader — pooling, aggregation, thresholds, abstentions, CPU float32, recomputedafter_weights— so the owner can compare their dev predictions before returning them (0 mismatches against the product loader on realxlm-roberta-baseweights with untrained heads). The spec states the pinned inference versions, the pooler requirement and how to save aXLMRobertaForSequenceClassificationmodel into this layout.artifact_rolesfor encoder returns, sovalidate-handoffno longer sends them back;dataset-split-sealed-v1.md§7 and the W07 handbook describe the pending→accept path for real data (an authoritative-source reader we provide once the data plan arrives; interface draft--source-authority, not implemented yet) and note that legitimate abstention rows are recorded as pending (exit 2) under the existing convention.training-notes.mdtemplate and offerswrite_predictionsas the test-prediction entry point, which verifies all six arrays and both vectorizers against the fitted objects before writing anything (linear_objects_not_exported: <field>), and checks the notes path before creating files.README.mdis untouched.timeout-minutes40 → 90 (the suite grew past 40 minutes this wave; the last greenmainrun took 36). Concurrency settings unchanged.Verification
Independent review
080-review-161.md: first round sent back for two small items (contradictory test-opening wording;write_predictionsonly verified part of the arrays), fixed inbeabc89with 14 tamper/error-path tests that were red before the fix. Targeted suites 441 passed (17 new IDs, 0 missing / 0 regressed against5a529ea); optional encoder suite 31 passed; XLM-R smoke 3 passed;check_fastand lint clean; 13 new default-environment IDs appended to the baseline. PackageSHA256SUMS(13 entries, own sha256073e6759…cf14) self-checks in place and after copying file by file. The full suite runs in CI onmain. Receipt: 完成回执/claude-coordination/161-v3回传包一插即用.md.