Repository navigation
Per-sentence R2.support advice with conservative aggregation, model cache and failure isolation - #40
Merged
Merged
Conversation
…ation Split each work subject by the r2-support-units-v1 rules, journal one advice request per sentence and aggregate: any contradicted sentence decides the work, all sentences supported gives SUPPORTED, everything else INSUFFICIENT. Synthetic registrations now show their labels and evidence as a mechanism demonstration (143 section 5.1 item 2).
Sentence units and synthetic labels change the advice semantics by design; request, job, whole-work input hash, registration and advisory flags stay byte-identical to the pre-move recording.
…t and golden vectors Register rolecraft-linear-return-v1 returns (fixture v2 outer manifest plus linear/model.json and linear/arrays.npz) as advisory-linear-registration-v1, dispatched from register_bundle on artifact_kind. The loader rebuilds a built-in-analyzer TfidfVectorizer from declared parameters and predicts with the product feature text rolecraft-linear-text-v1; the standard-library reference text and golden vectors cover the twelve fixture v2 inputs and bilingual synthetic samples. The registry CLI gains describe.
Each sentence request measures its input with the loaded model first; one over the registered token budget is journaled as abstained with unit_exceeds_encoder_limit and its measured size, and is never predicted.
…rations Encoder registrations now use advisory-encoder-registration-v2: they record the installed torch, transformers, tokenizers and safetensors versions from package metadata (no torch import), refuse to register without them, refuse to load or describe under different versions, and publish kind, dataset and weights_stage. v1 registrations serve no new advice; feedback recorded before stays readable.
…hanism labels Normal HTTP and worker runs with constructed marker weights: short and over-budget sentences are counted, insufficient sentences never make a supported work, one contradicted sentence decides it, a restarted job reads journaled sentences back, Chinese and English versions of one fact conclude alike, sentence inputs carry no probe, gold or rule derivations, and synthetic labels stay a marked mechanism that never changes scoring.
Each call checks the registration file hash and the size, mtime, ctime and inode of every registered file; any change means full verification again, so file drift still reaches new feedback as model_files_changed.
Registrations now record source hashes of models/v3/** and contracts/v2/{core,data}.py
only, with numpy, scikit-learn and pydantic versions still pinned, so unrelated product
changes no longer invalidate every registered model.
checkpoint.json v3 states variant (pack, pair, pair_aggregate), mean pooling, evidence_threshold and max_tokens, and model_revision hashes them with the weights. Registration checks head array names and shapes, the encoder config and the tokenizer capacity, and returns encoder_architecture_unsupported for structures outside the specification. The pretrained_untrained_heads stage registers only as synthetic_fixture with an honest outer manifest. Legacy fixture v2 era checkpoints keep registering with the fixed 0.5 threshold. pair_aggregate's deterministic aggregation follows the specification examples.
advisory-fusion-registration-v1 references a linear or encoder return on each side by registration id and sha, requires one dataset identity, a dev-only selection on the published alpha grid and no fusion-gain claim at an endpoint alpha, and keeps the fusion no broader in scope than its components. Loading re-verifies both components, so any component drift fails the fusion. The registry CLI gains register-fusion.
HuggingFaceCandidate gains pair_aggregate, the declared evidence threshold, the pretrained_untrained_heads stage (mark_untrained_heads) and writes the v3 layout with canonical JSON and byte-reproducible heads.npz; fixture v2 era checkpoints still load with the fixed 0.5 threshold. The standalone reference export script writes the same layout from a producer's encoder directory and head state, and seals the outer manifest. Optional checks prove the script and save_checkpoint register the same model for all three variants.
encoder-return.md states the layout, checkpoint.json fields, head arrays, the encoder input text, the three variants and the pair_aggregate rule, after_weights, model_revision, weights stages, registration and the runtime pin, the reference script and every refusal code. The reference script also gives the exact encoder input strings and refuses mismatched deliveries.
…oring Covers both linear_return_too_large paths (archive expansion and file size) and the refusal of linear and fusion registrations rewritten to affects_score or quality_validated true; the endpoint-alpha check now also asserts exit codes.
…behaviour
The registration source identity now covers src/career_lab/models/** (the legacy MLP
imports career_lab.models.linear) plus contracts/v2/{core,data}.py; RT-01 checks the
older model module too. The linear and fusion specifications mark the 1e-6 reload
tolerance and verify-return --fusion-selection as arriving with the v3 return verifier
and list every refusal code; the pre-v2 encoder manifest fallback also refuses repeated
label tiers.
validate_return_identity checks only the fixture v2 outer-manifest fields that every return kind shares, so linear returns that reuse it are accepted. The encoder path checks artifact_kind and artifact_protocol itself (is_encoder_return), encoder_registry.handles_return names the manifests it claims, and the encoder protocol does not claim linear or fusion registrations.
…lone With encoder registrations publishing kind and dataset (146, protocol v2), the fusion no longer reads a component's return manifest copy; a component without a published data identity is refused. FUS-01 now checks the encoder identity as published.
joint-first-dev-v1: only points with a positive joint correctness qualify; ties go to coverage, macro-F1, the cheaper endpoint and grid order; when no point qualifies there is no confirmable selection; fusion gain means a non-endpoint winner strictly above every endpoint. The dev input hashes must cover the whole dev partition.
…odel The light check now covers exactly the files load_registration opened under the registry, including a fusion's component registrations and weights, instead of the registration directory alone. Component drift after the cache is warm reaches new feedback as model_files_changed.
CACHE-03 changes a component's weights, a component registration and the fusion's own selection after the cache is warm. COMP-01 now uses a 145 fusion of two linear returns and checks both components and alpha. Contradicted sentences keep their evidence in the work's union; a partly failed work stays insufficient with true counts; an interrupted sentence stays unconfirmed.
…l feedback Scenario materials are external references the store accepts only once declared; a cited material is traced through its citation, as for rule items. Any unexpected advice failure becomes failed advice so feedback and its rule results are still written. A retired encoder registration is reported as an invalid reference.
validate_return_identity, which linear returns reuse, again requires a completed, trained delivery and refuses any other weights_stage. Encoder returns go through validate_encoder_return_identity, which admits the honest pretrained-untrained-heads declaration only after the manifest is known to be an encoder return and only for a synthetic_fixture registration. SEAM-03 covers linear manifests carrying it.
…e half A pair counts as support only when its support is at or over the threshold and above its unrelated probability, and pair_aggregate thresholds below 0.5 are refused at registration, in the candidate and in the reference script, so an unrelated-leaning pair never becomes support. The specification examples add the tie and most-likely-unrelated cases and two refused low thresholds.
checkpoint.json tasks other than relation now fail as model_return_task_or_labels_invalid, every file in the return directory must be listed in checkpoint_files with an allowed suffix, pair_label_order is refused on pack and pair, and heads must be float32. The specification lists every registration and load refusal code and says registration does not prove loading.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consumer line of the model-reception wave (task book 143, line 148). The relation models the data/model owner returns (linear, encoder, fusion) judge one claim against candidate evidence. Until now R2.support fed the whole work in as a single input, so on a 512-token encoder any long work simply abstained. This line makes R2.support advice per sentence, aggregated conservatively (decision 2026-10-12: any contradicted sentence → contradicted; all supported → supported; otherwise insufficient). Synthetic registrations may now carry labels and evidence (same-day decision), always marked
synthetic_mechanism_onlyand never scored. The change is visible in HTTP feedback. The current v4 UI rejects labelled synthetic advice and shows "模型建议失败 / Model advice failed" until 149 lands, so 149 must merge in the same batch or right after.What changes
models/v3/units.py, policyr2-support-units-v1(rule-based, not claim extraction):-,*,•or1.is one unit.。！？；!?;, plus an English period followed by whitespace or the end, not preceded by a digit, and not part of e.g./i.e./vs./etc. Decimals are not split.skipped_short. At most 24 units per package; the rest count asover_budget.unit_hash.consumer.py,projection.py,advisory.py):projection.sentence_input. The claim is the sentence; candidates, purpose and time context are those of the whole work.measure()runs first. Over the limit →abstained/unit_exceeds_encoder_limit, withinput_size{tokens, limit}in the journal; the input is not truncated and the model is not called.feedback-advice-unit-<digest(work request, index, unit_hash)>. Recovery reads recorded sentences back and infers only the missing ones. After aload_error, later sentences are only read back.units.aggregate:unavailable/all_units_abstained, orfailedwith the first failure code, orunavailable/claim_not_available.model_advice[].units[]: index, subject span/quote, unit_hash, status, label, evidence_ids, citations, reason_code.aggregation{policy, counts{supported, contradicted, insufficient, abstained, failed, skipped_short, over_budget}}.registrationcomes fromdescribe_registration(...).identity()(kind, dataset, components, fusion, weights_stage).affects_score=false; rule results, score ranges and coverage are unchanged.load_registration, an audit hook records every file opened under the registry, including the registrations, manifests and weights of fusion components.model_files_changedin new feedback.registered_inference_runtime_changed).main:object_not_found. Nowadvice_dependencieslists only session objects, and scenario material enters the read boundary only when cited.relation_advicenever raises; an unexpected error becomesfailed/model_prediction_invalidand rule feedback is still written.encoder_registration_retired→model_reference_invalid.== len(units)).models/v3/{units (new), consumer, projection, advisory}.py; teststest_support_units.py(new),test_registered_feedback_flow.py(MODEL-02–06),test_model_assembly_split.py(SPLIT-01).Review and verification
080-review-148.md(153). First round, onf4115f4, sent back 1 P1 and 3 P2:object_not_foundwhole-job failure.ebc6238: everything passed except one P2.261118e. The reviewer asked only for a rerun ofcheck_encoder_v3.pywith a coordinator check, not another full review.640999fall went red.ebc6238:eb494c0(3287),f4115f4(3340), 14523fb9df(3435) andmain95d4883(3329).261118e:check_encoder_v3.py: 17 collected, all passed (--extra encoder, offline).ebc6238, synthetic linear registration, three-paragraph work, zh and en):model_files_changed. Linear × encoder fusion identity: left=linear, right=huggingface_encoder.f4115f4was green (history only): backend 3309 / 31; frontend 404 / 7; 72 state checks; build; 0 problems; browser 20 steps zh+en; MCP bilingual pass. Full gate ran once on the integration tree11e53dd7…(146b765c88→ 14523fb9df→ 1471e2b85c→ 148261118e→ 1492e56be1, integration branch HEAD855b366; 157 receipt).Not claimed / registered
MAX_UNITS(perhaps 12, or batch sentences).RegisteredFeedback.tracedcomputes projection and dependencies beforerelation_advice. An error there, or empty dependencies, still fails the job; current flows do not reach this.Receipt: 完成回执/claude-coordination/148-产品消费端按句切分与聚合.md.