Repository navigation
feat(use): admit training-inference paths under a declared bound - #91
Conversation
The use contract can now require that the learner's and the engine's log probabilities agree on the tokens a rollout took, per token or as a path log ratio. A new Invar.Paths module measures both sides exactly from FP32 words, refuses pairs whose token path, distribution rule or forward weights differ, and lets a declared correspondence relate genuinely different weights. History admission derives the same observations from each generation's samples and probability artifact, and use inspect/admit attach them through the history options, so a history report and a use decision share one comparison. Probability artifacts move to v6 and record the adapter identity each optimizer step actually used; current words are therefore bound to the forward weights that produced them. The three relations are exposed as training-path-words, training-path-token-log-gap and training-path-log-ratio, with the learner side named explicitly. Signed-off-by: bohaotu <bohaotu@pony.ai>
History.paths paired each workload name with a whole role list, so a one-step two-input run only passed because the first step's current words equal the proximal words, and reference observations were dropped. It now pairs each name with its own proximal and reference observation, maps the cohort by the request's behavior version rather than the generation index, and leaves per-step current observations out of the claim because their weights differ from the behavior policy by construction. rollouts uses the same behavior version. The path implementation check no longer compares the adapter digest, which is policy content rather than implementation identity, and honors declared transfers. Reference observations now carry the request's reference adapter on both sides. The probability artifact's per-step policy is required to equal the applied step's before digest in both runtime and history admission, the path rule records truncation, negative path budgets are rejected, unknown path inputs are refused, and use accepts --history-log. The path contract fixture now runs two optimizer steps with learner-scored references, and the new tests pin reference identities, per-step policy binding, adapter-independent path implementation, and unknown inputs. Signed-off-by: bohaotu <bohaotu@pony.ai>
|
FeathBow
left a comment
There was a problem hiding this comment.
Thanks for taking this on. The exact rational comparison, the per-token relation with its cancellation test, and deriving learner_engine from the same observations all look right. A few things need to change before this can merge.
Needs changes
- Cohort lookup under staleness.
History.pathsandHistory.rolloutsnow select the workload cycle by the request's behavior version (src/Invar/History.hs:259,:267), while the runtime selects it by update index (src/Invar/Runtime.hs:443). With staleness 1, update 1 has version 0, so its samples get cycle 0's names and its observations attach to update 0's inputs. The previous positional lookup inrolloutswas correct, soinvar scoreover a history regresses as well. - The learner's implementation is not bound by the contract.
samePathImplementation(src/Invar/Use/Admission.hs:208) compares model, revision, tokenizer and base. On the learner side, model and revision are taken from the engine's report and the tokenizer is shared, so in effect only the base is checked. The assembly is never compared, andImplementationIdis not used outsideInvar.Paths, so a contract admitted for one learner implementation also admits another without a transfer. Ignoring the adapter makes sense; the assembly should be compared. - Path observations are not bound to the declared inputs. Generated pairs pass the domain and context checks, but path observations attach by cohort and sample name only (
src/Invar/Use.hs:88). Their prompt, seed, temperature and token limit should be checked against the input they attach to. - Incomparable pairs.
Unknowntakes precedence over a refutation in the evidence graph (src/Invar/Spec/Evidence.hs:163), so one pair with different forward weights makes the whole requirement undetermined and can hide a violation on a comparable pair. A run with staleness above zero and more than one update will almost always end undetermined. #79 asks for such pairs to be reported as policy differences and left out of the decision, with the requirement undetermined only when no comparable pair remains. Current observations are also excluded on the assumption that their weights differ after the first step (src/Invar/History.hs:258), while #79 decides comparability from the forward-weight identities alone. The design note says they are reported, butPaths.currentsis not used outsideInvar.Paths.
Smaller points
- Every learner observation records step 0 (
src/Invar/Paths.hs:267), whichever optimizer step it comes from. - The per-step
policyfield repeats what the core already has: eachcurrentreport carries astatethe core checks, and history has every applied step'sbefore. Deriving the identity from those would avoid the worker change and the new probability format, which makes retained runs in the old format unreadable byinvar inspect history. Correspondenceand theforward-weight-correspondencepremise have no producer, andinvar use preparedoes not predict that premise as the description says. They may fit better with #84.Rule.truncationtakes the response-lengthtruncatedflag rather than distribution truncation, and both sides' rules come from the same sample record, soRuleDiffercannot occur for history pairs.learner_engine.path_log_ratiosums over every sample of the generation rather than one path, andwithPathsrefuses a history with samples outside the contract's domain (src/Invar/Use.hs:92). Both are worth stating in the guide.
Tests #79 still needs
- the first update of a run with staleness above zero, whose behavior and proximal words are compared;
- an optimizer step that changes the learner state but not the forward weights, whose current words are compared;
- policy-difference pairs reported separately while comparable pairs are still decided.
Happy to discuss any of these.
History.paths and rollouts select the workload cycle by the update index again; the behavior version remains provenance only, so a stale update's observations stay with the cycle they were rolled out on. Path observations record prompt, seed, token limit and temperature; use admission checks them against the declared input. The contract gains an optional learner implementation. Path admission compares the learner observation with it and the engine observation with the declared side, including the assembly; a contract with a training-path relation without a learner implementation is invalid. use prepare reads the learner policy from artifacts.learner_policy. Comparability follows forward weights alone. Current observations are included; pairs with different weights are reported as policy differences and left out of the decision, while comparable pairs still decide the requirement. The learner observation records its real optimizer step. Probability artifacts return to v5. Current identities come from the core's current reports, which the core already checks, instead of a worker-written per-step policy; retained v5 runs remain readable. The path rule keeps the declared distribution rule and no longer treats response truncation as rule truncation. The correspondence type and premise are left to FeathBow#84. Tests add a stale run whose path observations span two cohorts, a policy difference beside comparable evidence, a current step with unchanged forward weights, learner assembly mismatch, input mismatch and v5 observations. Signed-off-by: bohaotu <bohaotu@pony.ai>
|
FeathBow
left a comment
There was a problem hiding this comment.
Thanks for the thorough update. I checked the new commit against each point of the previous review, and all nine are addressed: the cohort uses the update index again, the learner implementation is declared and compared including the assembly, path observations are bound to their declared inputs, policy differences are reported and left out of the decision, current observations carry their own step and the state the core checks, the probability format stays v5, and the correspondence is gone.
A few small things before this leaves draft:
- The PR description still describes the earlier revision: the correspondence and its premise in
use prepare, probability artifacts v6, the test list, and the note that differing forward weights stay unknown. Please bring it in line with the code. - In
docs/design/loop.md:107, "the probability artifact remains v5" reads as a note about this change rather than a description of the system, so it can go. The same paragraph says the learner-state digest is provenance, while thestateof acurrentreport is used as the adapter identity; saying that this field is the adapter content digest would avoid the confusion. - Optional test tightening:
test_a_stale_training_path_uses_the_update_cohortcould assert that cohort 0's pairs are compared and cohort 1's proximal pairs are reported as policy differences. AfromUpdatecase where a step's state equals the starting adapter and the bases agree would show such a step being compared through the real derivation;currentComparablebuilds the identities by hand.
With the description updated, I'm happy to approve.
The stale path contract now asserts that cohort 0 has comparable evidence while cohort 1's proximal pairs are reported as policy differences. A fromUpdate case with equal bases and a current step whose state equals the starting adapter checks that a state-only step is compared through the real derivation. The loop guide names a current report's state as the adapter content digest instead of describing the probability format change. Signed-off-by: bohaotu <bohaotu@pony.ai>
|
|
Thanks for the update. The earlier fixes look good, but I found two more issues:
Both cases reproduce on the current branch; the existing Paths and UseAdmission tests pass. |
Path claims now check that every declared input has at least one attached observation before deciding anything. An input without evidence leaves the requirement undetermined, even when another input would be refuted, and policy differences are excluded only after coverage is complete. Each path observation now records the admitted call, attempt and instance plus the SHA-256 of the source log that produced its words. The learner side names the admitted update and the engine side names the admitted rollout, so changing either execution changes the observation and its scope. Test coverage: a partial history with only a1 attached stays unknown for both a satisfied and a violated budget, and a changed learner binding changes the observation and scope while the engine execution remains stable. Signed-off-by: bohaotu <bohaotu@pony.ai>
Signed-off-by: bohaotu <bohaotu@pony.ai>
Path implementation checks now cover every attached observation, including pairs excluded from the decision as policy differences. A foreign learner or engine implementation therefore leaves the requirement undetermined instead of being admitted through another comparable pair. Engine executions are keyed by the probability sample name and checked against the admitted inference's behavior words, so a reordered or mismatched cohort cannot silently attach the wrong call or source log. Path judgments also reject observations whose declared inputs differ. Test coverage: a policy-difference observation with a foreign implementation is undetermined, a path contract without a learner implementation is invalid, duplicate path keys and missing engine executions are refused, and the HF history E2E asserts both sides carry call, attempt, instance and log digests. Signed-off-by: bohaotu <bohaotu@pony.ai>
FeathBow
left a comment
There was a problem hiding this comment.
Both issues are fixed. I re-ran the original repro cases and the focused tests; all passed. Looks good to me.
Why
A use contract can require a candidate implementation to reproduce the reference, but the training-inference gap — the learner and the engine scoring the same rollout tokens — was only a diagnostic summary in a history report. This PR turns it into a declared, replayable claim: a contract can require that the two sides agree on those exact paths, either bit for bit, per token, or as a whole-path log ratio.
What
Invar.Pathsmodule. A path observation carries the token path and prompt split, the distribution rule, the declared input (prompt, seed, token limit and temperature), the forward-weight identity (frozen base + trainable adapter), the implementation, the FP32 words and their provenance. Comparisons are exact onRationalwords.training-path-words,training-path-token-log-gapandtraining-path-log-ratio, each naming thelearnerside. The contract'slearnerfield declares the learner implementation;use preparereads it fromartifacts.learner_policy. The learner and engine implementations are compared by model, revision, tokenizer, base and assembly; the adapter is the forward identity.History.pathsandrolloutsuse the update index), andusechecks each observation's prompt, seed, token limit and temperature against the contract input. Samples outside the contract's domain are refused.History.pathsexposes the proximal, current and reference pairs.use inspectanduse admitattach them through the--history-*options (including--history-run/--history-log) and report policy differences infinding.path_policy_differences.currentreport'sstateis the adapter content digest the core checks; history derives current identities from those reports/applied steps. The path rule keeps the declared distribution rule. Retained v5 runs remain readable.Correspondenceand its premise are deferred to policy: publish a generation across artifact formats with a declared conversion #84.path_log_ratio, domain refusal and rule scope.Validation
cabal test: PASS.test/types.sh ghc,fourmolu --mode check,hlint src test app,cabal check: clean.use admit --history-runagainst the HF fixture returnsadmitted_under_declared_reliance.Limits
A pair with different forward weights stays a reported policy difference; relating genuinely different weights is #84. A contract with a training-path relation must declare its learner implementation. The path check binds each observation to the declared input and to the learner/engine implementations, not to a second execution of the model.