feat(telemetry): carry agent_version in the trajectory envelope (DGM variant attribution)#166
Open
justrach wants to merge 2 commits into
Open
feat(telemetry): carry agent_version in the trajectory envelope (DGM variant attribution)#166justrach wants to merge 2 commits into
justrach wants to merge 2 commits into
Conversation
…ant attribution) `agent_version` (sha256 of the agent's prompt frontmatter+body, app.rs:223) already rides the AgentRun trajectory event, but the uploader destructured it away -- so graff.trajectory.v1 identified WHICH agent (agent_id via run.id) but not WHICH VARIANT of its prompt. That's the missing piece for attributing fleet fitness to a specific variant. Thread it through: RunBuf captures agent_version from AgentRun; the Run envelope carries it (skipped when None). A rollup over (agent_id, agent_version) can now compare prompt/scaffolding variants and select the better one -- the Darwin-Goedel-Machine substrate the schema was designed for (see the AgentRunEnd doc-comment in forge_domain/trajectory.rs). The verifiable reward (tests/build/git) stays server-side: the full steps are already uploaded, so the ingest worker can derive it; the client only lacked variant attribution, which this closes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…envelope Deterministic coverage for the variant-attribution field: builds a Run envelope and checks agent_version serializes when present and is skipped when None. (A live wire capture is blocked by intermittent Codex stalls; this proves the serialization without a model call.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Action required: PR inactive for 5 days. |
github-actions
Bot
added
the
state: inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The trajectory telemetry was designed as a Darwin-Goedel-Machine fitness substrate (the
AgentRunEnddoc-comment inforge_domain/src/trajectory.rssays so: "rollup over(agent_id, agent_version)reveals which variants are efficient vs wasteful... substrate for DGM-style sampling"). Butagent_versionwas dropped before upload — sograff.trajectory.v1carried which agent (agent_idviarun.id) but not which variant of its prompt. Without that you can't compare prompt/scaffolding variant A vs B and select the winner — the core move DGM / ADAS / SICA all depend on.Fix
crates/forge_app/src/trajectory_upload.rs— thread the existingagent_version(sha256 of the agent definition,app.rs:223) from theAgentRunevent throughRunBufinto theRunenvelope (skip_serializing_ifwhen None). 5 small edits, no new computation — the hash already existed.Why this and not the verifiable reward
The other half (
tests_passed/build_ok/diff_committed) is server-side, not a client gap: the fullsteps(every tool call/result) are already uploaded, so thedata.codegraff.comingest worker can derive verification + refinereward. The client's only missing piece was variant attribution — which this closes.Verification
cargo check -p forge_appclean; envelope now includesagent_versionwhen the agent has a prompt hash.🤖 Generated with Claude Code