Skip to content

feat: join re_gent agent turns to runtime cost via git SHA (/api/attribution) #1

Description

@Yatsuiii

The chain

Neither tool closes this alone:

agent turn ──► git commit ──► deploy ──► runtime cost
└── re_gent ──┘                └──── llmtrace ────────┘

re_gent owns the authoring layer: which agent turn wrote which code, under which commit. llmtrace owns the runtime layer: which deploy moved the bill. Different vantage points, which is what makes joining them useful rather than redundant. Join them and a runtime cost spike walks back to the exact turn the agent authored the change that caused it.

The join key: git commit SHA

The one primitive missing on the re_gent side: steps aren't linked to a git commit SHA today, it's all the content tree. If a step stamps the git HEAD it produces, that SHA becomes the join key into llmtrace's deploy attribution.

llmtrace side: already shipped (testable now)

GET /api/attribution?sha=<commit-sha>      # short or full SHA both match

Given the commit a change shipped under, returns the deploy of that commit and any cost or latency anomaly the correlator attributes to it (time-window + lineage scoring: model change, prompt change, error spike), with confidence and evidence:

{
  "sha": "c4e2117",
  "matched": true,
  "deploy": {
    "id": "gha-2-summary-sonnet",
    "pr_number": 2,
    "title": "switch summary endpoint to claude-sonnet",
    "commit_sha": "c4e2117a8b1f",
    "completed_at": "2026-05-26T14:12:00Z"
  },
  "attribution": {
    "metric": "daily_cost", "date": "2026-05-26",
    "baseline": 4.68, "actual": 12.92, "delta": 8.24, "sigma": 28.0,
    "confidence": 0.55,
    "evidence": [
      { "kind": "model_change",
        "description": "dominant model shifted from claude-haiku to claude-sonnet (87% of post-deploy calls)" }
    ]
  }
}

A commit that deployed but didn't move the bill returns matched: true, attribution: null, and a note. An unknown commit returns matched: false. Innocent same-day deploys score low (time proximity only), so the real cause is isolated by confidence.

Live to test against:

https://llmtrace-681081536857.asia-south1.run.app/api/attribution?sha=c4e2117

re_gent side: the part to implement

  1. Stamp the git HEAD (commit SHA) a step produces onto that step.
  2. To explain a runtime cost spike: llmtrace surfaces the deploy SHA it blames, and re_gent maps that SHA back to the authoring step (rgt show). Or proactively: when a step ships a commit, query /api/attribution?sha= to see if it moved the bill.

Defaults (decided, flag if re_gent needs otherwise)

  • Deploy ingestion: llmtrace pulls deploys (with their SHA) from GitHub Actions via sync-deploys; a push endpoint can carry the SHA for non-GitHub flows.
  • Auth: none for now, single-tenant, same as the existing endpoints. Add a token if re_gent calls it across a network boundary.

The endpoint is real and live. The only piece on your side is stamping the SHA onto steps. Shape above is a starting point, poke holes whenever you get a sec.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions