Skip to content

[G1] Capture per-message raw facts burn needs: requestId, stop_reason, agent version, sidechain/meta flags, Codex turn ids #164

Closed

Description

@willwashburn

Part of #160 (group 1 — independent).

Problem

session_events (crates/ai-hist-core/src/lib.rs:138-155) stores message_id, parent_id, ts_ms, role, kind, text, model, token_json, event_uid. burn's analytics need per-message facts that relayhistory currently reads past. grep -rn "requestId\|request_id\|stop_reason" crates/ai-hist/src/lib.rs crates/ai-hist-core/src/lib.rs returns nothing.

What burn reads today and where it uses it (all in AgentWorkforce/burn, crates/relayburn-sdk/src/):

Raw field Harness burn reader burn consumer
requestId / request_id (envelope) Claude reader/claude.rs:1310-1312, claude/incremental.rs:821-830 reader/inference.rs:236 groups turns into API requests by requestId (RequestIdLookup); falls back to message_id for harnesses without one
message.stop_reason Claude claude.rs:370-470 reader/claude/span_tree.rs; in-progress-turn detection (claude/incremental.rs:646-664 backs the cursor up to the earliest assistant message without a stop_reason)
step-finish.reason OpenCode reader/opencode.rs:544-553 → StopReason::from_wire same
version / sourceVersion Claude claude/relationships.rs (source_version) relationship records, version-dependent quirks
isSidechain, isMeta Claude claude.rs sidechain attribution
turn_id from turn_context; task_started/task_complete boundaries Codex codex/incremental.rs:230-336 Codex turn boundaries (burn #504 is about exactly this)
cache_creation.ephemeral_5m_input_tokens / ephemeral_1h_input_tokens Claude claude.rs:587-643 Usage.cache_create_5m/1h for pricing

relayhistory stores message.usage verbatim in token_json (lib.rs:3070-3072), so the nested cache_creation object is already preserved — add a test proving it survives round-trip.

Scope

  1. Schema (crates/ai-hist-core/src/lib.rs SCHEMA + init_db_locked migration): add nullable columns to session_events: request_id TEXT, stop_reason TEXT, agent_version TEXT, is_sidechain INTEGER, is_meta INTEGER, turn_id TEXT. Add a migration marker (follow the session_relationships_v2 pattern) and bump HYDRATION_PARSER_VERSION (crates/ai-hist/src/hydrate.rs:15) so existing databases re-hydrate and backfill.
  2. Claude parser (crates/ai-hist/src/lib.rs:2999-3302): read requestId/request_id, message.stop_reason, version/sourceVersion, isSidechain, isMeta. Store stop_reason as the verbatim wire string (no enum mapping here — burn maps; relayhistory preserves).
  3. Codex parser (lib.rs:2101-2508): read turn_context.payload.turn_id and stamp it on every event until the next turn_context. (task_started/task_complete markers belong to the "dropped record types" group-1 issue; this one owns turn_id on events.)
  4. OpenCode: step-finish.reason → stop_reason on the assistant message events (only meaningful once the OpenCode parity issue lands; land the column and the mapping hook now, with a test that is #[ignore] until then).
  5. Expose the new columns on SessionEvent (crates/ai-hist-core/src/lib.rs:1512), NativeSessionEvent (crates/ai-hist-napi/src/lib.rs:249), sdk-ts/src/contracts.ts:185 and normalization.ts:177; bump SESSION_EVIDENCE_CONTRACT_VERSION and NATIVE_CONTRACT_VERSION per docs/architecture.md.
  6. docs/session-catalog.md capability matrix rows for the new fields per source.

Acceptance

  • Fixture-corpus snapshots show request_id and stop_reason populated for every Claude assistant message in multi-block-turn, and stop_reason absent for the in-progress message in incomplete-then-complete.
  • Round-trip test: token_json for a Claude message with cache_creation: {ephemeral_5m_input_tokens, ephemeral_1h_input_tokens} returns both nested values.
  • Codex multi-turn snapshot shows turn_id changing at each turn_context.
  • Existing databases upgrade in place; ai-hist sessions hydrate reports status: "updated" once after upgrade and unchanged after.

Out of scope

Mapping stop reasons to a normalized enum (burn does this in types.rs:287-357); grouping events into inferences.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions