Skip to content

refactor(runtime): unify request-shape authority and expose semantic prefix continuity #4299

Description

@Astro-Han

Problem

Maka currently has two generations of request-shape diagnostics:

  1. computeRequestShapeDiagnostic predicts a turn shape before dispatch from live system/tool/provider state and a lossy whole-history hash, then stores a session-scoped priorRequestShape baseline.
  2. PreparedProviderRequestCapture records the secret-free standardized model-call parameters and ordered request segments near the provider seam.

They overlap on the same question but can disagree. The predictive path cannot distinguish normal history append from mutation of an old block, hashes equal-length text as the same history shape, resets on process restart, and races under overlapping turns. The capture path is closer to the fact but is not yet sufficient to claim cache-prefix continuity: a capture can exist without a dispatch, its segments/hash are not all derived from one representation, redacted opaque content can compare equal, and it observes the AI SDK model-call seam rather than the final provider wire body.

PR #4278 removed one concrete cache-prefix breaker, the generic turn-tail injection. We need a general architecture that detects the class of failure instead of another feature-specific test.

This is a focused successor to the request-capture part of #615. PR #1277 supplied per-request capture and attempt telemetry; PR #2346 already projects captured segments into Inspector context composition.

First-principles authority model

Durable RuntimeEvents / checkpoints
              ↓
        context composition
              ↓
   PreparedRequestObservation
  (Maka semantic model-call input)
              ↓
      canonical ModelCallAttempt
  (dispatch, identity, lineage, usage)
              ↓
      rebuildable projections
  composition / prefix continuity / cache usage
              ↓
            Inspector

Each authority owns one fact:

  • RuntimeEvents/checkpoints say what durable history should be.
  • A prepared-request observation says what Maka handed to the model-call seam. It is not named or presented as the literal provider wire body.
  • The canonical model-call attempt says whether a physical call was dispatched and which causal lane/predecessor it belongs to.
  • Provider-reported usage is the authority for actual cache reads/writes.
  • Inspector only renders Runtime/Host projections; it never re-hashes requests or chooses a predecessor.

Semantic prefix continuity is evidence that Maka preserved its old model-facing blocks. It must never be presented as a provider cache-hit claim.

Invariants

  • A prepared capture without a canonical dispatched attempt never enters the continuity sequence.
  • Request serialization, digest, and segment fingerprints come from one secret-free representation.
  • Opaque/redacted content cannot produce preserved; the result is unknown across that boundary.
  • Predecessors are causal and durable, never a session-wide in-memory last, append order, or wall-clock guess.
  • Continuity is only compared inside an explicit non-secret domain: connection/provider/model, main-call lane, session/branch lineage, and transport/cache partition where known.
  • Retry, compaction, auxiliary calls, provider/model changes, restart, branch, and overlapping turns cannot silently reuse the wrong baseline.
  • Missing capture/observation is a first-class unavailable answer; metering never depends on diagnostics.
  • Historical telemetry remains readable after current writers stop.

Workstreams

Two sequential workstreams, each independently reviewable and end-to-end verifiable. One PR per workstream.

1. Make the dispatched attempt own one trustworthy request observation

Reshape the existing capture/attempt path rather than adding a parallel mechanism:

  • Define one bounded, secret-free prepared-request observation from the actual model-call parameters. Its serialization, digest, and ordered semantic segments must be derived from the same representation; mark comparison-opaque segments explicitly.
  • Give a canonical physical attempt a durable join to that observation. Only the attempt authorizes it as dispatched. Keep the full serialized request in the existing private artifact boundary.
  • Make warm and cold latest_context reconstruction use canonical attempts plus their observations, not a best-effort provider-attempt record as a second authority.
  • Retire computeRequestShapeDiagnostic, RequestShapeDiagnostic, AiSdkBackend.priorRequestShape, predictive request/prefix hash writers, and live PromptSegmentEstimate writers. Keep historical readers.
  • Move the compaction malformed-summary failure fingerprint to the compactor's own real inputs instead of retaining the main-request prediction path for requestShapeHashBefore.
  • Keep tool-availability behavior and any still-demanded tool-source attribution in its own owner; do not copy it into request shape.

End-to-end acceptance:

  • A completed main call can be reconstructed after restart as one canonical attempt with its exact bounded semantic observation and context composition.
  • Aborting after capture but before dispatch leaves no sent observation in that reconstructed sequence.
  • Hash/serialization tests cover non-plain JSON values and opaque redaction without false equality.
  • Context composition remains available through the existing Runtime Host diagnostics path, including cold rebuild.
  • Current production code no longer writes the predictive request-shape fields, and compaction failure-circuit behavior remains covered without priorRequestShape.

2. Expose semantic prefix continuity from Runtime through Inspector

Build the user-visible vertical slice on the authority from workstream 1:

  • Link each comparable main attempt to its explicit causal predecessor and continuity domain.
  • Derive a small continuity result from the two observations: no_predecessor, preserved, diverged, unknown, or unavailable, plus preserved/previous segment counts and the first divergent segment when knowable.
  • Extend the existing Context Diagnostics / latest_context projection and Runtime Host protocol; do not add another event or UI-side calculation.
  • Show a compact Inspector tag such as Request prefix 8/8 preserved or Request prefix diverged at message 3.
  • Display provider-reported cache read/write data separately so semantic continuity is never confused with actual cache outcome.

End-to-end acceptance:

  • An ordinary second turn preserves every model-facing block from the first request and reports preserved; tail append is not a divergence.
  • Mutating, deleting, inserting, or reordering any old block reports the first divergence.
  • Restart and branch/copy reconstruct the same result from durable lineage; overlapping turns do not race through a shared mutable baseline.
  • Retry, compaction/opaque content, auxiliary calls, and provider/model/domain changes yield the explicit expected status rather than a false preserved.
  • A Desktop Inspector test verifies the Runtime result is rendered without recomputing it, alongside but distinct from provider cache usage.

Non-goals

  • Capturing or normalizing every provider's literal HTTP/WebSocket wire body.
  • Predicting cache eviction, TTL, or provider-internal cache keys.
  • Building a general provenance manifest for every context block.
  • Moving tool-availability ownership into request-shape diagnostics.
  • Keeping old and new request-shape writers in permanent dual-write mode.

Done

  • Workstream 1 merged: canonical dispatched request observation and old authority retired
  • Workstream 2 merged: durable semantic-prefix projection and Inspector consumption

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions