Skip to content

feat(runner): capture bounded hosted debug checkpoints - #1081

Closed
bheemreddy-samsara wants to merge 1 commit into
tester-army:mainfrom
bheemreddy-samsara:feat/debug-checkpoints
Closed

bheemreddy-samsara wants to merge 1 commit into
tester-army:mainfrom
bheemreddy-samsara:feat/debug-checkpoints

Conversation

@bheemreddy-samsara

@bheemreddy-samsara bheemreddy-samsara commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What and why

  • Opt into awaited, bounded before/after capture attempts for selected hosted top-level public steps, independent of model vision and the nonblocking progress callback. Own and validate policy before asynchronous opening.
  • Mask formatter-owned scalar fields before clipping; omit historical selections only from checkpoints. Preserve default model/executor observations.
  • Bind pixels at capture start and JSON after final masking. Validate canonical unavailable records against original SDK boundaries before the first write and again before store handoff. Opaque providers withhold content before observation; fixed null-capture facts remain separate from content permission.
const session = await host.openSession({
  name: 'checkout',
  debugCheckpoints: {
    pixels: true, maxSteps: 10, maxBytes: 2 * 1024 * 1024,
    captureTimeout: 500, retain: 'retain-on-failure',
  },
});
await session.close();
const checkpoints = session.checkpoints();

The example overrides the unchanged defaults: 50 steps, 8 MiB, 1000 ms, pixels off, retain-on-failure.

Addresses #1077. Follow-up to merged hosted sessions PR, rebased onto main at 4b4b29727dde17eb135b030c031c10607a2d6299. This branch contains only this follow-up's scope.

Workflow

flowchart LR
  A[Opted-in before or after boundary] --> B{Prospective content proof?}
  B -->|Yes| C[Bounded capture with original ticket]
  B -->|No| D[Fixed unavailable facts without capture]
  C --> E[Operation and required closing lifecycle]
  D --> E
  E --> F[Retention and final source proof]
  F --> G[Validate bytes before first write]
  G --> H[Recheck after read before store]
  H --> I[Copied checkpoint view]
Loading

Verified

Ran it locally: yes, at 05d0f8dc3ad9d48a474c29d71b316ee4e819f4e1. One feature commit; pinned base unchanged.

  • pnpm check and builds passed, including emitted declarations, SDK assertions, schemas and docs. Units: 2,918 passed, 2 existing skips. Owning selected gate: 604 passed.
  • Parent gate: 510 passed. Independent 13-path correction review: 746 passed, plus 157 overlapping forced-UUID collision checks and a no-emit typecheck. Counts overlap, not additive coverage totals.
  • Public async-mutation/getter controls verify one owned, validated policy snapshot before engine opening, without freezing caller objects. Invalid policies reject before engine start.
  • Retention assertions now run outside swallowed callbacks; an independent mutant exposes the old oracle weakness. Windows separator normalization has positive/negative controls, not native Windows validation.
  • Viewport schema dependencies match capture identities. Fewer than 8 remaining bytes suppress pixels because even the PNG signature cannot fit; 1/7/8-byte controls verify the boundary. This is not a compressed-size or engine peak-memory estimate. Retained-content quota, 200-record ceiling and 2,048 private positions per capture remain distinct limits.
  • Built CLI Chromium: 15 files, 77 passed, 1 existing skip. Original custody, mask-before-caps, canonical unavailable facts, monotonic secrecy, retention and whole-JSON/first-byte privacy guards remain implemented. No ID filtering or weakened collision assertions.
  • All eight Cubic findings have published, attributed replies and resolved threads. Current independent review covered the correction delta and critical callers, not exhaustive historical approval. Private composition predates these corrections and is not current-head public interoperability proof.
  • Exact-head GitHub evidence: docs, spec, web benchmark and deterministic mobile gates all passed. Secret-dependent fork jobs are skipped, not passes. Cubic automatic re-review completed on this exact head with zero issues across all 13 correction files, and no threads remain open. No manual bot trigger was sent. Human approval and consumer/device validation remain separate.
  • Capture uses the existing deadline; unavailable facts may be recorded without observation. Not an inner-action/poll/fixture/CLI/MCP/replay timeline. Synthetic faults and Chromium are not native-device/cloud/live-model/physical-disk/backend/adoption proof. Full monorepo and live A/B were not rerun; human approval and merge authorization remain separate.

Contracts

  • Minor changeset, runner reference and debugging skill; emitted declarations and SDK type assertions; checkpoint schema and both fixtures.
  • No breaking change or new engine-contract requirement.

View guided diff

@bheemreddy-samsara
bheemreddy-samsara force-pushed the feat/debug-checkpoints branch 2 times, most recently from 3f1a3b4 to 48ed156 Compare October 13, 2026 19:28
@bheemreddy-samsara
bheemreddy-samsara marked this pull request as ready for review October 13, 2026 22:37

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 32 files

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread packages/e2e/tests/unit/debug-checkpoint-wiring.test.ts Outdated
Comment thread docs/reference/runner.mdx Outdated
Comment thread packages/e2e/src/run/open-session.ts Outdated
Comment thread packages/e2e/tests/unit/debug-checkpoints-prospective.test.ts Outdated
Comment thread packages/e2e/schema/debug-checkpoint-v1.schema.json
Comment thread packages/e2e/src/run/debug-checkpoints.ts Outdated
Comment thread docs/reference/runner.mdx Outdated
Comment thread .changeset/host-debug-checkpoints.md Outdated
Tree-identical consolidation of 3f1a3b4 onto pinned base 4b4b297. No content or base change.
@okwasniewski

Copy link
Copy Markdown
Member

Thanks for the thorough work. The runner already records each step's screen in the trace (step-screens.ts, added in #869), so a second checkpoint pipeline with its own schema (debug-checkpoint-v1), budgets, and retention modes is more surface than we can maintain. It also reads videoRetentionStatus/finalizeVideo from #1080 (execute.ts:147,149, debug-checkpoints.ts:167) that nothing in this PR sets.

If the current per-step trace evidence is missing something you need (for example an optional screenshot per step), could you describe that gap on #1077? We'd rather extend the existing trace data than add debugCheckpoints.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants