Add framed checkpoint v2 codec and bounded readback

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## Unreleased
 
+- Add an opt-in framed checkpoint artifact-v2 codec around the unchanged logical shared-checkpoint schema v1. Ordered configuration, scheduler, frontier, witness-ancestry, dedupe, semantic-index, finding, and metadata records are independently bounded and compressed; exact frame/component checksums and a terminal index support incremental readback without a whole stored artifact, decompressed artifact, or raw artifact string. Cross-format writers share one neutral no-clobber commit point, so a stable ID reuses exactly one verified `.json.gz` or `.inkcp` layout under the existing crash-recovery and retention protocol. Observed-before-work, preflight-without-output, and successfully encoded no-clobber losers reuse the winner; a private candidate that emitted bytes and then failed preserves its error for a later retry. Stable checkpoint IDs, exact resume state, default gzip writes, default CLI/MCP search behavior, search order, and resource policy are unchanged; storage listing and search-session disk accounting now recognize, reuse, and attribute either verified layout. This is a codec foundation only: framed v2 remains an explicit library opt-in pending Intercept plus second-family promotion evidence, and it does not complete #156, activate checkpoint/epoch/pressure policy, restart InkBench, or publish a release.
 - Extend the partial owner ledger across checkpoint reopen and report enrichment/finalization without adding another payload read, decode, parse, or stable-ID traversal. Successful checkpoint opens and resumes expose bounded stage-by-stage receipts, while typed corrupt/unsupported/resource-limit failures retain the work reached before failure. The opt-in accounted report builder measures the source exploration graph, finding-identity strings already materialized by enrichment, and the returned report; report saves separately distinguish created envelope graphs from reuse-parsed artifact graphs. These compact-JSON serialized-view proxies and their conservative potential sums are not exclusive retained-owner bytes or observed heap peaks. Runtime receipts remain outside saved report/checkpoint payloads and IDs, and bounded JSON streaming still avoids the monolithic report graph. This adds no checkpoint v2/framed readback, allocation, eviction, compaction, stopping policy, benchmark restart, or release; #156 and #216 remain open.
 - Add versioned retained-owner and finalization receipts without changing search policy or the checksum-bound observability-v1/v2 shapes. Shared pass telemetry now accounts for the bounded observability ledger through a non-recursive canonical UTF-8 projection and preserves its known high-water state across resume; older checkpoints remain readable and explicitly report incomplete owner history. Checkpoint saves report logical graph bytes, streamed artifact/chunk bytes, compressed output, configured compression capacity, durable payload/manifest bytes, and whether encode/compress work was applied or an existing stable ID was reused. Report saves similarly expose canonical identity-string, artifact-serialization, conservative potential string, and durable bytes outside the content-derived report. Configured finalization headroom is labeled unallocated capacity and remains excluded from retained/process totals. Stable-ID algorithms are unchanged: deterministic owner state participates in newly emitted checkpoint content, while post-write receipts never feed back into the IDs or payloads they measure. These fields are observational/accounting inputs only: they do not emit the reserved checkpoint/epoch/pressure reasons, change frontier order, activate a budget or stopping policy, add checkpoint v2, or publish a release; #156 and #216 remain open.
 - Extend shared-search observability to nested schema v2 with monotonic sample sequences, canonical coalesced reason vectors, an exact 15-bit trigger mask, a fixed eight-count boundary-local trigger-yield tuple, category/frontier/cadence/termination triggers, and category-specific first/last discovery, current/longest dry-state, and identities-per-million-transition facts. The redundant vector/mask/tuple preserves exact retained-boundary triggers across compaction while aggregate interval deltas are rebuilt: adjacent complete-history samples require the tuple to equal the semantic interval delta, while genuine compaction gaps or incomplete v1 history allow only a componentwise-bounded tuple. Observability-v1 migration synthesizes masks only from explicit historical boundaries and uses a zero tuple to mean no v1 trigger was recorded, not that an interval was event-free. Sequence validation applies monotonic and feasible record/state/cadence bounds without claiming exact replay of discarded compaction history. Emitted v2 checkpoints add a domain-separated, key-order-independent canonical SHA-256 self-check binding every fixed known ledger field; it detects stale or accidental mutation but is not authentication because a writer can recompute it, and persisted payload/manifest digests remain the external artifact-corruption boundary. V1 and pre-ledger inputs gain the nested checksum on their first emitted v2 checkpoint. Live process sampling and progress output remain bounded to cadence and final termination while the compacted deterministic ledger retains event boundaries. Checkpoint construction remains side-effect-free and deterministic: the persisted ledger participates in stable checkpoint identity, while observability-v1 checkpoints migrate explicitly with incomplete event history and nullable facts rather than invented history. This remains an independent partial #216 slice: `checkpoint`, `epoch`, and generic `pressure` reasons are reserved but not emitted; retained-owner accounting remains incomplete; no resource, allocation, or stopping policy is activated; #216 remains open; and the earlier V1 overhead study is not an exact-head overhead claim for this V2 implementation.

README.md

@@ -134,7 +134,7 @@ For a new project containing one `.ink` file, `inkcheck init` creates this confi
 
 `--save-report` atomically stores a versioned report under `.inkcheck/reports/` and returns its stable content-and-entrypoint-derived ID. A later session can use `inkcheck artifacts list --json` and `inkcheck artifacts show <report-id> --json`; reopening reports whether the saved evidence is `current`, `stale`, or `path_changed` against the present entrypoint. Reports can contain story text, variables, and exact witnesses, so the agent kit ignores them by default. See [local report artifacts](docs/local-artifacts.md) for the trust, privacy, and compatibility contract.
 
-Long base-shared runs can also persist their exact live frontier locally. Start with `--search=shared --no-min-repro --save-checkpoint`, then continue later with `inkcheck resume <checkpoint-id> --max-states N`; `N` is the larger total grant, not extra hidden work. `inkcheck checkpoints list/show` reports bounded metadata, durable compressed size, storage encoding, and source freshness. New checkpoint files are streamed gzip artifacts; older plain schema-v1 JSON remains readable. Checkpoints are private, atomic, source/config-bound, ignored by default, and retention-capped; they may contain authored text and runtime state. See [local resumable checkpoints](docs/local-checkpoints.md). Shared passes also expose a bounded [resource/yield observability ledger](docs/shared-search-observability.md) that keeps deterministic logical accounting separate from live heap/RSS observations. MCP agents can use the same exact foundation through durable [`start_search` / `inspect_search` / `continue_search` / `cancel_search` result windows](docs/mcp-search-sessions.md). They can also use `add_goal` for an explicit additive directed probe that starts from the story root and leaves that exact base frontier untouched. Portfolio, shared-variable, assertions, directed-frontier resume, and hosted jobs do not use this checkpoint contract yet.
+Long base-shared runs can also persist their exact live frontier locally. Start with `--search=shared --no-min-repro --save-checkpoint`, then continue later with `inkcheck resume <checkpoint-id> --max-states N`; `N` is the larger total grant, not extra hidden work. `inkcheck checkpoints list/show` reports bounded metadata, durable compressed size, storage encoding, and source freshness. CLI/default checkpoint files are streamed gzip artifacts; older plain schema-v1 JSON remains readable, and library callers can explicitly evaluate the opt-in [framed artifact-v2 codec](docs/shared-checkpoint-artifact-v2.md) without changing the logical checkpoint ID. Checkpoints are private, atomic, source/config-bound, ignored by default, and retention-capped; they may contain authored text and runtime state. See [local resumable checkpoints](docs/local-checkpoints.md). Shared passes also expose a bounded [resource/yield observability ledger](docs/shared-search-observability.md) that keeps deterministic logical accounting separate from live heap/RSS observations. MCP agents can use the same exact foundation through durable [`start_search` / `inspect_search` / `continue_search` / `cancel_search` result windows](docs/mcp-search-sessions.md). They can also use `add_goal` for an explicit additive directed probe that starts from the story root and leaves that exact base frontier untouched. Portfolio, shared-variable, assertions, directed-frontier resume, and hosted jobs do not use this checkpoint contract yet.
 
 ## Hosted checker
 
@@ -186,7 +186,7 @@ Within a single run, inkcheck spends its state budget across complementary searc
 
 `--search=shared` opts into an experimental alternative: deep, novelty-first, and seeded views draw from one deduplicated frontier and each reachable state is expanded at most once. This can spend a tight budget more efficiently when several strategies would otherwise rediscover the same state. JSON telemetry separates pending and active checkpoint JSON/variables, retained witness ancestry, dedupe and semantic indexes, frontier references, and findings; it also reports released ancestry and stale-view compactions. These are deterministic accounted payload/structural estimates, not exact V8 heap usage. It is not the default yet: some early-choice structures may still favor the portfolio's independent random and beam passes.
 
-Library consumers evaluating long-running shared jobs can use `exploreSharedResumable(...)`. When budget remains, it returns a source-bound, versioned JSON checkpoint containing the exact live base-shared frontier; a later call raises the total grant (for example 100k to 1m) and continues without replaying the first 100k. Split-run equivalence is regression-tested against an uninterrupted run. This is an engine foundation, not yet a CLI persistence feature: checkpoints may contain authored text, variables, runtime state, and witness paths, and schema v1 deliberately excludes assertions, goals, and variable/goal-aware shared modes. See [shared checkpoint schema v1](docs/shared-checkpoint-schema-v1.md).
+Library consumers evaluating long-running shared jobs can use `exploreSharedResumable(...)`. When budget remains, it returns a source-bound, versioned JSON checkpoint containing the exact live base-shared frontier; a later call raises the total grant (for example 100k to 1m) and continues without replaying the first 100k. Split-run equivalence is regression-tested against an uninterrupted run. This logical engine foundation also underlies the local CLI persistence above; its storage envelope is separate. Checkpoints may contain authored text, variables, runtime state, and witness paths, and schema v1 deliberately excludes assertions, goals, and variable/goal-aware shared modes. See [shared checkpoint schema v1](docs/shared-checkpoint-schema-v1.md).
 
 `--search=shared-variable` is a narrower experiment that gives 12.5% of shared-frontier selections to states reached through uncommon variable snapshots or transitions. The boost is bounded and the deep, novelty, and seeded views remain active. It can help mechanically driven storylet graphs, but it is not uniformly better; the checked-in [comparison table](docs/search-experiments.md) includes both gains and regressions.
 

benchmarks/agent-readiness-v1/expected.json

@@ -8,7 +8,13 @@
       "config": 1,
       "projectInspection": 4,
       "artifact": 1,
-      "checkpointArtifact": 1,
+      "checkpointArtifact": 2,
+      "checkpointArtifactSupported": [
+        1,
+        2
+      ],
+      "checkpointArtifactDefaultWrite": 1,
+      "checkpointListResponse": 1,
       "searchSession": 5,
       "regressionArtifact": 2,
       "campaignPolicy": 1

docs/agent-discovery.md

@@ -15,7 +15,7 @@ The default MCP profile is deliberately compact: `inkcheck_capabilities`, `inspe
 The response is deterministic for an installed Inkcheck version and reports:
 
 - `schemaVersion` and `inkcheckVersion`
-- report, configuration, project-inspection, report-artifact, checkpoint-artifact, search-session, and campaign-policy schema versions
+- report, configuration, project-inspection, report-artifact, checkpoint-artifact, search-session, and campaign-policy schema versions. `schemas.checkpointArtifact` is the latest supported persisted envelope (2); `checkpointArtifactSupported` lists `[1, 2]`, `checkpointArtifactDefaultWrite` remains 1, and `checkpointListResponse` remains 1. The logical shared-search checkpoint nested inside either artifact remains schema 1.
 - default and maximum state/depth limits
 - supported search modes
 - explicit feature flags, including `false` for features not yet available