Skip to content

fix: recover fork compression blocks from parent state when historical compress inputs are stripped #376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ranxianglei wants to merge 4 commits into
base: master
Choose a base branch
from
Open

fix: recover fork compression blocks from parent state when historical compress inputs are stripped #376

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bb23fe4
feat: recover fork compression blocks from parent state when historic…
Sep 9, 2026
5ba7484
merge: resolve conflict with master (storagePath warning + fork paren…
Sep 9, 2026
cfa2f1f
docs: record master merge + conflict resolution in devlog
Sep 9, 2026
8649ce4
Merge remote-tracking branch 'origin/master' into pr376-sync-master
Sep 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions devlog/2026-09-09_fork-parent-state-transfer/DESIGN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# DESIGN - Recover compression blocks from parent state when forks omit historical compress inputs

- Task ID: `2026-09-09_fork-parent-state-transfer`
- Home Repo: `opencode-acp`
- Created: 2026-09-09
- Status: Accepted

## 1. Problem Statement

- **What problem are we solving?** A fork reconstructs its prune state only by replaying completed `compress` tool parts that still carry `state.input`. When the fork copy strips those inputs, no blocks are rebuilt and the copied raw parent history becomes visible, blowing up token usage.
- **Why now?** A production fork hit 20,581+ messages / 925K tokens against a 400K limit with `no_executable_candidates` and no `rebuild: reconstructed` event (issue #375).

## 2. Goals & Non-Goals

- **Goals**:
- Recover the parent's compression blocks into a fork that has a `parentID` and no fork-local state.
- Translate block message coverage from parent raw IDs to fork raw IDs across the copied shared prefix.
- Save independent fork-local state without inheriting nudge cadence / current-turn state and without mutating the parent.
- Fall back to the existing historical replay when parent transfer is not possible.
- **Non-Goals**:
- Changing the `isSubAgent` classification of forks (we are robust to the resulting ref shift, but do not alter it).
- A shared/cross-session block store.

## 3. Current Architecture

- `ensureSessionInitialized` (lib/state/state.ts) loads the fork's state file; when absent it calls `rebuildCompressionState` (lib/state/rebuild.ts), which replays completed `compress` parts with an object `state.input` and rebuilds blocks from the recorded `startId`/`endId` refs.
- **Pain point**: replay depends entirely on `state.input` surviving the fork copy. Stripped inputs → 0 blocks.

## 4. Proposed Architecture

- **Overview**:

```
ensureSessionInitialized (fork, no local state)
│
├─ getForkParentId(client, sessionId) -> parentId | null
│
├─ if parentId:
│ recoverFromParentState(client, state, forkMessages, parentId, logger)
│ 1. loadSessionState(parentId) -> parent state (or null)
│ 2. parent blocks? (>=1) -> else 0
│ 3. client.session.messages(parentId) -> parent messages (or [])
│ 4. buildSharedPrefixMapping(parent, fork) # position + time.created + role
│ 5. assignMessageRefs(state, fork) # fork refs (idempotent)
│ 6. translate each parent block (raw IDs + startId/endId refs)
│ 7. state.prune.messages = loadPruneMessagesState(translated)
│ return activeBlockCount
│
├─ if recovered === 0:
│ rebuildCompressionState(state, forkMessages, config, logger) # existing replay
│
└─ if recovered > 0: saveSessionState(state) # fork-local, independent
```

- **Key components**:
- `recoverFromParentState` (lib/state/fork-transfer.ts) — orchestrates load → map → translate → apply.
- `buildSharedPrefixMapping` — matches parent/fork message lists by position + `time.created` + `role`, stopping at the first mismatch.
- `translateBlock` / `remapBoundaryRef` / `translateByMessageId` — remap a block's coverage and refs onto fork IDs.
- `getForkParentId` (lib/state/utils.ts) — returns the session's `parentID` (string | null).
- **Data flow**: parent state file → in-memory parent blocks → translated fork blocks → fork `prune.messages` → fork state file.
- **API / interface changes**: new exported `recoverFromParentState`; new exported `getForkParentId`; `isSubAgentSession` now delegates to `getForkParentId` (same boolean result). No persisted-format change.

## 5. Design Decisions & Rationale

| Decision | Options Considered | Chosen | Why |
|----------|--------------------|--------|-----|
| How to map parent→fork messages | (a) align by ref `mNNNNN`; (b) align by position + `time.created` + role | (b) | Forks have `parentID` ⇒ `isSubAgent` ⇒ `assignMessageRefs` skips the fork's first user message ⇒ fork refs are shifted by one vs the parent. Ref alignment would point blocks at the wrong messages. `time.created`/`role` are preserved by the fork copy; only raw IDs change. |
| Which blocks to transfer | only active; all (active + inactive) | all, then rebuild active sets | Mirrors how the parent's own state is loaded (`loadPruneMessagesState` rebuilds `activeBlockIds`/`activeByAnchorMessageId` from all blocks). Preserves nested/inactive lineage. |
| Blocks whose anchor is outside the shared prefix | fail the whole transfer; skip the block | skip the block | Handles partial forks and parents that continued after the fork; recover what is safely placeable. |
| When to fall back to replay | never; on any failure | on any failure (no parent state / no prefix / no blocks / fetch error / nothing translatable) | Preserves existing behavior as a strict superset. |
| Nudge / current-turn state | copy from parent; do not copy | do not copy | A fork is a fresh session; inheriting cadence would mis-time nudges. |

## 6. Impact Analysis

- **Backward compatibility**: none broken. New code path only triggers when `parentID` is set AND no fork-local state exists AND the fork path previously produced 0 blocks. Persisted format unchanged.
- **Performance**: one extra `client.session.messages(parentId)` call per fork init (rare, one-time). Translation is O(blocks + messages).
- **Security**: no new network targets (same `client`); reads the parent's local state file and its messages via the existing SDK client.
- **Dependencies**: none new.

## 7. Migration Plan

- **Steps**: none. The feature is additive and self-contained.
- **Feature flags / gradual rollout**: none required; behavior is a strict superset of the current replay path.

## 8. Open Questions

- [ ] Should forks stop being classified as sub-agents (which would remove the ref shift at the root)? Out of scope here, but worth a follow-up issue since it also affects the existing replay path's ref resolution.
60 changes: 60 additions & 0 deletions devlog/2026-09-09_fork-parent-state-transfer/REQ.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# REQ - Recover compression blocks from parent state when forks omit historical compress inputs

- Task ID: `2026-09-09_fork-parent-state-transfer`
- Home Repo: `opencode-acp`
- Created: 2026-09-09
- Status: InProgress
- Priority: P1
- Owner: ranxianglei
- References: issue #375 (ranxianglei/opencode-acp)

## 1. Background & Problem Statement

- **Context**: OpenCode forks receive a new session ID and a copied message history. ACP keeps per-session prune state (compression blocks) in `~/.local/share/opencode/storage/plugin/acp/{sessionId}.json`. A fork has no fork-local state file, so ACP must reconstruct its prune state.
- **Current behavior (symptom)**: `ensureSessionInitialized` (lib/state/state.ts) reconstructs fork state only by replaying completed historical `compress` tool parts whose `state.input` survives (`rebuildCompressionState` in lib/state/rebuild.ts). When the fork copy omits/strips those inputs, **zero** blocks are rebuilt, the copied raw parent history stays visible, and token usage can jump dramatically (observed: 20,581+ messages, 925,103 tokens vs a 400K limit; repeated `reason=no_executable_candidates`, no `rebuild: reconstructed …` event).
- **Expected behavior**: When a session has a `parentID` and no fork-local state, ACP should recover the parent's compression blocks by translating their message coverage onto the fork's message IDs, so the fork prunes the same shared prefix the parent already compressed.
- **Impact**: Forks of long, already-compressed sessions lose all compression and overflow their context window.

## 2. Reproduction (if applicable)

- **Environment**:
- Node: 22 / 24 (CI matrix)
- OS/Arch: linux
- **Minimal reproduction steps**:
1) Parent session compresses a range (a `compress` tool part with `state.input` exists in the parent, and the parent's ACP state file has ≥1 block).
2) Fork the session. The fork copies the parent's messages but the `compress` part's `state.input` is stripped/omitted.
3) On first fork init, `rebuildCompressionState` finds no replayable compress input → 0 blocks → the copied raw history is visible.
- **Relevant configuration**: default.

## 3. Constraints & Non-Goals

- **Constraints**:
- Backward compatibility: must not break existing fork replay (stripped-input case where no parent state exists) or sub-agent behavior. Persisted state format unchanged.
- Performance: one extra `client.session.messages` fetch per fork init (forks are rare; one-time). Acceptable.
- Resource limits: none.
- Must NOT mutate the parent state; must NOT inherit parent nudge cadence / current-turn state.
- **Non-Goals** (explicitly out of scope):
- Fixing the broader sub-agent/fork `isSubAgent` classification (forks are currently treated as sub-agents, which shifts refs by one). This change is *robust to* that shift but does not change it.
- Cross-session block sharing / a shared block store.

## 4. Acceptance Criteria (must be testable)

- **Correctness**:
- [x] A fork with a `parentID`, a parent state with ≥1 block, and stripped compress inputs recovers the parent's active blocks with message IDs remapped to the fork's raw IDs.
- [x] Recovery is robust to the fork's ref shift (verified with `isSubAgent=true`): blocks map to the correct fork raw IDs, not shifted refs.
- [x] `startId`/`endId` message refs are remapped to the fork's refs; `bN` block refs are preserved.
- [x] Fork-local state is saved as an independent file; the parent state file is not mutated.
- **Performance / Stability**:
- [x] Falls back to historical replay (returns 0) when: no parent state, no shared message prefix, parent has no blocks, or the parent message fetch fails.
- [x] Parent nudge cadence / current-turn state is NOT copied into the fork.
- **Regression**:
- [x] New test file `tests/rebuild-parent.test.ts` (7 tests) added and passing; full suite green (1084 pass / 0 fail).

## 5. Proposed Approach (optional)

- **Affected modules & entry files**:
- `lib/state/fork-transfer.ts` (new) — `recoverFromParentState()`.
- `lib/state/state.ts` — fork init path calls parent-transfer first, then falls back to replay.
- `lib/state/utils.ts` — new `getForkParentId()`; `isSubAgentSession` refactored to reuse it.
- **Risks**: mapping correctness if the fork copy reorders/drops messages (mitigated by position + `time.created` + role matching with a hard stop at the first mismatch); parent message fetch cost (one-time).
- **Rollback strategy**: revert the PR; the fork path reverts to replay-only behavior. No persisted-format migration.
95 changes: 95 additions & 0 deletions devlog/2026-09-09_fork-parent-state-transfer/WORKLOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# WORKLOG - Recover compression blocks from parent state when forks omit historical compress inputs

- Task ID: `2026-09-09_fork-parent-state-transfer`
- Home Repo: `opencode-acp`
- Status: InProgress
- Updated: 2026-09-09 20:30

## 1. Summary

- **What was done** (1–3 sentences): Added a parent-state transfer path for fork initialization. When a fork (session with `parentID`) has no fork-local state file, ACP now loads the parent's ACP state, maps the parent's message IDs to the fork's message IDs across the copied shared prefix, translates the parent's compression blocks onto the fork's IDs, and saves independent fork-local state. Falls back to the existing historical replay when the transfer is not possible.
- **Why** (1–3 sentences): Forks strip `compress` tool `state.input`, so replay-only reconstruction yields zero blocks and the copied raw parent history overflows the fork's context. Translating the parent's already-built blocks recovers the compression without depending on `state.input` surviving the copy.
- **Behavior / compatibility changes**: Yes — additive. New recovery path for forks with a `parentID`; existing replay is preserved as the fallback. No persisted-format change. Does not mutate the parent; does not inherit nudge cadence.
- **Risk level**: Medium (new async path touching fork init; mitigated by strict fallback to the existing replay on any failure).

## 2. Change Log

### Commits

| Commit | Description |
|--------|-------------|
| `b389f61` | feat: recover fork compression blocks from parent state when historical compress inputs are stripped |
| `5ba7484` | merge: resolve conflict with master (storagePath warning + fork parent-state transfer) |

### Conflict resolution (2026-09-09)

Merged master (`5135dfd`, incl. #380 storagePath) into the PR branch. Single conflict in
`lib/state/state.ts` `ensureSessionInitialized()`: this PR's fork-recovery comment block
overlapped the new storagePath default-location warning. Resolution keeps both — the
storagePath warning runs first, then the fork recovery path (parent-state transfer →
historical replay fallback). No behavior change on either side. Verified after merge:
`npm run typecheck` clean, `npm run build` success, `npm run test` → 1138 pass / 0 fail.

### Key Files

- `lib/state/fork-transfer.ts` (new) — `recoverFromParentState()` and its helpers (`buildSharedPrefixMapping`, `translateBlock`, `remapBoundaryRef`, `translateByMessageId`, `extractIdentity`).
- `lib/state/state.ts` — fork init path: resolve `parentId` via `getForkParentId`, try `recoverFromParentState` first, fall back to `rebuildCompressionState`, save if >0.
- `lib/state/utils.ts` — new `getForkParentId()`; `isSubAgentSession` refactored to reuse it (same boolean result).
- `tests/rebuild-parent.test.ts` (new) — 7 unit tests.

## 3. Design & Implementation Notes

- **Entry point / key function**: `recoverFromParentState(client, state, forkMessages, parentId, logger): Promise<number>` in `lib/state/fork-transfer.ts`.
- **Key logic explanation** (non-trivial):
- **Mapping**: `buildSharedPrefixMapping` reduces both the parent and fork message lists to `{ id, role, time.created }` identities (skipping synthetic `msg_dcp_*` / `msg_acp_*` messages and empty IDs) and matches them by **position + `time.created` + role**, stopping at the first mismatch. This is deliberately **not** ref-based: forks carry `parentID`, so `isSubAgent` is true and `assignMessageRefs` skips the fork's first user message, shifting the fork's `mNNNNN` refs by one relative to the parent. Matching on `time.created`/`role` (preserved by the fork copy) is robust to that shift.
- **Translation**: each parent block's `directMessageIds` / `effectiveMessageIds` / `directToolIds` / `effectiveToolIds` / `anchorMessageId` / `compressMessageId` are remapped through the mapping; `startId`/`endId` message refs are remapped via `parentByRef → parentRaw → mapping → forkRaw → forkByRawId → forkRef`, while `bN` block refs are preserved (block IDs are kept, so nested lineage stays valid). Blocks whose `anchorMessageId` or `compressMessageId` falls outside the shared prefix are skipped (partial forks / parent continued after fork).
- **State assembly**: translated blocks + translated `byMessageId` are passed to `loadPruneMessagesState`, which rebuilds `activeBlockIds` / `activeByAnchorMessageId` / `nextBlockId` / `nextRunId` exactly as when the parent's own state is loaded. Nudge and current-turn state are intentionally not copied.
- **Tool ids**: `directToolIds` / `effectiveToolIds` hold tool **call** ids (`part.callID`), which are preserved verbatim across a fork copy. They are copied as-is (deduped), NOT run through the message-id mapping — doing so would drop every entry (call ids never equal message ids).
- **Active-only transfer**: `state.prune.messages` is assigned only when at least one **active** block is translatable. If only inactive blocks survive the translation (e.g. the parent continued after the fork), the fork state is left untouched and the caller falls back to replay — otherwise replay would run on top of already-transferred inactive blocks.
- **Boundary refs (best-effort)**: `startId`/`endId` are remapped through the raw-id mapping. When the boundary message's fork raw id has no ref (only the fork's first user message, skipped under the sub-agent classification), the original parent ref is kept. This is metadata only (dedup key, GC-merge boundary, display); core pruning is `byMessageId`-based and unaffected.
- **Fallback**: returns 0 (→ caller runs `rebuildCompressionState`) when the parent state is missing, the parent has no blocks, the parent message fetch fails, the shared prefix is empty, no block is translatable, or no **active** block is translatable.

## 4. Testing & Verification

### Build & Test Commands

```sh
cd opencode-acp && npm run build
node --import tsx --test tests/*.test.ts
node --import tsx --test tests/rebuild-parent.test.ts
npx tsc --noEmit
```

### Test Coverage

- New/modified test files: `tests/rebuild-parent.test.ts` (new, 7 tests).
- Test count: 1084 total, 1084 pass, 0 fail (was 1077 before this change).
- Key scenarios verified:
- Recovery when `state.input` is stripped (the #375 case): 1 block recovered, `anchorMessageId`/`compressMessageId`/`effectiveMessageIds`/`byMessageId` all remapped to fork raw IDs; the compress message is not pruned. Also asserts `startId`/`endId` remap, summary text is preserved, and a tool call id (`tool-1`) survives the transfer.
- Ref-shift robustness: with `isSubAgent=true` (fork's first user message skipped, refs shifted), the same correct raw-ID mapping is produced; `endId` remaps correctly and the best-effort `startId` fallback is pinned.
- Fallbacks: no parent state → 0; no shared prefix (sub-agent messages) → 0; parent has no blocks → 0.
- Independence: fork does not inherit parent nudge tokens; parent state file is not mutated (reloaded: 1 block, nudge token preserved).
- Out-of-prefix skip: parent with two blocks (one anchored inside the copied prefix, one anchored after it) → only the in-prefix block is transferred; the out-of-prefix block is skipped.

### Results

- **PASS/FAIL**: PASS.
- **Key logs/data**: `npm run typecheck` clean; `npm run build` → `dist/index.js` 431.73 KB; `npm run test` → `# pass 1084 / # fail 0`.

## 5. Risk Assessment & Rollback

- **Risk points**: mapping correctness if a fork copy reorders or drops messages (mitigated by the hard stop at the first position/`time.created`/role mismatch); one extra parent message fetch per fork init.
- **Rollback method**:
- Revert commit(s): `<sha>`
- Rollback impact: forks revert to replay-only reconstruction (the pre-change behavior). No data migration.
- **Compatibility notes** (data format, config schema): No — persisted state format and config are unchanged.

## 6. Lessons Learned (optional)

- What went well: reusing `loadPruneMessagesState` to rebuild active sets kept the translation consistent with how the parent's own state is loaded.
- What could be improved: the ref shift caused by classifying forks as sub-agents is a latent issue that also affects the existing replay path's ref resolution — see DESIGN §8.
- Reusable conclusions: match fork↔parent messages on `time.created`/`role`/position, never on `mNNNNN` refs.

## 7. Follow-ups (optional)

- [ ] Consider whether forks should stop being classified as sub-agents (removes the ref shift at the root; also fixes the existing replay path's ref resolution).
Loading
Loading