EPIC — Native Pi-family structured sessions over Pi/OMP RPC

[44madfire]

Goal

Deliver first-class Pi and OMP structured Native Chat on current Orca architecture using one shared Pi-family provider implementation.

The project is now a re-port, not a continuation of the old #22–#30 implementation series.

Canonical design: #20
Prototype/reference implementation: #51
Pi upstream tracking: stablyai#21558
OMP upstream tracking: stablyai#22650

Architectural boundary

The project follows one rule:

We own Pi/OMP semantics; upstream Orca owns the generic structured-provider lifecycle.

Do not write the real Pi/OMP providers until we have selected and proven the upstream generic provider contract.

Why the epic was reset

The original child plan (#22–#30) targeted an older Orca structured-session architecture and assumed this fork would own:

• provider-router widening;
• durable provider handle plumbing;
• process lifecycle;
• journal/timeline translation;
• restart/handoff scaffolding.

Upstream Orca is now actively generalizing those areas for third providers. Continuing the old implementation would create a competing provider framework and a large long-term fork surface.

The old issues are retained as historical implementation/specification references but are superseded by #52–#59.

Current execution plan

Phase 0 — choose the upstream architecture before provider implementation

• PIF-0A — Sync the Orca fork to current upstream before Pi-family re-port #52 — PIF-0A: sync the Orca fork to current upstream
• PIF-0B — Select and document Orca’s upstream generic structured-provider contract #53 — PIF-0B: select/document upstream generic structured-provider contract — depends on PIF-0A — Sync the Orca fork to current upstream before Pi-family re-port #52
• PIF-0C — Prove the selected generic provider contract with a fake Pi-family adapter #54 — PIF-0C: prove selected contract with fake Pi-family adapter — depends on PIF-0B — Select and document Orca’s upstream generic structured-provider contract #53

Gate: no real Pi/OMP provider integration until #54 passes through the real StructuredAgentSessionHost.

Phase 1 — provider protocol semantics

• PIF-1 — Re-port the Pi-family RPC protocol/transport on the selected Orca provider contract #55 — PIF-1: re-port Pi-family RPC protocol/transport — depends on PIF-0C — Prove the selected generic provider contract with a fake Pi-family adapter #54
• PIF-2 — Define Pi/OMP dialects, version policy, and durable history normalization #56 — PIF-2: Pi/OMP dialects, OMP version policy, durable history normalization — depends on PIF-1 — Re-port the Pi-family RPC protocol/transport on the selected Orca provider contract #55

These issues should remain mostly independent of generic Orca journal/UI internals.

Phase 2 — real provider

• PIF-3 — Implement the real Pi-family structured provider on Orca’s generic provider stack #57 — PIF-3: real Pi-family provider on upstream generic stack — depends on PIF-0C — Prove the selected generic provider contract with a fake Pi-family adapter #54 + PIF-1 — Re-port the Pi-family RPC protocol/transport on the selected Orca provider contract #55 + PIF-2 — Define Pi/OMP dialects, version policy, and durable history normalization #56
• PIF-4 — Add Pi-family controls, interactions, cancellation, and restart hardening #58 — PIF-4: controls, interactions, cancellation, restart hardening — depends on PIF-3 — Implement the real Pi-family structured provider on Orca’s generic provider stack #57

Phase 3 — integration and upstream readiness

• PIF-5 — Current-main integration, cleanup, and upstream-ready Pi/OMP structured Native Chat #59 — PIF-5: current-main integration, cleanup, and upstream-ready implementation — depends on PIF-3 — Implement the real Pi-family structured provider on Orca’s generic provider stack #57 + PIF-4 — Add Pi-family controls, interactions, cancellation, and restart hardening #58

Dependency graph

#52 sync
  ↓
#53 choose upstream generic contract
  ↓
#54 fake-provider conformance spike
  ↓
#55 Pi-family RPC transport
  ↓
#56 Pi/OMP dialect + history + version policy
  ↓
#57 real Pi-family provider
  ↓
#58 controls/interactions/restart hardening
  ↓
#59 integration/cleanup/upstream readiness

#55 may begin exploratory pure-protocol work while #54 is finishing, but no production Orca-facing provider wiring should precede the #54 contract decision.

Upstream work to monitor

At the start of each child issue, re-check whether these have merged or materially changed:

• refactor(native-chat): keep the provider resume handle opaque to shared code stablyai/orca#24991
• refactor(native-chat): structured agents declare their capabilities instead of shared code naming Claude and Codex stablyai/orca#25076
• feat(native-chat): open structured chat's wire and stored records to registered agents, behind a negotiated capability stablyai/orca#25159
• Share the managed process lifecycle for structured providers stablyai/orca#25204
• Admit one provider event's journal writes as one queued operation, decided when it runs stablyai/orca#25141
• Add a shared timeline assembler for structured agent chats (not wired yet) stablyai/orca#25064
• Translate ACP traffic into shared timeline events stablyai/orca#25090
• feat(native-chat): Grok as a structured chat over the Agent Client Protocol stablyai/orca#25225

Also track:

• [Feature]: Versioned structured-agent adapter contract for Chat UI stablyai/orca#10099 — generic adapter design
• Native chat for third-party agents: pi (pi-coding-agent) adapter / public session-host contract stablyai/orca#21558 — Pi
• [Feature]: OMP in structured native chat (via omp --mode rpc) stablyai/orca#22650 — OMP
• feat(native-chat): add OMP RPC chat adapter stablyai/orca#18618 — old OMP RPC adapter, reference only

Do not build compatibility wrappers around an upstream draft that has already been superseded. Re-evaluate at the start of each phase.

PR #51 status

PR #51 demonstrated useful provider behavior but is not the merge plan.

Treat it as a reference for:

• Pi-family protocol behavior;
• OMP edge cases;
• history semantics;
• dispatch reliability;
• test scenarios.

Do not transplant its generic Orca lifecycle/journal architecture unless #53 explicitly finds that current upstream still requires it.

Epic invariants

• Pi and OMP remain distinct durable providers backed by one shared Pi-family implementation.
• Native Chat does not route through orca-pi or plugin IPC.
• Generic Orca provider infrastructure is reused rather than duplicated.
• Provider request/result correlation is exact and fail-closed.
• OMP whole-session quiescence uses the modern provider contract, not terminal agent_end alone.
• Unknown delivery is never automatically resent.
• Restart settlement uses provider evidence.
• Existing Pi terminal behavior and OMP terminal/transcript Native Chat remain functional.
• Same-session structured↔TUI handoff is out of the initial scope.
• Claude/Codex semantics must not be weakened.

Superseded implementation series

The following issues describe the previous architecture and are closed as superseded:

• [SUPERSEDED] PIF-1 — Make Pi and OMP first-class durable structured providers in shared routing/identity #22–[SUPERSEDED] PIF-9 — Remove superseded Pi/OMP bridge paths and harden end-to-end structured integration #30

They remain useful historical detail, especially for requirements and tests, but are not execution dependencies.

Epic completion criteria

• Fork is synced to a current upstream Orca baseline.
• A concrete upstream generic provider contract has been selected and documented.
• A fake third provider proves that contract through the real host.
• Pi-family RPC transport is implemented without generic Orca duplication.
• OMP support/version policy is explicit and modern lifecycle semantics are tested.
• Pi and OMP register as first-class structured providers.
• Models/thinking/commands/interactions/cancel/restart work through generic Orca capability surfaces.
• Current-main end-to-end tests pass for Pi and OMP.
• Claude/Codex and existing terminal Native Chat regressions pass.
• Prototype-only generic infrastructure from [REFERENCE / SUPERSEDED] Pi-family structured Native Chat prototype #51 is absent where upstream supersedes it.
• Final architecture and upstream-contribution boundary are documented.

[44madfire]

Upstream dependency is now satisfied: can1357/oh-my-pi#12900 merged on 2026-09-23 at 6f2233877756b5553ce520756dd90315d2ff6ee3.

The shared Pi-family routing plan is therefore unblocked against canonical upstream OMP. No downstream OMP compatibility fork is required for the core history/tree/thinking RPC surface.

Implementation can proceed with #22 (durable Pi/OMP identity + routing) and #23 (shared Pi-family RPC transport) as the first workstreams, followed by the existing dependency chain #24–#30.

[44madfire]

PIF epic implementation complete — all nine children merged into the epic integration branch

All child issues #22–#30 are implemented, independently reviewed, and merged into
44madfire/epic-native-pi-family-structured-sessions-over-p (head 7f7e9f785).
Nothing was merged to main.

Issue	PR	Review outcome
#22 PIF-1 durable pi/omp identity + routing	#42	BLOCK (fail-closed capability) → fixed → PASS
#23 PIF-2 shared Pi-family RPC transport	#43	BLOCK (Pi-only settle predicate exposed) → fixed → PASS
#24 PIF-3 acquisition, exact resume, proven close	#45	PASS
#25 PIF-4 dispatch admission + history settlement	#47	BLOCK ×2 (settlement race, fabricated identity) → fixed → PASS
#26 PIF-5 streaming journal + backpressure	#46	PASS (+ CI gate fix)
#27 PIF-6 cancellation + exactly-once prompt answers	#49	PASS
#28 PIF-7 options, command-discovery dialect, compaction	#44	PASS
#29 PIF-8 history reconciliation + structured↔TUI continuity	#48	PASS
#30 PIF-9 end-to-end hardening + superseded-path inventory	#50	BLOCK (OMP phantom-turn wedge) → fixed → PASS

Verification on the merged epic branch

• tsc --noEmit -p config/tsconfig.node.json → exit 0.
• vitest run src/main/pi → 597 passed, 6 skipped, 5 failed. All five failures are the
  supportsCreate(LOCAL, ...) location-gate assertions and were verified to fail
  identically on the pristine pre-epic commit f4273161e: the
  @vscode/windows-process-tree native addon cannot be rebuilt in this environment, so
  the Windows process start-time proof is unavailable and the gate fails closed by
  design. They are environment artifacts, not regressions.
• git diff f4273161e..HEAD -- src/main/claude src/main/codex src/main/claude-accounts src/main/codex-accounts
  is empty: Claude/Codex are untouched.
• The router-level integration suite exercises the real
  router → single Pi-family adapter → production backend → scripted pi/omp children.

Epic invariants verified

• Orca remains the only durable structured-session/lifecycle authority.
• One Pi-family adapter owns both discriminants; no sibling OMP adapter exists and
  neither provider routes through external, plugin IPC, or orca-pi.
• The provider discriminant is part of root identity; equal sessionId across Pi and OMP
  never compares as the same root.
• Session files are opaque and same-provider only; cross-open is refused.
• Final settlement is dialect-specific through the provider flavor (Pi agent_settled;
  OMP terminal agent_end { isTerminal !== false }) and lives on the live session, not
  the transport.
• Delivery ambiguity stays unknown and is never auto-resent.

Notable findings from review

Three blockers were real production defects rather than review noise:

1. [SUPERSEDED] PIF-1 — Make Pi and OMP first-class durable structured providers in shared routing/identity #22 — widening the durable handle guard to accept omp also made capability gates
   answer "structured launch supported" for omp while the RPC create/attach path still
   refused it. That stranded a provisional chat as visibility-unknown with no terminal
   fallback, and let an omp attach spawn a pi --mode rpc child before the durable
   store rejected it. Fixed with an explicit creatability set separate from handle
   validity; later widened by [SUPERSEDED] PIF-3 — Implement Pi-family acquisition, exact resume, process identity, and proven close #24 only once a genuine acquisition path existed.
2. [SUPERSEDED] PIF-4 — Implement Pi-family dispatch admission and history-backed submission settlement #25 — dispatch correlation was armed after the prompt ack, so a boundary frame
   coalesced into the same stdout read was observed with nothing pending and never
   retried (reproduced as a 5/15 flaky test; now 15/15 after arming before the write,
   mirroring the existing Codex ordering). Separately, a failed pre-dispatch history read
   was indistinguishable from a genuinely fresh session, so settlement could adopt an
   unrelated entry id. Both re-verified closed by a second review.
3. [SUPERSEDED] PIF-9 — Remove superseded Pi/OMP bridge paths and harden end-to-end structured integration #30 — prompt_result { agentInvoked:false } (OMP local-only) never cleared the live
   turn, so a second prompt was refused with already-streaming and readResumeHistory
   threw PI_HISTORY_BUSY, permanently wedging the conversation and breaking the [SUPERSEDED] PIF-8 — Implement Pi-family provider-history reconciliation and structured↔TUI continuity #29
   TUI→structured rebuild. The regression guard was confirmed by reverting only the driver
   change and watching the strengthened test fail.

The changed-code quality gate caught real type-assertion violations on #25/#26 that
reviewers had graded non-blocking; because CI enforces that gate, both were fixed before
their PRs were submitted.

Retention decision on #30

Issue #30's inventory listed ten legacy paths for potential deletion. Review audited each
one independently and agreed with the implementer's retention of all ten: they are the
live direct-RPC implementation (transport, driver chain, translation), the adapter's own
dependency contract, the separately-gated SNC1.3 dev seam that packaged Orca never
installs, or ordinary TUI/status functionality still needed outside structured Native
Chat. No deletion was safe, so none was made — a verified finding rather than an omission.

Known follow-ups (recorded in PR bodies, not blocking)

A deterministic test pinning the arm-before-write ordering; stale-trigger fencing;
prompt_result id correlation; an end-to-end runtime late-settlement test; the double
image read on the dispatch path; payload bounding; and the fact that settlement remains
event-driven, so a single-turn session can retain a pending until the restart path
reconciles it. #30 additionally noted that its new suite is router-level, so #30's
"host-level" wording is satisfied by composing it with the existing restart-settlement and
handoff suites.

Not done

The optional real-binary smoke tests (pi --mode rpc / omp --mode rpc readiness and
clean close) were not run, since they require installed provider binaries and were
deliberately specified not to become a CI dependency.

[44madfire]

Completion-report review — BLOCK before epic close / main merge

The report is well supported on the integration branch itself: all nine child PRs are merged into 44madfire/epic-native-pi-family-structured-sessions-over-p (current merge head 7f7e9f785), the branch is 27 commits ahead / 0 behind main, and the child PR review history matches the report.

I found two current-upstream OMP compatibility blockers plus one unmet #30 acceptance requirement.

1. BLOCK — current OMP local-only prompt completion is not handled

The merged Orca branch still models OMP local-only completion as a later:

prompt_result { agentInvoked:false }

and PiRpcSessionDriver.deliverPiRecord() clears activeOp only when that record arrives.

But canonical OMP has moved on. In current OMP 18.4.5:

• an accepted prompt completes exactly once;
• if it completes locally, the prompt response itself carries data.agentInvoked:false;
• no later prompt_result is emitted for that prompt;
• otherwise a later prompt_result is emitted and correlated by the prompt command id.

OMP documents this explicitly in docs/rpc.md, and its own RpcClient returns immediately when response data.agentInvoked === false.

Orca currently discards that information: PiRpcConnectionCommands.prompt() returns Promise<void> and ignores the response payload. Against current OMP, a locally handled command can therefore leave activeOp armed forever — the same user-visible wedge #50 fixed for the older scripted behavior (follow-up dispatch rejected as already-streaming; resume history remains busy).

Required repair: carry the prompt acknowledgement/result metadata (and request id) through the Pi-family connection/driver, retire a local-only op directly from data.agentInvoked:false, and update the OMP scripted fixture/test so local-only completion has no later prompt_result.

This is also the right place to close the existing “prompt_result id correlation” follow-up: current upstream explicitly requires correlation on id.

2. BLOCK — OMP final-settlement predicate is stale

The epic currently treats:

OMP final settle = agent_end { isTerminal !== false }

Current OMP now explicitly separates:

• terminal agent_end = one agent run yielded;
• session_settled = the entire session is quiescent (no live/admitted run, queued message, or background work that can wake it again).

Current OMP also puts a sessionSettled boolean on correlated prompt_result.

That distinction matters to Orca: using terminal agent_end as whole-session settlement can clear activity/prompts, run history settlement, or allow handoff/new work while agent-owned background work can still wake OMP again.

Required repair: revise the OMP flavor/lifecycle contract to use current upstream session-quiescence semantics. Treat agent_end as a run/yield boundary, use correlated prompt_result for the prompt's outcome, and use session_settled (or equivalent proven state) for whole-session idle/finality. Update #20/#21's dialect table and the #24/#25/#26/#30 fixtures accordingly.

The original baseline at OMP PR stablyai#12900 was valid when designed, but #21 says OMP is supported at or after that baseline. The completed branch needs to match canonical OMP today, not only the older fixture.

3. MAJOR — #30's required host-level integration suite is still missing

#30 explicitly requires:

at least one host-level scripted integration suite that exercises the real Orca structured host/router/adapter stack

and says not to bypass the host by testing only the adapter.

The new pi-family-router-integration.test.ts starts at:

StructuredAgentSessionAdapterRouter
  → PiStructuredSessionAdapter
  → production backend
  → scripted child

It does not instantiate StructuredAgentSessionHost, the durable record store, lease/fence lifecycle, or real journal host path.

The completion report acknowledges this and argues that the router suite plus separate restart/handoff/record tests “compose” to satisfy the requirement. I checked those suites: pi-family-restart-settlement.test.ts and pi-family-tui-handoff.test.ts also do not instantiate StructuredAgentSessionHost.

That is useful layered coverage, but it is not the explicit host-level integration gate we wrote into #30.

Required repair: add one scripted Pi/OMP integration test that actually enters through StructuredAgentSessionHost and reaches the shared adapter/production backend/scripted child. It need not duplicate every #30 scenario, but it should prove the primary happy-path ownership/journal lifecycle through the host boundary.

Validation caveat (not a code blocker by itself)

There are no GitHub workflow runs on integration head 7f7e9f785. The reported 5 src/main/pi failures were reproduced on the pre-epic base and are credibly environment-specific, and Claude/Codex source directories are unchanged. Still, before merging the epic branch to main, the final integration PR should run the normal CI on a supported environment so the epic criterion “Claude/Codex regressions remain green” is demonstrated rather than inferred.

What looks good

I do not see a reason to reopen the earlier child-PR blockers that were fixed:

• durable Pi/OMP identity + same-adapter routing;
• fail-closed OMP creatability transition;
• transport-only shared RPC layer;
• arm-before-write dispatch correlation;
• fail-closed pre-dispatch history proof;
• exactly-once prompt-answer ownership;
• command-discovery dialect;
• structural Pi/OMP history normalization;
• exact same-provider session-file handoff;
• proven-exit discipline;
• [SUPERSEDED] PIF-9 — Remove superseded Pi/OMP bridge paths and harden end-to-end structured integration #30's retention inventory.

The remaining items in the report such as double image reads, stale-trigger attempts (where settlement proof itself is generation-fenced), dead-code hygiene, and optional real-binary smoke tests can remain follow-ups once the three items above are resolved.

Review outcome: implementation is close, but I would not mark #21 complete or open the final main-merge PR as ready until the current OMP prompt/session-settlement contract is updated and the required host-level integration test exists.

[44madfire]

Re-review status — blockers still open

I re-checked the implementation after the previous completion-report review.

There is currently no new code delta on the epic integration branch:

• branch: 44madfire/epic-native-pi-family-structured-sessions-over-p
• head remains: 7f7e9f7851939f89ebebfdf1970d9c742e6e42b2
• comparison against the previously reviewed head: identical (0 commits / 0 files changed)
• no newer PR targeting the epic branch was found after PIF-9: Pi-family end-to-end hardening and superseded-path inventory #50

I also re-checked canonical OMP upstream. Current OMP remains on the newer prompt/session lifecycle contract described in the prior review:

1. local-only prompts may complete in the immediate prompt response with data.agentInvoked:false and no later prompt_result;
2. later prompt_result frames are correlated by prompt request id;
3. terminal agent_end is a run/yield boundary, while session_settled represents whole-session quiescence.

The epic branch still:

• discards prompt response data in PiRpcConnectionCommands.prompt(): Promise<void>;
• clears OMP local-only activity from an uncorrelated later prompt_result;
• treats terminal OMP agent_end as final settlement;
• lacks the explicit StructuredAgentSessionHost → router → adapter → production backend → scripted child integration test required by [SUPERSEDED] PIF-9 — Remove superseded Pi/OMP bridge paths and harden end-to-end structured integration #30.

So the previous review outcome remains unchanged: BLOCK before epic close / merge to main.

Once a repair branch/PR is pushed, the next review should focus narrowly on:

• prompt response metadata + request-id correlation;
• OMP prompt_result / session_settled handling;
• the host-level integration test;
• final CI on the resulting epic head.