Paginated Codex history stops projecting after provider metadata relabeling

[luvs01]

Client or integration

Codex App

Area

Other

Summary

OpenCodex provider-history relabeling appends a cloned session_meta record with its old top-level ordinal (observed: 0) to a Codex paginated rollout. The native history projector stops at that record while the raw rollout continues receiving messages. Two long-running conversations became stale in the native UI and thread/read despite intact later messages. A second symptom was Model provider opencodex not found after the provider table was removed while a thread still referenced it.

Reproduction

1. Use a Codex App version with paginated history and a rollout containing monotonically increasing top-level ordinals.
2. Exercise an OpenCodex provider-history transition, such as authless Desktop startup or restoration.
3. src/codex/history-provider.ts:updateSessionMeta copies the latest session_meta, changes provider/source/timestamp, and calls appendRolloutLine without allocating a new ordinal.
4. Native projection refuses the appended ordinal 0. The app continues appending raw records, but native history queries stay at the old projection boundary.

Observed boundary A: next ordinal 40625, actual 0. Boundary B: next ordinal 31617, actual 0. The offending records are same-thread session_meta with model_provider=opencodex and history_mode=paginated. Service logs independently record authless history relabeling and restoration during the incident.

With Codex completely closed, rewriting only ordinal digits in each unprojected suffix restored native history reads. Message text, IDs, timestamps, and earlier bytes were preserved; the native projector subsequently advanced beyond both boundaries. This is incident recovery evidence, not a recommendation to rewrite live rollouts.

Do not fix this by merely reading the last ordinal and appending N+1: the native writer may concurrently allocate the same ordinal or retain a stale cursor. The durable solution needs native-writer ownership, or no external metadata mutation for paginated history. A transition must also retain the provider definition while any existing thread depends on it.

Version

OpenCodex 2.51.0; source still contains the append path at dev 7a0513c. App Codex executable 0.153.4; OpenCodex's configured CLI reports 0.146.0.

Operating system

Windows 11, kernel build 26100, x64

Provider and model

Provider-history transition between openai and opencodex; model independent.

Logs or error output

failed to project durable rollout: thread history projection expected ordinal 40625, got 0
failed to load configuration: Model provider `opencodex` not found
Injected opencodex as default provider into Codex config (authless Desktop mode: requires_openai_auth = false).
Codex resume history: 17 thread(s) made visible for opencodex; originals backed up for restore.
history-migration: restored original provider metadata for 17 manifest-backed thread(s).

Screenshots and supporting files

No private conversation payloads attached. Minimal regression fixtures can use one session_meta ordinal 0 followed by event ordinal 1. Both problematic suffixes parsed cleanly and contained no compacted/ocx1 records, so direct compaction-content loss was not observed.

Redacted configuration

{"codexDesktopAuthless": true}

The incident also involved subsequent removal of codexDesktopAuthless/codexClientCompaction. The deletion provenance does not identify every historical toggle. Compaction-only mode in current inject.ts skips history relabeling; authless mode does not.

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, and personal data.

[Ingwannu]

Confirmed the unsafe external-write path in dev 7a0513c: updateSessionMeta copies the existing record and changes payload/timestamp, then appendRolloutLine writes the serialized record without handling an existing ordinal. O_APPEND only protects the file append; it does not coordinate Codex's ordinal/projection ownership.

The reported projector incident is reporter evidence; I have not independently reproduced it against a live Codex installation. The source defect is sufficient to keep this open. Draft #4313 already targets it, so I will not open a duplicate fix.

Two completion boundaries must remain explicit: refusing paginated history writes must also prevent settings transitions from removing a provider definition still referenced by those threads; legacy-to-paginated migration must not race an external write. Preserve rollout bytes, database state and restore provenance on refusal. Do not repair this with an external last-ordinal-plus-one append or by rewriting a running session.

@lidge-jun this deserves priority because the failure affects history visibility. #4313 is a partial defensive Draft, not a deployed fix; this issue should remain open until the complete transition is verified.

[luvs01]

Author follow-up: the defensive fix is now at be08fe0 in #4313.

The implementation refuses affected rows in any store whose schema supports history_mode, including rows still marked legacy, rather than racing native migration. Apply, sync/async restore, inline journal restoration and direct config removal preserve configuration and provenance on refusal. No-transition exits for external providers and desired/foreign state are kept ahead of that guard.

For remaining genuinely legacy stores, append validation and writes use the same existing descriptor, with current-path identity checked before/after append; no O_CREAT is used. Integrity errors are no longer swallowed. Deterministic replacement and late-conversion tests verify rollback and preservation. Commit-boundary refusals are returned as structured results after compensation.

CodeRabbit's four findings and Codex's three findings have been addressed and replied to. History-provider tests pass 83/83; commit-boundary and external-provider subprocess regressions pass; TypeScript, structure checks and public docs build (425 pages) pass. English plus all seven translated integration guides document the refusal and safe recovery boundary.

The PR remains Draft while full-head validation and incremental review complete. The first broad run used the wrong local Bun setup and encountered long-path ACL failures; the bundled 1.4.2 runtime and a short junction with --preserve-symlinks corrected the reproduced environment failures without disabling ACL protection. The second full run subsequently ended at the 900-second watchdog (exit 124) with failing cases; it did not pass. No installed-runtime deployment or issue closure is claimed.

Latest author follow-up: head 676b97f adds post-artifact migration checks with preimage compensation, aborts every restore path after config failure, rolls back the coordinated remove transition, and binds first-line patches to the validated file identity. The two affected test suites passed 151/151, followed by 3/3 strengthened restore cases including durable generation/transaction rollback. TypeScript and structure checks passed. The last commit changes only a test comment. The two new inline findings are answered/resolved and the outside-diff finding is addressed; another review was requested. This remains Draft, undeployed, and not a full-suite pass. Keep this issue open.

[lidge-jun]

리뷰 · 우선순위 74 / 80

설명

이 이슈는 Codex App의 페이지 분할(paginated) 히스토리에 OpenCodex가 프로바이더 메타만 다시 붙일 때, 예전 session_meta에 있던 최상위 ordinal(관측값 0)을 그대로 복제해 append하면서 네이티브 프로젝터가 그 지점에서 멈추는 사고다. 원본 롤아웃 뒤에는 메시지가 계속 쌓이는데, 네이티브 history/thread 읽기는 경계에서 멈춰 UI와 thread/read가 오래된 상태로 보인다. 보고된 경계는 next ordinal 40625·31617인데 실제 append는 0이었고, 같은 스레드 session_meta에 model_provider=opencodex·history_mode=paginated가 찍혀 있었다. 서비스 로그에는 authless Desktop 시작·히스토리 복원 경로의 relabel이 같이 남아 있다. 부수 증상으로 스레드가 아직 opencodex를 가리키는데 프로바이더 테이블에서 정의를 지워 Model provider opencodex not found가 났다.

지금 dev HEAD 7a0513c2f(#4292 warm-key/quota 풀 배선 직후)의 src/codex/history-provider.ts updateSessionMeta는 id로 최신 session_meta를 찾아 provider/source·timestamp만 바꾼 뒤 appendRolloutLine으로 직렬화한다. 기존 레코드에 ordinal이 있어도 새 순번을 할당하지 않고, O_APPEND는 파일 끝에 붙이는 것만 보장할 뿐 Codex가 소유하는 ordinal·프로젝션 커서와는 맞춰 주지 않는다. Ingwannu 코멘트도 같은 HEAD에서 그 unsafe 외부 write 경로를 확인했고, 라이브 Codex 재현은 없어도 소스 결함만으로 이슈를 열어 두자고 했다. Draft #4313이 방어적 거절(history_paginated_requires_native_writer)로 막는 쪽을 이미 겨냥하고 있다.

이 버그는 계정 풀/#4292 열차와 파일이 겹치지 않지만, 대화가 안 보이는 실사용 장애라 우선순위가 높다. 고칠 때 이슈 본문이 못 박은 대로 외부에서 last-ordinal+1을 읽어 붙이는 임시방편은 안 된다. 네이티브 writer가 동시에 같은 순번을 쓰거나 커서가 lagged일 수 있다. 페이지 분할 기록은 네이티브 writer 소유로 두거나, 외부 메타 변경을 아예 하지 않는 쪽이 맞다. 전환 중에도 스레드가 참조 중인 프로바이더 정의는 남겨야 한다. types.ts/config.ts 분리 캠페인과는 무관하다. compaction-only inject는 히스토리 relabel을 건너뛰지만 authless는 건너뛰지 않아, 같은 설치에 paginated 스레드가 있으면 그 경로가 사고를 만든다.

라인 - 이게 무슨 문제다

src/codex/history-provider.ts updateSessionMeta (대략 1017–1141) - session_meta를 복제해 append할 때 ordinal을 갱신하지 않는다. paginated 롤아웃에서는 프로젝터가 expected ordinal ≠ 0에서 멈춘다.

appendRolloutLine - O_APPEND만 쓰고 Codex ordinal/프로젝션 ownership과 조율하지 않는다. 파일 무결성과 프로젝션 무결성은 별개다.

src/codex/inject.ts authless / provider-table 전환 경로 - compaction-only는 history relabel을 건너뛰지만 authless는 건너뛰지 않아 paginated 스레드가 있으면 이 경로가 사고를 만든다.

프로바이더 정의 삭제 vs 스레드 참조 - history relabel 거절·복원과 설정에서 opencodex 행 제거가 한 묶음으로 묶이지 않으면 Model provider not found 부수 장애가 남는다.

메인테이너의 판단이 필요한 지점

• #4313 Draft의 “외부 write 전면 거절”만으로 충분한지, 아니면 네이티브 writer 협조 API가 나올 때까지 authless/relabel 자체를 paginated 설치에서 막는 제품 결정을 할지
• 이미 깨진 롤아웃의 현장 복구(ordinal 수동 수정)를 공식 가이드/도구로 줄지, 닫힌 대화·백업 전제 수동 절차로만 둘지
• 이 이슈를 #4313 완전 검증 전까지 열어 둘지(Ingwannu 권고와 동일)

너의 추천

이슈는 열어 두고 #4313이 Draft를 벗어나 동시 migration·shutdown/restore·프로바이더 정의 보존까지 검증될 때까지 닫지 않는 걸 추천한다. 외부 N+1 append나 실행 중 롤아웃 재작성 PR은 받지 말 것. 우선순위는 history 가시성 장애라 풀 열차보다 위에 둬도 된다. 중복 이슈 없음. close-don't-rebase 대상 아님.

이 댓글은 grok-bot이 작성했습니다

[luvs01]

Author maintenance update: the defensive change from #4313 was integrated through merged #4342. Current dev includes the pending-manifest preflight and readable-preimage compensation amendments, so those changes do not need a second implementation.

The remaining restore-time migration finding is distinct: restoreCodexConfigInlineImpl() can pass its initial history preflight, then restoreJournalState() or removeCodexConfig() can remove provider artifacts before a later history operation notices newly paginated history and refuses. The next focused regression should introduce that format change inside the restore write interval, then require config/profile/journal bytes and the coordinated remove transition to remain preserved when history refuses.

Keeping this issue open for that residual and the previously stated native-writer acceptance. Prior local full-suite logs predate the final containment amendments and timed out; they are not evidence for the integrated head. The carry's hosted run 34674692660 is queued and no installed service or live conversation file was changed during this handoff.

[luvs01]

Author follow-up: Draft #4380 addresses the remaining restore-interval migration case described above. It adds a postflight history check after successful journal or fallback restoration, before reporting success. A migration detected during restoration enters the existing preimage compensation and coordinated rollback paths.

The new deterministic cases exercise actual restore writes; six cases failed before the fix. Focused migration and legacy controls now pass. The PR also aligns existing fixture paths, import-time spies, and compensation expectations with the current contracts. Full cross-platform CI and review remain pending.

Please keep this issue open. This is a defensive restore-boundary fix, not a native writer lock or a general repair for previously damaged history. It does not establish safety against a migration occurring after the final check.

[luvs01]

Additional isolated version comparison: installed 2.51.0 can strand a newly created compaction-only task on OFF, whereas current dev refuses that transition and preserves resumability. This is separate from the restore-interval change in #4380.

Using installed OpenCodex 2.51.0 and Codex app-server 0.154.0-alpha.6.2, with synthetic homes, synthetic authentication and a loopback mock Responses provider:

1. Enable client-side compaction while leaving authless disabled.
2. Create a new task. Its provider is opencodex.
3. Compact it, restart app-server, and resume the exact task. The persisted plaintext summary is sent in the next request; no ocx1: payload is present.
4. Disable client-side compaction. Injection reports success, returns new tasks to the built-in openai provider, and removes the opencodex table.
5. Restart and resume the exact task: failed to load configuration: Model provider opencodex not found.

Running the same actual-CLI fixture against dev d42a136 gives a different and safer result: OFF returns success: false with history_paginated_requires_native_writer, the provider table remains, and the exact task still resumes successfully. The defensive guard prevents the installed-version failure in this paginated case. It does not complete the requested OFF transition. Static inspection of the remaining removal code alone would miss this distinction.

For the failed 2.51.0 fixture, a process-local authenticated, non-default compatibility alias restored the exact task and summary delivery while leaving the OFF configuration bytes unchanged; a fresh task still selected openai. This validates a recovery direction, not a deployed fix. Safely enabling the currently refused OFF transition would require ownership and lifetime rules: preserve user-owned provider settings, honor configured sqlite_home, survive reinjection, and avoid deleting an alias solely because a reference query currently returns zero while a native writer may still cache the old provider. No existing history or provider tags need to be rewritten for that approach. This is mock-provider protocol validation, not a hosted-provider compatibility test. #4380 does not claim to implement that transition policy.

[lidge-jun]

Status after the 60-plus contributor backlog pass, which landed 12 lanes on dev. This issue is not closed by it, and the reason is specific rather than a blanket deferral.

#4380 compensated native restore when history migrates during writes, #4342 contained paginated metadata writes and preserved restore artifacts, and #4411 refreshes the catalog when paginated history refuses injection. All three reference this issue and none of them claims it.

The unresolved part is the one #4342 named explicitly: it is defensive containment, not native paginated-writer integration. Until a writer exists that projects paginated history after provider metadata relabeling, the projection stop you reported can recur.

[lidge-jun]

The reported projection stop is fixed on dev by 44027ae.

src/codex/history-provider.ts now inspects a newest-record window rather than trusting the head, and refuses in-place paginated tails, so history keeps projecting after provider metadata relabeling instead of stopping at ordinal 0.

This issue reported a second symptom alongside it — Model provider opencodex not found, with the provider definition not surviving the relabel — and that is a different defect which this change does not address. Rather than close this issue as if both were done, that residual is now tracked on its own at #4582.

Closing this one against the merge above; follow #4582 for the provider-definition half.