Skip to content

test: stabilize Windows Codex integration fixtures - #3808

Merged
lidge-jun merged 9 commits into
devfrom
codex/track2-windows-fixture-cache
Sep 6, 2026
Merged

test: stabilize Windows Codex integration fixtures#3808
lidge-jun merged 9 commits into
devfrom
codex/track2-windows-fixture-cache

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Summary

Stabilize three Windows Codex integration fixtures while retaining their real processes, ownership checks and behavioral assertions.

  • Seed composed-acceptance children with a fixture-owned copy of the prepared PowerShell module cache. Same-VM original/empty/copied/restored controls isolated the cold-cache delay; cleanup also covers cache setup failure before fixture registration.
  • Give the competing-OFF test a separate Windows preparation allocation. The previous final run consumed 52.7 seconds before discovery, leaving too little of its 85-second budget to start the unchanged 40-second flip. The child/test budgets become 125/130 seconds only on Windows; the flip and reap limits stay unchanged.
  • Bound the unreadable-config shim fixture with a Windows 45-second child deadline and 5-second cleanup allowance, preserving POSIX limits and existing install/advisory assertions. A delayed CLI passes under the new limit, while rethrowing the unreadable-config error still fails the original assertion.
  • A remote 52-second total-preparation control rejects the old allocation and passes the new one. An OFF-persistence mutation still fails the original result assertion after the real flip completes. Diagnostic instrumentation and workflows are excluded.

The top integration head also includes current dev eff908e0f. The only merge conflict was adjacent independent tests, both preserved and independently reviewed. No product policy or repository-workflow change is introduced by this support layer.

Verification

  • Independent source reviews passed, including the merge of dev eff908e0f. Claude admission/persistence and diagnostic workflows received explicit independent security review. The Windows fixture changes also passed independent source review.
  • Local tests, typecheck, installs and builds were not run, per maintainer instruction. Branches were pushed with --no-verify.
  • Final integration CI passed all 25 jobs at exact head 4a759716a940595b999805be40d1514c100d63cf, containing all five layers plus dev eff908e0f. All expected jobs completed successfully.
  • Dev advanced after final CI started to f89b815090020f52089801b50f569a777bdfdc0a. Independent source interaction review passed. That track's 25-job CI verifies its own source; its initial watchdog failure remains recorded there.
  • The conflict-free combined source bf5ce1c3a0ada78f8e506334a82c64b525b859da additionally passed remote Bun 1.4.0 typecheck, 541 tests across 11 relevant files, and the 425-page documentation build. This supplements the full cross-platform run at 4a759716a; it is a focused combined-tree check, not a second complete platform matrix.
  • Earlier final CI failures were investigated before changing fixtures: cache-policy controls isolated composed-fixture cold-cache costs; preparation-budget and semantic-mutation controls verified the competing-OFF allocation and retained assertion sensitivity. Shim advisory controls additionally verified an owned process deadline while retaining failure on a broken advisory collector. These diagnostics are not full integration proof.
  • Automatic PR matrix runs are suppressed by commit skip directives. Lower diagnostics were used only after final-matrix failure. No skipped lower check is described as passed; diagnostic-only workflows are excluded from the delivery stack.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Manual review chain (integrate bottom-up)

Layer PR Review base
1 #3803 dev
2 #3804 codex/track2-protocol-foundation
3 #3805 codex/track2-canonical-efforts
4 #3806 codex/track2-refusal
5 #3808 codex/track2-claude-compatibility

Maintainer integration decision: @lidge-jun explicitly integrates this PR into dev under MAINTAINERS.md at head 9dc8988d79a29741de8a8b89bd52790c132d751e, using the final-head-first verification above and the authorized admin merge. This is maintainer integration, not a self-approval. Lower skipped checks are not passing evidence. Actor, base, head and review state are refreshed before merging.

Final integration path: #3803, #3804 and #3805 landed first. #3808 integrates the remaining Claude head and fixture repairs together. Its final history-only integration head is 9dc8988d79a29741de8a8b89bd52790c132d751e; tree eb4473dfe9e6b95072e3eb31901df635c73dd0eb exactly equals the remotely checked combined source. The full 25-job matrix is attributed to 4a759716a, and combined-tree typecheck/541 tests/docs to bf5ce1c3a; no new full-platform run is claimed for the history-only head.

Landing confirmed: included in dev 5759d9ea2f1e7281cdc01eb9628f2e0a123fb59c. All five PR heads are ancestors of fetched dev. Final tree eb4473dfe9e6b95072e3eb31901df635c73dd0eb exactly equals the remotely checked combined source. #3806 was integrated with #3808; GitHub records both as merged.

Summary by CodeRabbit

  • New Features

    • Added optional Claude Messages compatibility modes: shadow and enforce.
    • Enforce mode rejects requests using unsupported Claude protocol features with clear errors.
    • Shadow mode allows requests while recording detected compatibility details in request and usage logs.
    • Compatibility checks cover translated requests across supported adapters while preserving native passthrough behavior.
  • Documentation

    • Added configuration reference covering compatibility settings, supported behavior, and limitations.
  • Tests

    • Added coverage for compatibility detection, enforcement, shadow logging, persistence, and invalid configurations.

t and others added 4 commits September 7, 2026 03:44
…ity policy

Reimplements PR #3730 from source head 18d6474 under the accepted track2 roadmap. Apply one policy before translation, retain only closed diagnostic codes, reject invalid configured modes, and persist shadow evidence through hydration.

Validation: NOT RUN by explicit user instruction (no local tests, typecheck, build, install, prepush or live proxy). Parent owns test-layout registration, exact-head remote CI and independent implementation security review.

Co-authored-by: SB Yoon <44089734+yansigit@users.noreply.github.com>
Co-authored-by: Yumi <automation@sbyoon.com>
@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: e3f70bb9-36a6-4bb3-9d00-22ffd34564e9

📥 Commits

Reviewing files that changed from the base of the PR and between eccbdc4 and 9dc8988.

📒 Files selected for processing (18)
  • devlog/_plan/260907_track2_protocol/070_windows_fixture_cache.md
  • devlog/_plan/260907_track2_protocol/080_windows_sync_preparation.md
  • devlog/_plan/260907_track2_protocol/090_windows_shim_budget.md
  • docs-site/src/content/docs/reference/configuration/server.md
  • scripts/test-layout/layout.json
  • src/claude/compatibility.ts
  • src/server/claude-messages.ts
  • src/server/request-log.ts
  • src/types/config.ts
  • src/usage/log.ts
  • tests/claude-integration/claude-compatibility.test.ts
  • tests/claude-integration/claude-messages-endpoint.test.ts
  • tests/codex-integration/codex-composed-acceptance.test.ts
  • tests/codex-integration/codex-shim-readiness.test.ts
  • tests/codex-integration/codex-sync-api.test.ts
  • tests/fixtures/test-layout-expected.json
  • tests/usage/request-log.test.ts
  • tests/usage/usage-log.test.ts

📝 Walkthrough

Walkthrough

This change adds configurable Claude Messages compatibility detection, enforcement, shadow logging, and persistence. It also isolates Windows fixture caches and adjusts Windows-only deadlines for composed, shim, and competing-OFF integration tests.

Changes

Claude compatibility admission

Layer / File(s) Summary
Compatibility policy and feature detection
src/claude/compatibility.ts, src/types/config.ts
Defines "shadow" and "enforce" modes, detects unsupported protocol features, normalizes feature codes, and generates bounded reasons.
Messages admission gate
src/server/claude-messages.ts
Validates configured compatibility mode after native passthrough and before translation. Enforce mode returns 400; invalid configuration returns 503; shadow mode records evidence.
Compatibility evidence persistence
src/usage/log.ts, src/server/request-log.ts
Normalizes compatibility evidence before storage, request-log retention, API hydration, and usage.jsonl persistence.
Compatibility coverage and documentation
tests/claude-integration/*, tests/usage/*, docs-site/src/content/docs/reference/configuration/server.md, scripts/test-layout/layout.json, tests/fixtures/test-layout-expected.json
Covers feature detection, admission precedence, sanitization, persistence, hydration, and documented configuration behavior.

Windows integration-test reliability

Layer / File(s) Summary
Windows fixture cache ownership
tests/codex-integration/codex-composed-acceptance.test.ts, devlog/_plan/260907_track2_protocol/070_windows_fixture_cache.md
Copies the PowerShell cache into fixture-owned storage, passes the owned path to children, and removes the temporary root on construction failure.
Windows competing-OFF preparation budget
tests/codex-integration/codex-sync-api.test.ts, devlog/_plan/260907_track2_protocol/080_windows_sync_preparation.md
Adds one Windows boot interval for preparation while retaining the existing flip and reap budgets.
Windows shim child deadline
tests/codex-integration/codex-shim-readiness.test.ts, devlog/_plan/260907_track2_protocol/090_windows_shim_budget.md
Uses the Windows spawn budget for the child and adds a five-second outer cleanup allowance while preserving the non-Windows timeout.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant ClaudeMessages
  participant CompatibilityPolicy
  participant RequestLog
  participant UsageLog
  Client->>ClaudeMessages: translated /v1/messages request
  ClaudeMessages->>CompatibilityPolicy: analyze body and mode
  CompatibilityPolicy-->>ClaudeMessages: allow, shadow, or reject
  ClaudeMessages->>RequestLog: record shadow evidence
  RequestLog->>UsageLog: persist normalized evidence
Loading

Suggested reviewers: invalid-email-address

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/track2-windows-fixture-cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Deterministic PR hygiene checks passed.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-06T19:33:49.965544Z 0f8936b Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0f8936b1f6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/codex-integration/codex-composed-acceptance.test.ts Outdated
@lidge-jun

Copy link
Copy Markdown
Owner Author

리뷰 · 우선순위 50 / 80

이 PR은 Track2 제품 스택(#3803#3804#3805#3806) 위에 얹는 Windows 전용 테스트 지원 층이다. 베이스는 codex/track2-claude-compatibility(#3806), 헤드는 codex/track2-windows-fixture-cache다. 제품 코드는 건드리지 않고, tests/codex-integration/codex-composed-acceptance.test.ts의 Fixture만 고친다. Windows CI에서 composed-toggle 케이스가 HTTP deadline(약 30초) 안에 PowerShell 명령 캐시를 매번 새로 만들다가 타임아웃 나는 문제를, 픽스처가 소유한 준비된 모듈 분석 캐시 복사본으로 줄인다.

지금 dev(HEAD eff908e0f, 2.45.0, Track1 #3791–#3794까지 포함)의 같은 파일을 보면 Fixture 생성자는 홈·lock·config만 만들고, env()는 HOME/USERPROFILE/CODEX_HOME 등 whitelist만 넘긴다. PSModuleAnalysisCachePathpowerShellCacheEnv도 없다. 그래서 이 델타는 dev 단독 cherry-pick도 가능하지만, 스택 본문·플랜 문서가 말하는 역할은 “최종 통합 CI의 Windows shard를 초록으로 만들기 위한 지원 층”이다. 제품 의미가 아니라 CI 게이트 의미라서 우선순위는 중간(50)이다. #3803/#3805 채팅 의미나 #3806 Claude 게이트보다 낮다. 다만 스택을 Windows에서 랜딩하려면 이 층이 사실상 관문이다.

변경의 핵심은 win32에서만 this.root/module-analysis-cache 경로를 만들고, 부모 환경의 PSModuleAnalysisCachePath(대소문자 무시)가 절대경로 파일이면 copyFileSync로 그 소유 경로에 복사한 뒤, env() 반환값 맨 앞에 그 변수를 섞는 것이다. 심볼릭 링크는 건너뛰고, 복사 후 크기가 다르면 즉시 지운다. ENOENT/ESTALE만 삼키고 다른 오류는 던진다. 캐시는 Codex manifest 밖(this.root)에 있으므로 기존 cleanup()removeTreeWithRetry(this.root)에 같이 지워진다. 타임아웃·assertion·SID/서비스 홈·coordinator·provider barrier는 그대로다.

같은 VM에서 캐시 정책만 바꾼 비교(액션 34053472964)가 설득력 있다. 원본/복원 컨트롤은 OFF 요청이 약 25–26초이고, 빈 소유 경로만으로는 비슷하게 느리며, 준비된 복사본만 3.3초대로 떨어진다. “빈 디렉터리만 주면 된다”는 가설을 스스로 깨 둔 점이 좋다. 플랜 문서 devlog/_plan/260907_track2_protocol/070_windows_fixture_cache.md도 그 표를 남긴다. types/config 대분리와 무관하고, close-don't-rebase 대상이 아니다.

운영상 주의할 점은 두 가지다. 첫째, 이 PR만 dev에 먼저 넣어도 해롭지는 않지만 Track2 랜딩 순서(#3803→…→#3806#3808)를 지키면 최종 통합 헤드와 문서가 맞는다. 둘째, 본문이 말한 최종 통합 CI(런 34054412656, 헤드 0f8936b1)가 아직 진행/조건부 merge라면, 초록 확인 전에는 합치지 않는 편이 맞다. 로컬 스위트/typecheck는 메인테이너 지시로 안 돌렸다고 명시되어 있다.

라인 tests/codex-integration/codex-composed-acceptance.test.ts Fixture 생성자 - win32에서만 소유 캐시 경로를 시드한다. 부모 캐시를 자식에게 직접 넘기지 않아 부모 파일을 갱신하지 않는다
라인 동일 파일 env() - powerShellCacheEnv를 whitelist 맨 앞에 펼친다. 다른 홈/프록시 차단 정책은 유지된다
경로 copyFileSync + size 검사 - 복사 실패·크기 불일치 시 캐시 파일을 지워 “반쯤 준비된 캐시”로 착각하지 않게 한다
경로 ENOENT/ESTALE만 허용 - 권한·디스크 오류는 숨기지 않는다. CI에서 간헐 ESTALE이 나오면 재시도 여부는 후속 판단이다
경로 devlog/_plan/260907_track2_protocol/070_windows_fixture_cache.md - 증거 표와 범위(제품/워크플로 변경 없음)가 코드와 맞다

메인테이너의 판단이 필요한 지점

  • Track2를 #3808까지 한 번에 올릴지, Windows 지원 층만 먼저 dev에 넣을지
  • 최종 통합 CI 34054412656이 완전 초록일 때만 merge할지(본문 조건부 merge와 동일)
  • 부모에 PSModuleAnalysisCachePath가 아예 없는 러너에서 “빈 소유 경로”로 둘지, 미리 워밍업하는 헬퍼를 둘지(지금은 빈 경로로 두고 증거상 개선은 작음)
  • 이 픽스처 패턴을 다른 Windows composed/integration 테스트에도 공통 유틸로 뺄지

너의 추천
최종 통합 CI가 초록이면 Track2 제품 층(#3803#3806) 다음에 이 PR을 붙여 merge한다. 제품 회귀가 아니라 Windows composed deadline을 지키는 지원 층이라, 스택 랜딩을 막는 이유가 되지 않게 하는 편이 맞다. 라벨은 바꾸지 않는다. types/config 대분리로 닫을 필요는 없다.

이 댓글은 grok-bot이 작성했습니다

@lidge-jun lidge-jun changed the title test(windows): retain owned module cache in composed fixtures test: stabilize Windows Codex integration fixtures Sep 6, 2026
@lidge-jun
lidge-jun changed the base branch from codex/track2-claude-compatibility to dev September 6, 2026 21:00
@lidge-jun
lidge-jun merged commit 5759d9e into dev Sep 6, 2026
8 of 9 checks passed
@lidge-jun
lidge-jun deleted the codex/track2-windows-fixture-cache branch September 6, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant