Skip to content

test: stabilize QueryGuard keepalive interruption test - #2323

Merged
jatmn merged 1 commit into
mainfrom
test/queryguard-keepalive-flake
Oct 22, 2026
Merged

jatmn merged 1 commit into
mainfrom
test/queryguard-keepalive-flake

Conversation

@jatmn

@jatmn jatmn commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Stabilize keepalives and parsed-but-ignored frames cannot reset QueryGuard in src/services/api/codexShim.interruption.test.ts.
    • Root cause: the test raced a real 25 ms stream interval against QueryGuard's 250 ms monotonic idle deadline and asserted that more than three frames were emitted first. Under CPU contention the interval callbacks fall behind while the monotonic deadline doesn't wait, so the guard fired after only two or three emissions, before every frame variant had arrived. It fails in fresh processes too, so state leaking between test files isn't required.
    • Fix: this test alone now runs on Bun fake timers. It advances the clock in 25 ms steps and yields a real event-loop turn between steps, so each frame is processed before the next one. It then asserts consumption is still pending at 249 ms and settles at exactly 250 ms (elapsedMs is 250). All four frame variants, the 25 ms interval, the 250/500/1,500 ms deadlines, and every existing assertion are unchanged. Cleanup restores real timers and settles the consumption promise before restoring trace state.
  • Why: the test failed intermittently in single-process whole-suite runs and under load, including on unmodified main.

Impact

  • user-facing impact: none; one test file changes.
  • developer/maintainer impact: the test no longer depends on scheduler latency, and it now checks the exact idle-deadline boundary, so any reset of the deadline caused by ignored input is caught directly. No tests skipped or removed, and no timeouts changed.

Testing

  • I ran the required local preflight.
  • exact commands and results:
    • Repro and fix, fresh processes with isolated HOME/TMPDIR: bun test --isolate --feature=UNATTENDED_RETRY --max-concurrency=1 ./src/services/api/codexShim.interruption.test.ts

      Method (32 file runs each) Before After
      Sequential 0/32 failed 0/32 failed
      4 parallel copies 0/32 failed 0/32 failed
      4 parallel copies + 24 busy-loop CPU workers 24/32 failed 0/32 failed
      4 parallel copies + 48 busy-loop CPU workers 29/32 failed 0/32 failed

      Every "before" failure was this test (two or three emissions, then the idle deadline fired). On the rebased head, I repeated the 24-worker batch: 32/32 passed.

    • Whole suite in one process: bun test --isolate --feature=UNATTENDED_RETRY --max-concurrency=1, run twice on the final head: 10,485 pass, 0 fail, 3 skip across 780 files each time.

    • bun run check: pass

    • bun run typecheck, bun run typecheck:type-tests: pass

    • node bin/openclaude --version, NODE_DISABLE_COMPILE_CACHE=1 node bin/openclaude --version: pass

    • bun run security:pr-scan -- --base FETCH_HEAD --head HEAD: pass

    • git diff --check: clean

  • focused tests: the changed file passed 10/10 consecutive fresh-process runs.
  • documented skipped checks, platform limitations, or verified pre-existing failures: web checks don't apply. The whole-suite runs and bun run check above ran on the previous main. After rebasing onto the current main, whose two new commits don't touch this file, I reran the changed file together with src/utils/ShellCommand.test.ts 10/10, plus bun run typecheck, bun run check:static, and the intent scan, all passing. In two earlier heavy-contention batches (before the final clean ones), a different, untouched test in the same file hit Bun's default per-test timeout once; it didn't recur in the final batches.

Notes

  • provider/model path tested: none; no provider behavior changes.
  • screenshots attached (if UI changed): n/a
  • follow-up work or known limitations: none. No docs change, since no contributor instructions describe this test.

Summary by CodeRabbit

  • Tests
    • Improved the reliability and timing consistency of interruption-related test coverage.
    • Added cleanup checks to help ensure tests leave no pending work behind.

@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: Twigpine/openclaude/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Essentials
  • Run ID: 59eeb12a-e43c-45a6-b01f-75c1e140007f
📥 Commits

Reviewing files that changed from the base of the PR and between 1c42a10 and 1cf50c4.

📒 Files selected for processing: 1
  • src/services/api/codexShim.interruption.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout: 17
  • GitHub Check: unit-tests (5)
  • GitHub Check: unit-tests (4)
  • GitHub Check: unit-tests (6)
  • GitHub Check: unit-tests (2)
  • GitHub Check: unit-tests (7)
  • GitHub Check: unit-tests (8)
  • GitHub Check: unit-tests (3)
  • GitHub Check: typecheck
  • GitHub Check: launcher-compatibility (24.11.x)
  • GitHub Check: launcher-node-floor
  • GitHub Check: smoke-and-tests
  • GitHub Check: dedicated-tests
  • GitHub Check: unit-tests (1)
  • GitHub Check: launcher-compatibility (22)
  • GitHub Check: CodeQL (javascript-typescript)
  • GitHub Check: CodeQL (actions)
  • GitHub Check: Analyze (javascript-typescript)
🧰 Additional context used
📓 Path-based instructions: 3
Review provider routing, model selection, env precedence, auth/token handling, OpenAI-compatible shims, retries, proxy behavior, and outbound HTTP behavior with high scrutiny.

⚙️ CodeRabbit configuration file

Files:

  • src/services/api/codexShim.interruption.test.ts
Review tests for meaningful coverage of the changed behavior, isolation of global/env/config state, async cleanup, fake timers, provider profile leaks, and Windows-compatible assumptions.

⚙️ CodeRabbit configuration file

Files:

  • src/services/api/codexShim.interruption.test.ts
Apply the OpenClaude maintainer review rubric from AGENTS.md.

⚙️ CodeRabbit configuration file

Files:

  • src/services/api/codexShim.interruption.test.ts
🔇 Additional comments: 1
src/services/api/codexShim.interruption.test.ts: 1

713-713: 🩺 Stability & Availability

No change needed.

jest.useFakeTimers() already runs before makeTimedStream(...), so the stream’s interval is created after fake timers are enabled. The cited ordering concern does not apply.

📝 Walkthrough

The interruption test now uses fake timers and real event-loop turns to verify that consumption remains pending at 249 ms and settles at 250 ms. Cleanup restores real timers and boundedly awaits remaining consumption.

Changes: Interruption timeout test

Layer / File(s) Summary
Deterministic timeout check
src/services/api/codexShim.interruption.test.ts
The test advances fake time in 25 ms steps and yields to real event-loop turns. It checks that consumption remains pending at 249 ms and settles at 250 ms. Cleanup restores real timers and boundedly awaits any remaining consumption.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Suggested reviewers: chioarub

Merge Risk: ⚪ Minimal · up to 1cf50

The change is limited to test timing and is ready to merge after normal checks.

Pre-merge checks: Passed 7
Check name Status Explanation
Title check Passed The title is concise, scoped to the test change, and accurately describes stabilizing the QueryGuard keepalive interruption test.
Description check Passed The description follows the required template and provides the change rationale, impact, testing commands and results, limitations, and notes. It is complete and directly related to the diff.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Risk Surface Disclosed Passed PASS — The PR changes only src/services/api/codexShim.interruption.test.ts. The diff adds fake-timer control and cleanup to a test; it does not change authentication, provider routing, permissions, …
No Hidden Policy Change Passed PASS — The PR changes only src/services/api/codexShim.interruption.test.ts. The diff adds fake-timer control, event-loop yielding, boundary assertions, and cleanup for the test. The stream frames, Q…
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1 files.

Full details: Risk Surface Disclosed

Explanation

PASS — The PR changes only src/services/api/codexShim.interruption.test.ts. The diff adds fake-timer control and cleanup to a test; it does not change authentication, provider routing, permissions, outbound network behavior, background execution, startup/config-home behavior, skills/plugins/MCP, CI permissions, or release scripts. No listed risk surface applies, so no blocker disclosure is required.

Full details: No Hidden Policy Change

Explanation

PASS — The PR changes only src/services/api/codexShim.interruption.test.ts. The diff adds fake-timer control, event-loop yielding, boundary assertions, and cleanup for the test. The stream frames, QueryGuard settings, and production imports remain test fixtures. No product behavior, trust model, routing default, telemetry/network behavior, or permission policy changes are present.

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@jatmn jatmn self-assigned this Oct 22, 2026

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— jatmn's PR Review agent

Test-only stabilization for QueryGuard keepalive idle timing in codexShim.interruption.test.ts. The diff is surgical (one file, no production paths). I verified fake-timer scoping, the 249/250 ms boundary, and that the test still guards against keepalive-driven idle resets. Clear the merge-readiness items before landing.

Merge readiness

  • Rebase onto current main. Live main is 75c73ea3811f359a8bad23b073bd4713846ea833 (docs: add CONTRIBUTING notes for CI caching and subprocess test timeouts (#2322)). This head branches from 1c42a10628df59cdaaf3a75a89978737f592721a, so it does not include that docs commit. Rebase (or merge main forward) before merge so the upgrade-plan stack stays current.
  • Mark the PR ready for review. GitHub still lists this PR as a draft (draft: true). Mark it ready when you want the merge gate to treat it as landable.
  • Merge-base decision: live-base-three-dot — reviewed range 1c42a10628df59cdaaf3a75a89978737f592721a..1cf50c499d2e2d231d1eb026c49c8cd65cb0abf5 (one commit). No scope creep in the diff.
  • CI at head 1cf50c499d2e2d231d1eb026c49c8cd65cb0abf5: changes, pr-intent-scan, smoke-and-tests, dedicated-tests, typecheck, all eight unit-tests shards, launcher compatibility jobs — success. CodeQL (actions), CodeQL (javascript-typescript), and Analyze (javascript-typescript) — success. Aggregate CodeQL check — success. web — skipped (expected for this path set). Commit status context CodeRabbit — success (“Review approved”). No PR-owned failing required checks on this head.

Test fix verification

  • Fake timer scope and cleanup: jest.useFakeTimers() runs only inside keepalives and parsed-but-ignored frames cannot reset QueryGuard. jest.useRealTimers() runs in the test finally before trace/env cleanup. The next test in the same file still uses real setInterval timing and passed in a full-file local run (14/14) and CI.
  • Real event-loop yield: nextTurn() wraps host setImmediate captured before fake timers, so each jest.advanceTimersByTime is followed by a real turn. That drains driveQueryEvents / stream parsing between ticks instead of racing the monotonic idle deadline against a backed-up interval queue.
  • 249 ms / 250 ms boundary: The loop advances through 249 ms of synthetic time with settled === false, then the final 1 ms triggers idle timeout with settled === true and outcome.timeout.elapsedMs === 250, matching idleTimeoutMs: 250 and QueryGuard’s performance.now()-based idle math.
  • Regression sensitivity: Production keepalives and ignored parsed frames must not call registerActivity (only yielded query events do). If idle fired too early, the 249 ms assertion would fail; if keepalives incorrectly extended activity, the 250 ms settled assertion or elapsedMs check would fail; existing assertions still require getEmissionCount() > 3, ignored/control frame trace counts, and idle reason with only message_start yielded.

Automated review audit

Source Thread / check ID Status on head Disposition
CodeRabbit Issue comment #6330167366 Summary / in-progress UI only; no review line comments Drift — nothing actionable to promote
CodeRabbit Commit status context CodeRabbit Review approved No open bot findings
CodeQL Check runs CodeQL, CodeQL (javascript-typescript), Analyze (javascript-typescript) on 1cf50c499d2e2d231d1eb026c49c8cd65cb0abf5 Completed success Drift — no PR-introduced alert; test-only diff
CodeQL Earlier aggregate neutral “configuration not found” note Superseded by successful javascript-typescript run on head Drift — environmental, not author fix

Findings

🟢 Thanks for the contribution. I do not see any actionable issues from my review.

@jatmn
jatmn marked this pull request as ready for review October 22, 2026 01:13
@jatmn
jatmn merged commit 7844534 into main Oct 22, 2026
22 checks passed
@jatmn
jatmn deleted the test/queryguard-keepalive-flake branch October 22, 2026 01:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant