[freeqaz]

Problem

A mutation sweep of the rules that landed on 10-16 evening found conditions their own tests did not check. Each mutant below survived the owning test files on main (product file restored after every mutant):

• Startup probe classification and retry (fix(runtime): retry a transient startup model probe once and name the held failure #2639, fix(runtime): hold provider 4xx and configuration failures at startup without a retry #2650), codex-native-startup:
  • a credential marker vetoing a transient one: dropping auth, forbidden, credential or the bare 401/403 alternative, or the veto's case-insensitivity, changed nothing;
  • the 4xx guard bounds: a provider 400 with its status (only the bare invalid_request_error body was covered) and the top of the range (499);
  • transient markers connection reset, connection closed and connection failed (the only failed case also said error sending request), and the markers' case-insensitivity;
  • a transient event in an attempt that then timed out (the transient check could move after the timeout check);
  • the retry budget boundary (now + 1000 < deadline vs <=);
  • the -attempt2 mark on a cause with no detail (a WRAPPER_ERROR after a retry would carry undefined-attempt2).
• Console (fix(console): open recovery links in view, hold the loading page, quiet deploy polls #2646):
  • a Refresh pressed during a deploy poll could be dropped instead of queued (console-agent-overview);
  • patchChildren keeping stale attributes, keeping children a shorter render no longer has, rewriting unchanged text, and patching a node in place when its role or role class changes (no direct test);
  • the loading page's 300 ms minimum when it appeared because the session check outlasted the startup delay (console-retained-revalidation covered only the reads path);
  • a superseded status read that fails (only a stale 200 was covered, console-agent-logs);
  • a modified (Ctrl, Cmd, Shift, Alt) or middle click on a recovery link being taken over by the page.
• Finding 1606: no test checked that a declined carried Channels draft is forgotten, so a later visit could try it again (channels/shared-ui.mjs; the surviving mutant from the console-agent-saves consolidation).

Change

Test-only. Rows and cases next to the existing tests, no new helpers or files:

• tests/conformance/codex-native-startup.test.mjs: 3 transient rows (each runs twice: recovers, then held as -attempt2), 7 held-at-once rows (400 with status, 499, one credential marker each, the API key in mixed case), and 3 scenarios (exact one-second budget, transient then timeout in one attempt, wrapper error after a retry).
• tests/browser/console-agent-overview.test.mjs: the deploy-poll test also checks the patched row loses muted, and that Refresh pressed during a held poll shows busy and reads after it; a new case drives patchChildren directly; the reduced-motion recovery-link test checks each modified click and a middle click are left to the browser.
• tests/browser/console-agent-logs.test.mjs: a third supersede case where the stale status read fails with 504.
• tests/browser/console-retained-revalidation.test.mjs: a fresh load whose session check is held past 400 ms keeps "Reading this Agent." for 299 ms after it appears and shows the Agent at 300 ms.
• tests/browser/console-agent-saves.test.mjs: after a declined carried Slack draft, leaving for the Agents list and opening the Agent again shows no note and no editor.

Verification

• Each new row or case fails on its mutant and passes on main: on main the owning files left alive 22 of 72 probe mutants, 1 of 7 cause-suffix mutants, 3 of 7 key-Namespace mutants (Describe the service key used for the refused occ request #2648), 0 of 4 generation-skip mutants (Use the returned generation when recovering a stale Configuration save #2651's recovery path), 15 of 33 console mutants and the 1606 mutant. On this head, with 5 more mutants from review (veto case, each link modifier): 8 of 73, 1, 3, 0, 6 of 37 and 0, all listed below.
• Full touched files on this head: codex-native-startup 121/121; on the first head console-agent-overview 28/28, console-agent-logs 38/38, console-retained-revalidation 56/56, console-agent-saves 36/36; CI's Console Browser lanes ran them on this head.
• Pinned at 2 CPUs (taskset, 2 cores) on 1516c27, 20/20 each: codex-native-startup (121 tests), and the 14 touched browser tests (overview 4, logs 3, retained-revalidation 5 with the four existing fresh-load cases, saves 2). After merging main (test(console): speed up the Agent overview tests #2660 moved the overview file to one shared browser; the new patch case now takes it), the four touched overview tests again 20/20 on a982961, and the patch, Refresh-queue and link-modifier cases still fail on their mutants.
• prettier --check and eslint --max-warnings 0 on the five files.
• Left unpinned:
  • Equivalent: unauthori[sz]ed (always matched by auth); >= 400 / < 600 on the 5xx check (a 4xx already returned, statuses are [45]xx); result.status === null with a kill signal; transient === true vs truthy; "-attempt" + attempt vs "-attempt2" (two attempts at most); includes vs endsWith for the cause suffix (no detail token contains -attempt2; the runtime appends it once, at the end); a duplicate reveal timer or an uncleared one after abort (the late call finds the stand-in gone and its steps check the page is current).
  • Not worth a pin: the case-insensitivity of the held-at-once exclusion (invalid_request_error|content_filter|certificate|Name does not resolve|…; the pinned Codex build prints these markers in fixed case); the ns_ pattern's 64-character cap and character class (a sanity cap; real IDs are ns_ plus a UUID); the timeout retry's own budget boundary (predates these fixes; the transient one is pinned).
  • Narrow, left as gaps: a superseded first status read scheduling a second poll (needs a refocus during the first read), a download alone marking logs as allowed (the automatic first read already does), and a status reread while the refusal is remembered.
• Not run here: anything needing PostgreSQL, docker or a cluster (none of these rules need one).

Internal review: approve with nits, all applied (every link modifier and the middle button, the credential veto's case, the real RUNTIME_LOGS_TIMEOUT shape for the stale failing read, comments on the synthetic rows).

Risks

Test-only. The browser additions use the existing paused-clock and held-request helpers; the new waits are bounded by the suite's test timeout. console-agent-saves changes stay clear of #2593's lines (git merge-tree clean), and codex-native-startup changes stay clear of #2487's.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[freeqaz-openai]

#2660 has landed. I checked the overlap in console-agent-overview.test.mjs only, at your current 1516c273. The file merges cleanly; no need to refresh the branch just to satisfy an outdated conflict report.

One small integration detail: the new deployment poll patches rewrite only changed text… case still calls newPage(t, fixture), so it would launch a second Chromium. Once #2660’s file-scoped browser is present, use newPage(t, fixture, { browser }) there too. I ran that exact joined file in sandboxed Chromium: the two viewport/poll tests and the new patch test all passed, one browser launch, separate test contexts. This is targeted integration evidence, not a review or approval of the whole PR.

[clawsweeper]

Codex review: needs changes before merge.

What this changes

This PR adds regression tests for startup-probe retry rules, console polling, loading transitions, recovery links, and discarded Slack drafts.

Example: A superseded runtime-status request returns HTTP 504 after a newer successful response.

• Before: Existing tests cover stale successful responses but do not check this failure path.
• After: The added test checks that Pod cards remain visible and the stale timeout message stays hidden.

Review scores

Measure	Result	What it means
Overall readiness	🦐 gold shrimp (3/6)	Useful focused coverage with successful exact-head CI, limited to harness evidence and containing one unsupported synthetic case.
Proof confidence	🦐 gold shrimp (3/6)	This maintainer-authored test-only PR changes no shipped behavior; browser fixtures and the generated-entrypoint harness support coverage assessment without requiring new live-runtime media.
Patch quality	🐚 platinum hermit (4/6)	1 actionable review finding remain.

Product

Kind: Test only · Worth it: Yes · Fix scope: Complete
User problem: Regressions in recent startup and console fixes could escape the existing assertions.
Reason: The maintainer-authored direction is established, and most additions protect distinct observable behavior without changing product contracts.

Merge readiness

⛔ Needs changes before merge - 1 item remains

Keep open for maintainer triage: this maintainer-authored PR adds useful regression coverage, with one low-impact test-integrity correction.

Priority: P3
Reviewed head: 1516c2735e3a5b0527aabf519b1f8238ddd85f1e

Before merge

• [P3] Replace the impossible numeric-stdout fixture (P3) - This scenario reaches WRAPPER_ERROR by making the mocked spawnSync return stdout: 1. The production call uses encoding: "utf8", and Node returns text or null here, so this TypeError cannot arise from native process output. The assertion therefore pins behavior invented by the fixture, contrary to AGENTS.md's test-integrity rule. Remove this row or exercise a concrete, reachable wrapper exception instead.

Findings

• [P3] [P3] Replace the impossible numeric-stdout fixture — tests/conformance/codex-native-startup.test.mjs:817-818

Tests

• Low-value test tests/conformance/codex-native-startup.test.mjs: The numeric-stdout wrapper-error scenario exercises an impossible dependency result; remove or replace it as described in the finding.
• Missing end-to-end proof: The added cases' intended mutant-failure comparisons remain pending in the supplied verification; passing CI does not establish those comparisons.

Agent review details

How this fits together

Browser tests exercise the console and fixture-backed controller routes; startup conformance tests execute the generated runtime entrypoint with scripted native-process results and inspect readiness and failure diagnostics.

flowchart LR
  BrowserTests[Browser scenarios] --> Console[Console modules]
  Console --> Controller[Fixture controller API]
  Controller --> BrowserResults[Visible state assertions]
  StartupTests[Startup scenarios] --> Entrypoint[Generated runtime entrypoint]
  Entrypoint --> StartupResults[Readiness and diagnostic assertions]

Loading

Technical review

Best possible solution:

Retain the focused regression coverage and replace or drop the unreachable numeric-stdout scenario.

Do we have a high-confidence way to reproduce the issue?

This is a coverage PR rather than a current product-bug report; source inspection establishes the unsupported fixture input without executing target code.

Is this the best way to solve the issue?

Extending existing scenarios is appropriate, while the numeric-stdout case should use a reachable failure or be removed.

Full review comments:

• [P3] [P3] Replace the impossible numeric-stdout fixture — tests/conformance/codex-native-startup.test.mjs:817-818
  This scenario reaches WRAPPER_ERROR by making the mocked spawnSync return stdout: 1. The production call uses encoding: "utf8", and Node returns text or null here, so this TypeError cannot arise from native process output. The assertion therefore pins behavior invented by the fixture, contrary to AGENTS.md's test-integrity rule. Remove this row or exercise a concrete, reachable wrapper exception instead.
  Confidence: 0.97

Overall correctness: patch is correct
Overall confidence: 0.91

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 0dc13f25a432.

Provenance checked

• Deployment polling and recovery browser scenarios keeps the original intent (6088f70: fix(console): open recovery links in view, hold the loading page, quiet deploy polls #2646 intentionally preserved unchanged DOM, queued Refresh during polls, respected recovery navigation, and held newly shown loading pages.)
• Superseded runtime-status responses keeps the original intent (6088f70: fix(console): open recovery links in view, hold the loading page, quiet deploy polls #2646 requires stale successes and failures to yield to the newest status read; this PR adds the missing failure assertion.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

Acceptance criteria:

• node --test tests/conformance/codex-native-startup.test.mjs
• pnpm lint
• pnpm format:check
• git diff --check

What I checked:

• Pinned test-only scope: The introduced diff changes five test files, adding 261 lines and removing eight; it changes no production implementation, configuration, schema, or workflow. (1516c2735e3a)
• Unsupported process-result fixture: The added wrapper-error scenario supplies stdout: 1, whereas the production spawnSync call requests UTF-8 output and then trims the returned text. (tests/conformance/codex-native-startup.test.mjs:818, 1516c2735e3a)
• Native process contract: Node documents spawnSync stdout as Buffer, string, or null, never a number; this contract applies because the generated runtime directly consumes spawnSync results. (doc/api/child_process.md:1422, 0d10bfe3cb96)
• Exact-head validation: GitHub reports successful CI Required, Checks and Conformance 1, Console Browser, Console Browser 2, and Static Checks for the reviewed head. Detailed job-log retrieval was blocked by the artifact host allowlist, so no leaf-log results are claimed. (1516c2735e3a)
• Current-main integration discussion: The scoped discussion credits freeqaz-openai's joined-file Chromium check after test(console): speed up the Agent overview tests #2660 merged, and suggests passing its shared browser to the new helper test; it explicitly makes no whole-PR approval claim. (tests/browser/console-agent-overview.test.mjs:1028, 1516c2735e3a)
• Repository policy applied: The complete root instructions were read; no nested test instructions or maintainer-notes directory were found. Test-integrity requirements apply. The explicit single-reviewer instruction overrides the autoreview-helper requirement, and no helper, second reviewer, tests, or builds ran. (AGENTS.md:355, 1516c2735e3a)

Likely related people:

• Free Wortley: Raw commit 04d64a9 adds tests/conformance/codex-native-startup.test.mjs:477 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 04d64a9937a1; files: tests/conformance/codex-native-startup.test.mjs)

Review metrics

Metric	Value	Why it matters
Production versus test changes	Production +0/-0; tests +261/-8	Coverage growth mostly extends existing scenarios without introducing production seams.

Labels

Label changes:

• add P3: This is test-only maintenance with a low-impact fixture correction and no demonstrated runtime regression.
• add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
• add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.

Label justifications:

• P3: This is test-only maintenance with a low-impact fixture correction and no demonstrated runtime regression.
• rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
• status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 16, 2026, 8:54 PM ET / October 17, 2026, 00:54 UTC.