Skip to content

test: pin the row fill's view check and de-time two waits - #2847

Merged
freeqaz merged 2 commits into
mainfrom
test-deslop-1018
Oct 18, 2026
Merged

freeqaz merged 2 commits into
mainfrom
test-deslop-1018

Conversation

@freeqaz

@freeqaz freeqaz commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

Problem

A survey of the tests added or changed on 2026-10-18 (outside files that open PRs touch) found three gaps:

  • apps/controller/src/console/agents/list.mjs: the delayed row fill checks context.isCurrent() && reading.isConnected. No test pinned the context.isCurrent() half. Dropping it passed all 12 row-reading tests in console-agents-list-return.
  • tests/integration/workspace-files-native.test.mjs still compared Number(counter.stdout.trim()) with 0, so empty counter output passed (Number("") === 0). Require an explicit zero in runtime listener tests #2838 fixed the same pattern in the native listener test.
  • Two of today's waits were time-based:
    • kubernetes-real-kubectl's two-teardown late-cleanup test ordered its steps with 20/20/50 ms sleeps.
    • returnWithHeldRows().reveal() polled page.evaluate every 20 ms against a hand-rolled deadline.

Change

This PR changes tests only.

  • console-agents-list-return
    • New case: a refocus holds the same list on screen while its own list read revalidates. The held list is inert and still connected, but no longer current. A reread that landed and is waiting out its 150 ms must not fill a held row. The case advances the page clock past the fill and checks the held rows still read "Reading deployment…".
    • reveal() now uses page.waitForFunction with the same 5 s bound in place of the poll loop.
  • kubernetes-real-kubectl
    • The two-teardown test now orders its steps with promises plus one event-loop turn (setImmediate). Late cleanups drain in microtasks, so that turn is enough.
    • It now asserts that the second teardown's cleanup runs while the first teardown's cleanup is still blocked, and then checks the final order.
    • Both after() throws now check the stopsSetup message. That regex moved to module scope.
    • A finally releases the first cleanup, so a failing run never hangs the file.
  • workspace-files-native: assert.equal(counter.stdout.trim(), "0").

Verification

  • Test names are unchanged except for the one added case. No new files, so the suite mapping is unchanged.

  • Hand-made mutants in list.mjs, run against the row-reading tests (13 with the new case):

    Mutant Before After
    render + 250 ms estimate 4 fail 5 fail
    hold 150 ms when never shown 2 fail 3 fail
    animationstart 5 fail 6 fail
    drop reading.isConnected 1 fail 1 fail
    drop context.isCurrent() in the timer 0 fail 1 fail

    Each run's "After" result was the same before and after the reveal() change.

  • Mutants in tests/helpers/kubernetes-real.mjs, run against the two-teardown test:

    Mutant Before After
    file-wide count gates the drain (the original bug) killed killed
    missing throw to setup killed killed
    one file-wide serial drain queue survived killed

    A FIFO late order survives both versions here; the earlier late-cleanup test covers ordering.

  • 20/20 on 2 CPUs (taskset) for console-agents-list-return and kubernetes-real-kubectl. workspace-files-native needs prepared runtime images, so it is left to its CI lane.

  • Prettier and ESLint are clean on the touched files.

Risks

Test-only. The new browser case uses the page clock and the hold pattern this file already uses. workspace-files-native and kubernetes-real-kubectl keep their CI lanes.

- console-agents-list-return: add a case where a refocus holds the list
  (inert, still connected, no longer current) while a landed reread waits
  out its 150 ms; the held rows must stay unfilled. Kills the mutant that
  drops context.isCurrent() from the delayed fill (finding 1877).
  reveal() waits with page.waitForFunction instead of a 20 ms poll loop.
- kubernetes-real-kubectl: the two-teardown late-cleanup test orders its
  steps with promises and one event-loop turn instead of 20/20/50 ms
  sleeps, checks the second teardown drains while the first's cleanup
  still runs, and checks both throws' messages (stopsSetup hoisted).
- workspace-files-native: require an explicit "0" from the model-call
  counter, so empty output no longer passes (finding 1876).
@freeqaz
freeqaz requested a review from a team as a code owner October 18, 2026 12:48
@clawsweeper

clawsweeper Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-repro Preliminary submission classification; full source review pending. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-repro Preliminary submission classification; full source review pending. labels Oct 18, 2026
@clawsweeper

clawsweeper Bot commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge.

What this changes

This PR strengthens stale-row and cleanup-order tests, replaces timing-based waits, and requires explicit zero model-call output.

Example: The native workspace test receives empty counter output.

  • Before: Number("") evaluates to 0, so the assertion passes.
  • After: The assertion fails because the output must explicitly contain "0".

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) Useful, focused test hardening supported by specific mutation results and successful checks for the touched suites.
Proof confidence 🐚 platinum hermit (4/6) This maintainer-authored PR changes only tests; its reported mutation comparisons and head-linked CI are proportional validation, and no new shipped behavior requires separate live proof.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Test only · Worth it: Yes · Fix scope: Complete
User problem: Regression tests could miss stale writes into a held list, serialized cleanup queues, or absent model-counter observations.
Reason: The maintainer-authored direction adds independently useful assertions to existing suites without expanding product scope.

Merge readiness

✅ Ready for maintainer review

Keep open: this PR adds useful, distinct regression coverage without changing product behavior. No actionable introduced defect was found.

Priority: P3
Reviewed head: fe6378f5e9c42181d5b7cacdbbb7201be8184e7f

Before merge

None.

Findings

None.

Agent review details

How this fits together

The existing browser, conformance, and native-runtime suites exercise console view lifetimes, test-resource cleanup, and workspace operations, then assert DOM state, cleanup ordering, and model-call observations.

flowchart TB
  T[Test suites] --> C[Console view lifecycle]
  C --> D[Held row assertions]
  T --> H[Bounded teardown helper]
  H --> O[Independent cleanup assertions]
  T --> N[Native workspace runtime]
  N --> Z[Explicit zero assertion]
Loading

Technical review

Best possible solution:

Retain the focused assertions and deterministic coordination in the existing suites.

Do we have a high-confidence way to reproduce the issue?

Source inspection confirms that empty counter output passes the old numeric assertion and that the old cleanup assertion permits serialization; the PR also reports targeted mutation comparisons for the added browser case.

Is this the best way to solve the issue?

The patch strengthens existing test owners without adding production seams, duplicating a capability, or changing the runtime contract.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 6f62cbd9ea28.

Provenance checked

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

Review metrics

Metric Value Why it matters
Change scope Production +0/-0; tests +66/-22 The growth adds distinct held-view and concurrent-cleanup assertions within existing suites.

Labels

Label changes:

  • add P3: This is focused test reliability and coverage maintenance with no shipped behavior change.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

  • P3: This is focused test reliability and coverage maintenance with no shipped behavior change.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 18, 2026, 9:01 AM ET / 13:01 UTC.

… the held-view read

Review nits: the second after() assert now sits inside the try, so a failure never leaves the first cleanup holding the file for its step limit; a comment says why the held list is read directly.
@freeqaz
freeqaz temporarily deployed to integration-qa-pr October 18, 2026 13:06 — with GitHub Actions Inactive
@freeqaz
freeqaz temporarily deployed to integration-qa-pr October 18, 2026 13:06 — with GitHub Actions Inactive

@freeqaz-openai freeqaz-openai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep this change. I checked that the new teardown test catches a real loss of coverage: in a disposable copy, I made two teardowns share one late-cleanup queue. The test on main passed with that defect. This version failed because the second cleanup had not run while the first was held, then passed with the actual per-teardown queues. The finally also releases the held cleanup if the assertion fails.

The other two changes make sense: the browser test observes the old list while it is still present but inert, and the native workspace test no longer mistakes empty output for an observed zero.

I verified the hosted results on the merge preview of 6f62cbd9 + 2b85ca7b: 339 browser, 2,579 conformance/baseline, and 18 runtime-startup cases passed, including all three changed cases. My review covers source and those results; the visible ClawSweeper verdict still names the previous head, so I am leaving final review and merge with the owner.

@freeqaz
freeqaz merged commit fb28ccf into main Oct 18, 2026
43 of 44 checks passed
@freeqaz
freeqaz deleted the test-deslop-1018 branch October 18, 2026 13:19

This branch was previously deployed

1 inactive deployment
integration-qa-pr — 2b85ca7b Deployed Oct 18, 2026 by freeqaz via QA advisory (compose) #2208
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants