Skip to content

test(ci): add five duration rows and refresh console-focus-ring's - #3011

Merged
freeqaz merged 3 commits into
mainfrom
ci-fix-durations
Oct 20, 2026
Merged

freeqaz merged 3 commits into
mainfrom
ci-fix-durations

Conversation

@freeqaz

@freeqaz freeqaz commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

The flake-sweep-1020 window (10-19 18:00Z to 10-20 04:18Z) found five shared files with no row in file-durations.json: console-namespace-create, console-shell, console-page-header, credential-source-api and agent-create-validate-api. A file without a row starts first in its lane, ahead of longer measured files. console-focus-ring's row was 11.1 s, but #2977 grew the file and it now measures about 15 s.

Change

scripts/ci/test-suites/file-durations.json gets five new rows and one refreshed row. Each row uses the table's documented method: the slowdown-normalised median over the green main runs whose head contains the file's last change. Each run lane's factor is the median, over its files other than these six, of their time divided by their current row. The source text records this.

File Row Runs
console-namespace-create 15.6 s (new) 3 (main run 81c57a7 and #3002's two branch runs, after #3002 grew it from 7 to 10 tests)
console-shell 12.8 s (new) 2 (last changed by #2996)
console-page-header 8.5 s (new) 15
console-focus-ring 11.1 -> 15.5 s 15
credential-source-api 4.2 s (new) 21
agent-create-validate-api 3.2 s (new) 21

Data: the sweep's downloaded ci-results-<lane> artifacts from main runs in the window, plus #3000. console-namespace-create is the exception: #3002 grew it after the window, so it uses the runs listed in the table. The branch is merged with main.

Verification

  • node scripts/ci/run-tests.mjs audit passes. It validates the duration table along with the suite manifests.
  • The ci-runner duration and start-order cases pass.
  • Lane balance (longest-first greedy over fileConcurrency slots, using the new times): Console Browser 88.6 s, Console Browser 2 90.3 s, Checks and Conformance 1 54.4 s, Checks and Conformance 2 56.5 s, unchanged from the old start order within 0.1 s. The new rows move the files out of the "unmeasured, start first" group: Console Browser goes from 3 unmeasured files to 1 and Console Browser 2 from 1 to 0. The one left, console-create-agent-member, landed in fix(console): let members with agent:create use the create form #2998 after the window.
  • prettier --check passes.

Risks

  • Start order only. No test changes. The console-shell row comes from only 2 runs and the console-namespace-create row from only 3, so they are the least certain.

The flake-sweep-1020 window found five shared files with no duration
row, which start first in their lanes, and console-focus-ring's row at
11.1 s against about 15 s measured since #2977 grew it.

Each row is the slowdown-normalised median (the table's documented
method) over the green main runs whose head has the file's last change:
console-namespace-create 12.3 s (8 runs), console-shell 12.8 s (2),
console-page-header 8.5 s (15), console-focus-ring 15.5 s (15),
credential-source-api 4.2 s (21), agent-create-validate-api 3.2 s (21).
@freeqaz
freeqaz requested a review from a team as a code owner October 20, 2026 05:09
@clawsweeper

clawsweeper Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

#3002 grew the file from 7 to 10 tests after the sweep window. Its row is
now the slowdown-normalised median over the main run on 81c57a7 and
#3002's two branch runs: 15.6 s (was 12.3 s from the window's runs).
@clawsweeper clawsweeper Bot added clawsweeper:intake:feature Preliminary submission classification; full source review pending. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed clawsweeper:intake:feature Preliminary submission classification; full source review pending. labels Oct 20, 2026
@clawsweeper

clawsweeper Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge.

What this changes

This PR adds five measured test-file durations and updates the console focus-ring estimate.

Example: Scheduling console-shell.test.mjs in Console Browser

  • Before: The missing estimate places it ahead of every measured shared file.
  • After: Its 12,800 ms estimate places it behind longer measured files.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The patch is focused and supported by existing integration coverage and successful exact-head checks; its measurement calculations and claimed lane balance were not independently demonstrated.
Proof confidence 🦐 gold shrimp (3/6) This collaborator-authored CI metadata refresh is outside the ordinary contributor proof gate; the inspected console screenshots concern linked UI work and do not prove scheduling behavior.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Maintenance · Worth it: Yes · Fix scope: Complete
User problem: CI scheduling estimates become incomplete or stale as test files are added and expanded.
Reason: This maintainer-authored refresh follows the existing scheduling design with a small metadata-only change.

Merge readiness

✅ Ready for maintainer review

Keep this PR open for maintainer triage: it is a focused refresh of CI scheduling metadata, with no blocking correctness or security finding.

Priority: P3
Reviewed head: cf659b7b1baacd28bd23ec59d23cfad798aeaed6

Before merge

None.

Findings

None.

Agent review details

How this fits together

The CI runner reads duration estimates and suite manifests, orders eligible shared files, and executes them within the lane's concurrency limit.

flowchart LR
  A[Suite manifests] --> C[CI runner]
  B[Duration estimates] --> C
  C --> D[Shared file ordering]
  D --> E[Concurrent test execution]
  E --> F[CI results]
Loading

Technical review

Best possible solution:

Keep measured scheduling estimates current within the existing table and runner.

Do we have a high-confidence way to reproduce the issue?

The current-main table and runner establish the missing-estimate ordering directly; no target code was executed during this read-only review.

Is this the best way to solve the issue?

Refreshing the existing table is appropriately scoped, although the credential-source-api estimate is currently inert because that file runs serially.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 8fe865c1649a.

Provenance checked

  • Measured longest-first scheduling keeps the original intent (ci(tests): start shared lane files longest first from measured times #2655: Avoid leaving long shared files waiting behind short files and refresh estimates as tests change.)
  • Console focus-ring duration keeps the original intent (ed489de: The earlier console-test consolidation adjusted estimates for moved tests; the later file growth warrants remeasurement.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

Labels

Label changes:

  • add P3: This is low-risk CI scheduling maintenance without changes to application behavior, test assertions, or test selection.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

  • P3: This is low-risk CI scheduling maintenance without changes to application behavior, test assertions, or test selection.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 20, 2026, 1:35 AM ET / 05:35 UTC.

@freeqaz
freeqaz merged commit 52c1a20 into main Oct 20, 2026
62 of 63 checks passed
@freeqaz
freeqaz deleted the ci-fix-durations branch October 20, 2026 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant