Skip to content

test(e2e): measure fluid charts only once style.css applies (#148) - #157

Merged
dborup merged 2 commits into
masterfrom
codex/issue-148-fluid-charts-flake
Oct 1, 2026
Merged

dborup merged 2 commits into
masterfrom
codex/issue-148-fluid-charts-flake

Conversation

@dborup

@dborup dborup commented Oct 1, 2026

Copy link
Copy Markdown
Owner

Relates to #148

Cause

test-analytics-fluid-charts.js (the "Kpa-clawbot#1058 Analytics fluid charts E2E" step in the Playwright job) writes a file:// harness that links public/style.css with <link rel="stylesheet">, navigates with waitUntil: 'domcontentloaded' and measures right away. DOMContentLoaded does not wait for stylesheets. When style.css has not been applied yet, the .analytics-charts cards are plain blocks and stack in one column, so every "≥2 columns" check reads got 1.

The three 1-column checks (768/760, 1440/600 and the AC3 shrink) also pass on an unstyled page, so before this change they could pass without the CSS under test having been applied.

Failures in CI (attempt 1; a rerun went green each time)

PR Run / job Failing step
#135 run 36543893158, job 109332087103 viewport 2560 / wrapper 2520px → side-by-side (≥2 cols), no overflow: expected ≥2 columns at wrapper 2520px; got 1
#139 run 36557644736, job 109377157851 AC3: layout reflows on resize (1440 side-by-side → 768 stacked): precondition failed: expected ≥2 cols at 1300px; got 1
#141 (3ff60504) run 36812985385, job 110216676683 AC3: layout reflows on resize (1440 side-by-side → 768 stacked): precondition failed: expected ≥2 cols at 1300px; got 1

Reproduction (commit 5fd36858)

  • The test routes style.css through the browser context, so it can be held back. Playwright route interception works for file:// URLs.
  • A new step measures the 1300px case with style.css held back for 1 s. On the unfixed measurement it fails with expected ≥2 columns at wrapper 1300px; got 1, 10 times out of 10 locally.
  • FLUID_CHARTS_CSS_DELAY_MS=<ms> holds style.css back on every load. With FLUID_CHARTS_CSS_DELAY_MS=1500 the unfixed test fails exactly the steps seen in CI, with the same messages:
    ✗ analytics-charts opts in to container queries: expected container-type to be inline-size; got "normal"
    ✗ viewport 1440 / wrapper 1300px → side-by-side (≥2 cols): expected ≥2 columns at wrapper 1300px; got 1
    ✗ viewport 1440 / wrapper 1300px with style.css held back 1s → side-by-side (≥2 cols): expected ≥2 columns at wrapper 1300px; got 1
    ✗ viewport 1920 / wrapper 1880px → side-by-side (≥2 cols), no overflow: expected ≥2 columns at wrapper 1880px; got 1
    ✗ viewport 2560 / wrapper 2520px → side-by-side (≥2 cols), no overflow: expected ≥2 columns at wrapper 2520px; got 1
    ✗ AC3: layout reflows on resize (1440 side-by-side → 768 stacked): precondition failed: expected ≥2 cols at 1300px; got 1
    3 passed, 6 failed
    

Fix (commit 26b934e9)

  • load() navigates with waitUntil: 'load'. It then calls waitForStyles(), which waits until the harness's stylesheet <link> has its sheet, then for two animation frames.
  • The AC3 resize step calls the same waitForStyles() after setViewportSize, in place of the bare two-frame wait.
  • The wait is on the stylesheet itself, not on container-type or the grid template, which the test asserts. A regression in the CSS under test still fails on its own assertion instead of as a timeout.
  • Assertions (column counts, overflow, reflow) are unchanged. Only test-analytics-fluid-charts.js changes; no CSS and no workflow change.
  • Checked by mutation with FLUID_CHARTS_CSS_DELAY_MS=1500: the sheet wait alone, with domcontentloaded kept, passes 9/9. Without the wait, the six steps above fail again.

Runs (local, macOS, Playwright 1.58.2, Chromium 1208)

  • Fixed test, 50 runs in a row: 50 passed, 0 failed.
  • Fixed test with FLUID_CHARTS_CSS_DELAY_MS=1500, 20 runs in a row: 20 passed, 0 failed.
  • Unfixed measurement with the new step: 10 of 10 runs fail.
  • Runtime: about 1.4 s → 2.6 s per run, mostly from the new step's 1 s hold.

Similar tests

  • test-charts-fluid-1058-e2e.js (the other [#1050] Task 5: Analytics chart containers — fluid + auto-stacking Kpa-clawbot/CoreScope#1058 analytics layout test) also navigates with domcontentloaded, but against the SPA. It measures only after the analytics JS has rendered .analytics-row with data, and the app's parser-inserted scripts do not run until the stylesheets before them have loaded. Against a local server on the migrated fixture it passed 23/23. It also passed 23/23 with style.css held back 1.5 s on all 9 loads (via a local preload that was not committed). It does not have this bug and is not changed.
  • test-analytics-fluid-charts.js is the only test that measures on a file:// harness linking style.css. The other tests that reference style.css and use domcontentloaded run against the SPA, like the test above.

Not verified

  • CI on this branch: see the checks on this PR.
  • That a slow stylesheet is the only cause of the CI flakes. The CI logs do not show timing. The reproduction gives the same failures with the same messages, and the fix removes that race.

🤖 Generated with Claude Code

dborup and others added 2 commits October 1, 2026 06:52
…et (#148)

The harness navigates with waitUntil 'domcontentloaded', which does not
wait for style.css. Route style.css through the context so it can be
held back, and add a step that holds it back for a second before
measuring the 1300px case. Unstyled cards stack in one column, so the
step fails with "got 1", the same message as the CI flakes. It fails
10/10 locally until the measurement waits for the stylesheet.

FLUID_CHARTS_CSS_DELAY_MS holds style.css back on every load, which
reproduces the failing steps seen in CI (2520px side-by-side, AC3
precondition).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Navigate with waitUntil 'load' and wait until the harness's stylesheet
link has its sheet, then two frames, before measuring. The AC3 resize
step uses the same wait after setViewportSize. The wait is on the
stylesheet itself, not on container-type or the grid template, so a
regression in the CSS under test still fails on its own assertion.

Assertions (column counts, overflow, reflow) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dborup
dborup marked this pull request as ready for review October 1, 2026 09:56
@dborup
dborup merged commit 727efca into master Oct 1, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant