e2e: BDD/Stagehand suite + E2E-on-local-k8s CI workflow - #4
Open
xiduzo wants to merge 62 commits into
Open
Conversation
Switch the e2e project to playwright-bdd (Gherkin features generated into .features-gen), add Stagehand/OpenRouter + zod + otpauth deps, and point the `@/*` path alias at the repo root. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete the old spec-based tests and Azure/Maykin login helpers superseded by the playwright-bdd suite. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gherkin features + steps for testscripts 1,3,4,5,9,10,11 (plus @Blocked stubs for 6/7/8), driven by Stagehand for admin/beheer UI mutations and verified deterministically through the session-authenticated page or public API. Organised vertically by GPP-Woo bounded context: each bdd/@<app>/ holds its features, steps and support/ helpers; bdd/_core/ holds the shared harness (fixture graph, ENV, resource-manager, Keycloak, Stagehand plumbing, signIn). Deterministic own-and-cleanup via a makeResourceManager() factory + an adminResource()/adminDriver() seam over the Django admin; run-wide E2E-data sweep in global-teardown. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The @admin steps each rebuilt an adminDriver locally (`function org(sh){…}`)
and threaded `adminStagehand` through every step. Promote the three nouns to
`orgAdmin`/`topicAdmin`/`pubAdmin` fixtures so steps receive a ready-built
driver by destructuring, per the playwright-bdd add-fixtures pattern. Noun→URL
configs move to support/admin-uis.ts. Step text unchanged; typecheck + lint
clean; bddgen 11 features → 11 specs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The TOPIC_IMAGE path doubled its prefix after the vertical restructure moved topic.ts into support/ (ENOENT at runtime). Also enable playwright-bdd's aiFix promptAttachment so failing tests carry a ready-to-paste fix prompt in the report. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unblock the three parked eindgebruiker/document testscripts. TS6/7 (gpp-app publicatie create/withdraw): the blocker was data — no account was in an authorised gebruikersgroep, so /api/mijn-gebruikersgroepen was empty and "Nieuwe publicatie" errored. New authProfile fixture seeds the prerequisite over the odpc API (identity from /api/me); publicatie-ui.ts fills the form deterministically by input value=uuid (option-groups are custom widgets), Stagehand drives navigation + Publiceren/Intrekken. Verified via the deterministic publicatiestatus read through the publicatiebank admin. TS8 (document beheer): provision the Documenten API (setup/provision-documenten-api.sh wires the woo-publications DRC service + GlobalConfiguration; OpenZaak was already set up + also applies the token-auth fix live). document.ts seeds a published document over the token API (Host header via APIRequestContext so OpenZaak resolves the informatieobjecttype URL; upload URLs rewritten to localhost), docAdmin driver withdraws/deletes it, verified via the admin changelist. Needs --workers=1. Also: documents added to the global-teardown sweep; plan doc updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stagehand 3.6 supports zod 4; migrate the env/user schemas to the zod-4 API (z.email()/z.url()). openai@4 still declares an unsatisfiable optional peer on zod@3, so .npmrc sets legacy-peer-deps to keep clean/CI installs flag-free. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the single @Anthropic model tag with a role×tier routing table (ROLE_MODELS in bdd/_core/stagehand.ts): each Stagehand op (act→worker, extract→verification, observe→planner) picks a model by the scenario's cost tier. Scenarios are tagged @ai (skip-without-OPENROUTER_API_KEY guard); @cheap-ai/@expensive-ai shift the tier and @model:<id> pins one model. Per-request routing is done in the OpenRouter fetch wrapper, so a single llmClient serves all roles. The worker/verification default is gpt-4o-mini: gemini-2.5-flash intermittently returns action JSON that fails Stagehand's zod schema (observed as recurring "Zod schema validation failed" flake); gpt-4o-mini is comparably cheap without it. Claude Sonnet stays the @expensive-ai tier. checks/stagehand-routing.check.ts is a standalone node self-check for the tier/override/table resolution and the model-rewriting fetch; tsconfig and eslint exclude *.check.ts (standalone run-scripts) and untyped markdown code samples. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keep ad-hoc run reports/logs (scratch/) out of the tree. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Formatting only — normalise table column widths and emphasis markers (*x* → _x_) to satisfy the repo's prettier config. No content change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Stagehand used to launch its own Chromium, so Playwright's trace only captured the near-empty page/API context — the AI browser (where all the UI work happens) was invisible: about:blank, no filmstrip. Now Stagehand attaches to the Playwright-launched Chromium over CDP (localBrowserLaunchOptions.cdpUrl → the chromium project's --remote-debugging-port, per worker) and adopts its existing page. So the AI drives the SAME target Playwright authenticated and traces: one trace.zip per scenario with screencast + DOM snapshots + network + the AI's actions. Cookie injection and the duplicate Keycloak host-arg are gone — the shared context already carries both. Consequences: - The `page` context must carry the session the AI needs, so storageState maps @beheer → burgerportaal beheer-admin and other @ai → admin. - CDP is Chromium-only, so @ai/@beheer are now chromium-only (the AI browser was always Chromium regardless; only verification-page reads were cross-browser). This also removes the flakiest firefox/webkit AI runs and the odrc 500s from 3-browser concurrent load. - @beheer is now tagged @ai and shares the single @ai guard (skip without key + chromium-only); its bespoke guard is removed. Validated: unified trace confirmed (33 screencast frames, 21 DOM snapshots, the AI's admin navigations as trace actions); full chromium project 38 passed / 0 Zod errors / no non-AI regressions. Residual @ai mutation flakiness is unchanged and tracked separately. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Exclude @ai (Stagehand) scenarios from the firefox/webkit projects with grepInvert: /@ai/ — Stagehand drives Chromium over CDP, which firefox and webkit don't expose, so those scenarios errored (fixture CDP fetch) instead of skipping, and could clobber shared global resources when run in parallel. Enable a single local retry (CI stays at 2). The @ai act()/SPA-teardown flake rotates across scenarios run-to-run, so per-scenario tagging can't catch it; a retry absorbs the transient miss while a real failure still fails twice. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- adminDriver.open() now settles the change page after the row-click act, like every other driver method. Without it, callers querying form fields (e.g. #id_publicatiestatus) via the no-auto-wait Stagehand understudy locator could hit the DOM before it rendered (StagehandElementNotFoundError). - openBeheer waits up to 15s (was the 5s default) for the admin Dashboard link: the Beheer redirect + admin render exceeds 5s on firefox/webkit against an emulated backend. - document.ts getJson: status-checked GET with one retry on 5xx, replacing blind res.json() — turns an opaque "Unexpected token '<'" into "GET x -> 500: <body>". Documents that the token seed needs provision-documenten-api.sh (wires the Documenten API + patches ODRC token auth to AnonymousUser; a down/up reverts it). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
act() on the publicatiestatus dropdown is the single flakiest Stagehand step. Set it with a native selectOption on #id_publicatiestatus instead (Stagehand still opens + saves). The publicaties step previously used act(); documenten already did this. Both use the Stagehand understudy locator on the same tab after adminDriver.open() settles the change page. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- @expensive-ai (Claude Sonnet) on the repeat AI-flaky scenarios: the whole gebruikersgroepen feature (serial-first flake cascades), onderwerpen "Edit omschrijving", organisaties "Rename". - @fixme quarantine on the documenten (TS8) feature: the seed works once provision-documenten-api.sh is run, but the withdraw step is unreliable — adminDriver.open()'s row-click doesn't consistently reach the document change page (its changelist link differs from the publicatie one). Pre-existing gap; real fix is to open the change page by URL. - Drop folder-name tags repeated on Feature lines (@publicatiebank/@admin, @gpp-app, @burgerportaal) — tags-from-path already applies them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add an explicit rule in "Authentication by tag": don't repeat folder-name tags on the Feature line; tags-from-path already applies every @-segment. - Update the TS8 entry: it's implemented (seed works) but @fixme-quarantined on the withdraw step, and needs provision-documenten-api.sh per fresh stack (a down/up reverts the ODRC auth patch, reintroducing the user-less-token 500). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Stagehand act() row-click did not reliably land on the document change page; the changelist search + row click now runs via the session page (openDocumentAdmin). Stagehand still drives the save/delete mutations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…uard, declared sweep order Architecture pass to keep the suite scalable as features multiply: - _core/fixture.ts becomes a composition root; fixture bodies move to _core/core-fixtures.ts and per-app fixtures.ts (publicatiebank -> gpp-app -> burgerportaal chain; gpp-app extends publicatiebank because authProfile seeds through its organisations fixture). Step files are untouched. - _core/roles.ts replaces setup/paths.ts: storage-state paths plus the tag -> session table (stateForTags) consumed by the storageState fixture, auth.setup.ts and global-teardown.ts. New role = one row + one login line. - _core/singleton-guard.ts captures the snapshot/restore pattern for shared server-side singletons (beheer config), stating the @mode:serial / chromium-only contract once. - global-teardown sweep order is declared per row via dependsOn and topo-sorted (_core/topo.ts, loud on unknown labels/cycles) instead of encoded in array position; checks/teardown-order.check.ts covers it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Give every feature a matching <feature>.steps.ts and reserve _shared.steps.ts for steps genuinely reused across features. Step files are glob entrypoints (matched to Gherkin by text, never imported), so renames are safe. - rename <feature>-steps.ts -> <feature>.steps.ts; single-feature steps.ts files renamed to their feature (sitemap, configuratie, organisaties, ...) - move informatiecategorieen steps next to their feature under @admin/ - split @gpp-app/steps.ts and @admin/steps.ts: feature steps to their file, the reusable login precondition and cross-vertical @ai hook to _shared.steps.ts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Spin up the full GPP-Woo stack from the Helm charts on a kind cluster (the same charts/hack/up.sh used locally) and run the Playwright suite against it, so the real charts are validated on every PR. Checks out all app repos as siblings (up.sh builds every image from source; open-zaak is cloned by up.sh) and points the suite at the NodePort URLs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Identify and scaffold significant E2E test coverage gaps from manual test scripts by introducing `@todo` tagged scenarios. - New `.feature` files contain `@todo` scenarios outlining missing assertions for Burgerportaal rendering, sitemap details, search functionality, and GPP-app user group/publication management. - Corresponding stubbed step definitions (`.steps.ts`) are added to keep `bddgen` green. - A new `bdd/_core/todo.steps.ts` hook automatically skips `@todo` scenarios at runtime, enabling progressive implementation without blocking CI. - Adds a `dev` script to `package.json` for quicker local development. ```
xiduzo
had a problem deploying
to
maykin and icatt dev environments
July 21, 2026 11:49 — with 
GitHub Actions
Failure
GitHub Actions
Failure
- e2e-k8s.yml: charts checkout needs ref explore/local-via-k3d — hack/
(up.sh, kind-config.yaml) isn't on charts' main, so the kind config
was missing and cluster creation failed.
- tsconfig: add lib ES2022+DOM so Error(msg, { cause }) typechecks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
xiduzo
had a problem deploying
to
maykin and icatt dev environments
July 21, 2026 11:56 — with
GitHub Actions
Failure
GitHub Actions
Failure
- e2e-k8s.yml: add azure/setup-helm@v4 (v3.18.0). up.sh's zoeken install uses `helm upgrade --force-conflicts` (server-side apply), added in Helm 3.18; the runner's pre-installed Helm was older and errored out. - Clear 86 eslint errors in BDD step files: drop unused fixture keys (use `_`/`()` where the destructure empties, keep Cucumber capture args), split multi-statement guard lines, single quotes. - Auto-format docs/skills markdown. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
xiduzo
had a problem deploying
to
maykin and icatt dev environments
July 21, 2026 12:47 — with
GitHub Actions
Failure
GitHub Actions
Failure
azure/setup-helm@v4 cached the binary in a nested linux-amd64/ dir it doesn't add to PATH, so up.sh still ran the runner's older Helm and hit `unknown flag: --force-conflicts`. Install over /usr/local/bin/helm directly and print `helm version` to confirm 3.18 is the one used. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit refactors several BDD test suites to use plain Playwright locators instead of AI-driven browser automation (Stagehand + OpenRouter).
* **Motivation:** AI browser automation proved flaky and costly for testing structured UIs like the Django admin and Vue SPAs, where stable locators are readily available.
* **Changes:**
* `@beheer`, `@publicatiebank/@admin` (onderwerpen, documenten, publicaties), and `@gpp-app` (publicaties) scenarios are updated to use direct Playwright `page` interactions.
* The `@ai` tag and associated skips are removed from these features.
* The `AdminDriver` and related fixtures are refactored to operate directly on the Playwright `page`.
* The `@chromium-only` tag is introduced for scenarios mutating shared global resources to prevent race conditions in parallel runs.
* **Outcome:** Improves test stability, reduces external API costs, and simplifies test logic by leveraging direct locator capabilities. The Stagehand plumbing is retained but currently unused, for potential future use cases with genuinely unaddressable UIs.
```
This commit completes several previously skipped BDD test scenarios and enhances existing ones for improved reliability and accuracy. * **Expand BDD scenario coverage:** Implement end-to-end tests for document and information category management in the Publicatiebank admin, removing `@todo` tags from numerous scenarios. * **Improve GPP-app UI interaction reliability:** Introduce dedicated Playwright helper functions (`confirmWrite`, `fillTitel`, `selectProfiel`) to robustly handle common UI flakiness in the GPP-app, such as timing issues with form updates and button clicks. * **Refine WebKit skips:** Extend the `@no-webkit` tag to the GPP-app SPA and clarify the underlying `Content-Security-Policy: upgrade-insecure-requests` issue that prevents both SPAs from booting on `http://localhost` in WebKit. * **Increase authentication timeout:** Adjust the admin authentication setup timeout to mitigate potential flakiness with one-time codes during test runs.
Co-authored-by: Cursor <cursoragent@cursor.com>
…tching Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Wire seed+index helpers, short sitemap-cache waits, and exact DiWoo tag matching so previously @Blocked search and sitemap coverage runs deterministically. Co-authored-by: Cursor <cursoragent@cursor.com>
Implement previously `@blocked` scenarios by wiring up document content search via Elasticsearch attachment ingest and enabling document withdrawal in the GPP-app UI. This resolves long-standing blocks related to: * **Document content search:** Configures OpenZaak SENDFILE and `zoeken` download services for document body ingestion by Elasticsearch. Seeds documents with unique `fileContent` to verify hits. * **Document withdraw:** Ensures ODPC→ODRC document mutations work, allowing the GPP-app to withdraw documents via the UI. Adds dedicated Playwright helpers for document management.
Baseline was 235 passed / 22 failed / 12 flaky / 35 did-not-run; now 304/304 with no flakes. Details and per-item evidence in E2E-FIXES-2026-08-10.md. Most of the "flakiness" was a starved backend, not test code: the kind node ran at 340 MB free of 7935 MB and the kernel SIGKILLed gpp-publicatiebank (exit 137), which surfaced as `POST documenten -> 502`, a Beheermenu that never rendered and beforeEach timeouts — all of which passed in isolation. Cap local workers so the stack is not pushed into swap. Test-side root causes: - The sitemap is eventually consistent with the token API (ODRC registers an uploaded document with the Documenten API asynchronously), so a just-seeded document is briefly absent and a withdrawn one briefly still present. Poll in support/sitemap.ts instead of reading once, and give the feature @timeout:120000 — the monthly-sitemap fetch is O(n) in seeded documents. - zoeken: the filter assertion snapshotted an article count before the list re-rendered, so it could read a Publicatie on its way out. Poll the rendered type labels, scoped to the result list. - intrekken: PromptModal.vue renders a bare <dialog> with no aria-label, and a <dialog> does not take its name from an inner heading, so getByRole('dialog', {name: /intrekken/i}) matched nothing at all. Locate by heading, and confirm via confirmWrite — a bare click lands before the app wires its handler and is silently dropped, so nothing was withdrawn. - The ingetrokken assertion resolved to the hidden <option value="ingetrokken"> of the status filter; read the row's role="status" instead. - organisaties: the Django "Actief: Ja" filter is a full navigation, so reading th.field-naam a immediately could return []. Also stop requiring every active organisatie in a global snapshot to resolve in the GPP-app — other workers create and delete their own E2E rows mid-run. Stack provisioning: - Provision the two GlobalConfiguration publication URL templates. They ship empty, so the API serves urlPublicatieExtern as "" and the GPP-app renders "Bekijk online" with href="" — which reloads the GPP-app instead of opening the burgerportaal. This is the long-standing :8130-instead-of-:8140 failure; it was a provisioning gap, not a GPP-app deploy bug. - provision-documenten-api.sh was not idempotent despite saying so: it keyed Service.objects.update_or_create on slug while api_root is the UNIQUE column, so re-running died on api_root_key before reaching the config save. Also: repo-wide lint is clean again (missing node: imports for process and Buffer, plus formatting). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…auth fix
Two reasons the pipeline failed 81 scenarios while the same suite ran 304/304
locally. Neither was test logic — the identical 81 failures, with identical
grouping, are present on the previous commit too.
webkit (43 failures, all `browserType.launch: Target page, context or browser
has been closed`): the Chromium-only `--host-resolver-rules` flag was set in the
shared `use.launchOptions`, so it was passed to every browser. WebKit on Linux
hard-errors on an unknown option:
Cannot parse arguments: Unknown option --host-resolver-rules=...
so the browser never launches. macOS WebKit tolerates the flag, which is exactly
why this only ever showed up on CI. Firefox cannot use it either, and the
scenarios that need it are @chromium-only. Moved to the two chromium projects —
including `setup`, which performs the real Keycloak sign-in and is the one place
that genuinely needs the rule.
GPP-publicatiebank was checked out at its default branch, where
TokenAuthentication still returns `(None, token)`; SessionProfileMiddleware then
dereferences request.user, so every token API call 500s while an admin session
is live. Pinned to fix/token-auth-anonymoususer until that lands on main.
The remaining 35 CI failures were `Bad Request (400)` DisallowedHost pages from
ODRC, fixed on the charts side (the e2e seed's Host header was not in
ALLOWED_HOSTS there).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
52 of the remaining 55 CI failures were `POST documenten -> 500` (all three browsers, almost all of sitemap.feature). The pipeline went straight from `hack/up.sh` to the suite, so nothing ever wired the Documenten API: no DRC Service, no GlobalConfiguration.documents_api_service, and no ORC service in OpenZaak for the publicatiebank catalogi. Every scenario that seeds a document therefore failed at the seed. This is the same once-per-fresh-stack step the README already documents for a local bring-up, so run the existing script rather than duplicating the wiring in the workflow. It is idempotent, and it also sets the publication URL templates, which ship empty and otherwise make the gpp-app "Bekijk online" scenario open the gpp-app instead of the burgerportaal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…pty sitemaps The dump selected pods with `app.kubernetes.io/instance=odrc|zoeken`, but the publicatiebank pods carry no instance label and those selectors match only the *redis* pods — so on every failure it collected redis logs and never the app or worker logs. `-l app=openzaak` matched nothing either. Select by deployment name instead. Then add setup/dump-odrc-state.sh. The current CI failure is 44 sitemap scenarios against a sitemap the run itself reports as holding 0 documents, and none of the three possible reasons is visible in any pod log: the burgerportaal lists a document only when it is gepubliceerd AND upload_complete (the worker registered it with the Documenten API) AND its publisher is not zelf_toegevoegd. The script prints all three, plus the GlobalConfiguration wiring, so the next run says which one it is instead of leaving it to inference. Useful locally for the same question. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
18 sitemap scenarios failed with "SITEMAP_CACHE_WAIT_MS is unset". That guard is deliberate — waitForSitemapCacheExpiry refuses to guess, because the production output cache is ~23h and sleeping that long is never what a test wants. The workflow just never set it. charts dev/values/odbp.yaml pins sitemapCacheDurationHours to 0, so 0 is the correct wait here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
36 of the 38 remaining CI failures are "document … present in a sitemap", and the message does not distinguish the two causes, which need opposite responses: - ODRC has not yet flipped isGereedVoorPublicatie (the worker registers the uploaded file with the Documenten API asynchronously) — a stack/timing problem; - or it is gereed and the burgerportaal still does not list it — a filtering or cache issue in ODBP. So both sitemap assertions now report the document's ODRC state and how many entries were actually searched. "not found among 0 entries, ODRC says isGereedVoorPublicatie=false" and "not found among 40, gereed=true" are different bugs; the report should not leave that to inference. documentReadiness never throws — it exists to explain a failure, not to add one. Also note in dump-odrc-state.sh that it runs after globalTeardown, so its document counts legitimately read 0 in a CI dump and only the wiring lines mean anything there. That dump did confirm the wiring is correct (documents_api_service and gpp_search_service set, rsin set, url template set, one actieve gemeentelijst organisatie), which rules out provisioning. Local sitemap.feature stays 24/24. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
xiduzo
force-pushed
the
explore/BDD-stagehand
branch
from
August 11, 2026 09:18
0a4242a to
a0d8f14
Compare
The suite was 39 min of a 54 min job, and the stack bring-up another 10. Three changes, in order of what they buy: - workers: 2 on CI. ubuntu-latest is 4 vCPU / 16 GB and the kind stack sits at roughly half of it, so there is room for a second browser. If the backend starves it shows up as 502s on documenten / exit 137 / beforeEach timeouts that pass in isolation — drop back to 1 rather than chase them. - Pull requests run chromium only. firefox and webkit are 138 of the 310 tests and catch engine differences, which move on the browsers' release schedule and not on a PR. All three run nightly at 02:00 UTC and on a workflow_dispatch with browsers: all. - Cache the five app images, keyed on the app repos' resolved HEADs plus a hash of up.sh (which holds the build commands and the OPENZAAK_VERSION default). A hit skips ~8 of up.sh's ~10 min via its existing SKIP_BUILD. Staged only after a clean bring-up, written temp-then-renamed, so a half-built or half-saved tar can never be restored under a valid key. The report needed to keep up: a chromium-only run must not read as a green cross-browser one. The Confluence summary now says which browsers the run covered, splits the count into scenarios (170) versus scenario × browser runs (310), and breaks each row's status down per browser — 🔵 chromium, 🦊 firefox, 🧭 webkit, with ➖ for a browser the scenario never ran in (excluded by tag, or the run was scoped). Skip reasons keep the glyph of the browser they belong to. Single-browser runs drop the breakdown, which would just repeat the status. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Playwright's JSON reporter does not merge a spec across projects: a three-browser run emits the same scenario three times as sibling specs sharing a location, each holding a single `test`. The publish script read `spec.tests` as "this scenario on every browser", so every row showed one lit glyph and the summary reported 304 scenarios for 304 test runs. Fold specs on file:line:column before computing status and breakdown, and link the merged row to the browser whose status won rather than to whichever project serialised first. Also drop `setup` from the browser list: it is a project, not a browser, and it was hanging a dead `setup-` off all ~300 rows. Its own spec stays in the table, without a breakdown. The selfcheck fixture was hand-written in the merged shape, which is why this slipped through; it now uses the shape the reporter really emits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Ports the e2e suite to playwright-bdd with a vertical
bdd/structure and Stagehand AI scenarios, and adds a CI workflow that runs the whole suite against the real Helm charts in a kind cluster — the samehack/up.shflow used locally.Why open this PR
The
E2E on local k8sworkflow (.github/workflows/e2e-k8s.yml) lives only on this branch, so it wasn't visible in the Actions tab or runnable viaworkflow_dispatch(both require the default branch). This PR triggers it viaon: pull_request, making the first run appear here.Highlights
.github/workflows/e2e-k8s.yml— spins up kind, brings up the GPP stack from charts (USE_EXISTING_CLUSTER=1 hack/up.sh), raisesvm.max_map_countfor ES/ECK, runs Playwright, uploads report + dumps cluster state on failure.bdd/_corefixtures, per-app role registry, Stagehand routing by role×cost tier,@todocoverage stubs.Note
The workflow pins
GPP-appandGPP-burgerportaaltoref: feat/local-dev-stack— tracking those feature branches, not their mains.🤖 Generated with Claude Code