Skip to content

fix(playwright): stabilize shared AUT state and test oracles - #31788

Draft
harshach wants to merge 65 commits into
mainfrom
fix/nightly-e2e-stability
Draft

fix(playwright): stabilize shared AUT state and test oracles#31788
harshach wants to merge 65 commits into
mainfrom
fix/nightly-e2e-stability

Conversation

@harshach

@harshach harshach commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

What

Stabilizes the Playwright failures that repeated in both database lanes of nightly AUT runs 32180256250, 32276251958, and the strict integration run 32304987037.

  • keep shared beforeAll fixtures in one worker for Column Bulk Operations and Platform Lineage
  • restore the table-to-topic edge after the destructive lineage test
  • wait for the whole Explore filter menu to close before opening the next menu
  • dispatch summary-card clicks on the description container so rich-media children cannot steal the click
  • establish an origin before writing the ingestion-bot IndexedDB token
  • poll the exact glossary-tagged table in the search index before mounting the Assets tab, instead of waiting for an optional browser request that can fire eagerly
  • bracket literal IPv6 webhook hosts; the companion nightly PR now advertises a Docker DNS name because the URL guard correctly rejects private IP literals
  • explicitly persist the Following widget before navigation when a product-specific fallback layout is not yet authoritative
  • wait for Context Center browse/search state and open the exact seeded permission-test article
  • change search-settings sliders relative to their persisted value so Save cannot remain disabled
  • stop waiting for an optional cached Organization request in the Teams helper
  • wait for the exact user PATCH when changing the default persona
  • reopen a certification picker whose async entity refresh cleared its card list
  • use the stable Edit action as the permission-ready signal instead of assuming Re-deploy is present while a pipeline is still active
  • let a dedicated persona user start from an unauthenticated context instead of inheriting the shared admin storage state
  • accept both supported invalid-entity route states: the full PageNotFound view and the entity shell's empty placeholder
  • poll the exact deleted Context Center file through the search API; repeatedly refilling a debounced input with the same value issued only one request
  • repeat the deferred-widget reveal step when a persona layout slot attaches just after the initial lookup

Why

The retained reports show deterministic test-state and locator races, not an EKS/RDS or database outage:

  • the same attachment, glossary, webhook, document, owner-picker, permission, and DAR signatures occurred in both MySQL and PostgreSQL
  • background editors remained mounted, eager requests completed before waiters were armed, and cached data prevented requests that tests assumed were mandatory
  • several tests selected the first global/hierarchy result rather than the entity they seeded
  • the n-gram test could set the already-persisted value and then wait forever for an enabled Save button
  • the Container certification failure waited on a missing Radio.Group after the controlled popover was remounted; a fresh open passed in both lanes
  • the Kind webhook callback was syntactically invalid when an IPv6 literal was unbracketed, and even a valid private IP literal is intentionally rejected by URLValidator; nightly Need icons for messaging services KAFKA and Pulsar #304 supplies a reachable DNS hostname instead
  • the deleted-folder trace contained only one search request immediately after deletion, proving the old UI-input loop was not polling the backend
  • the late-widget trace showed the correct server-side persona layout containing KnowledgePanel.Domains-85; its below-fold slot attached milliseconds after the helper's one-time check and therefore was never scrolled into view

Increasing retries only hid these causes. The strict reference keeps retries at zero and treats retry-dependent results as failures.

Validation

  • Prettier and git diff --check pass for all changed Playwright files
  • all changed TypeScript files parse successfully with esbuild
  • current OpenMetadata PR head: f923065d1f
  • current OpenMetadata integration source: 13914b10038f5c70672a79b12ac4139fea7c4bf9
  • initial no-build MySQL diagnostic (five assertions failed identically in all three repetitions): 32316620717
  • no-build PostgreSQL DAR/inbox control (42 focused UI executions and the migration suite all expected, zero retries): 32316620780
  • replacement no-build MySQL repeat run that isolated the last four deterministic test-oracle defects: 32318795210
  • corrected MySQL repeat run: exactly 12/12 focused UI executions expected, unexpected=0, flaky=0, retries 0: 32320942047
  • PostgreSQL remaining-signature run: seven repaired titles passed 3/3; Following exposed an unpersisted fallback layout before the final fix: 32319672728
  • Following persistence repeat run: exactly 3/3 focused UI executions expected, unexpected=0, flaky=0, retries 0: 32321463847
  • first unfiltered, two-database, zero-retry diagnostic: 32322104557. The strict gate exposed deterministic failures instead of masking them (17 PostgreSQL and 11 MySQL unexpected, 0 flaky); every signature was assigned to a focused repair or the separate DAR product bug.
  • Glossary invalid-character and Search Index checks, both databases: exactly 5 expected per lane, unexpected=0, flaky=0, retries 0: 32333769101
  • certification picker repair, both databases: exactly 24 expected per lane, 18 focused targets, unexpected=0, flaky=0, retries 0: 32334386291
  • latest combined unfiltered, two-database, three-worker, zero-retry diagnostic: 32337614256. PostgreSQL reported 7,618 expected / 12 unexpected / 0 flaky; MySQL reported 7,620 expected / 14 unexpected / 0 flaky. Its healthy API/database/OpenSearch telemetry ruled out a generic capacity collapse and supplied the focused failure set repaired here.
  • repaired Collate workflow-chain control 32346530819: 116 expected / 0 unexpected / 0 flaky on each database, retries 0
  • pipeline-permission correction 32348157701: 4 expected / 0 unexpected / 0 flaky on PostgreSQL, retries 0
  • exact deleted-document polling 32348732342: 4 expected / 0 unexpected / 0 flaky on MySQL, retries 0; the target completed in 10 seconds
  • 32348505752 confirmed the invalid-task correction and exposed the late persona-layout slot described above; the repeat-and-reveal correction is clean in 32350112186: 5 expected / 0 unexpected / 0 flaky, retries 0, and the formerly failing target completed in 8.1 seconds

The focused repairs are clean. The PR remains draft until the final unfiltered, two-database, zero-retry reference confirms the combined test and product image set.

Related work

AgentLogStream is covered by #31774, merged #31764, and #31769. The exact test bundle used by nightly also includes those commits. The DAR assignee-filter product bug is covered separately by #31795 and Collate #5925. Companion test/orchestration PRs: Collate #5919 and nightly #304.

Rohit0301 and others added 4 commits August 19, 2026 16:54
Auto-follow could resume two ways — the toolbar toggle or the user scrolling
back to the tail — and the two left different internal state. `pauseFollow()`
zeroes `viewerScrollAtRef`, and only `resumeFollowingTail()` re-armed it, so a
pause followed by a scroll back to the tail resumed following with no
viewer-scroll grace window. The pause decision then keyed on the
direction-agnostic `userMovedTheView`, so the next report in which the
library's own follow-scroll landed short of the tail — which a virtualised
list emits when a second append arrives mid-scroll — was read as the user
leaving the tail, and following paused itself.

Extract `beginFollowingFromTail()` and use it from both resume paths so they
cannot drift apart again, and gate the pause on a new direction-aware
`movedTowardsTail` fact: a view that travelled towards the tail and stopped
short of it is the viewer landing approximately, not the user leaving.

`movedTowardsTail` is positive evidence of direction rather than the absence
of `movedAwayFromTail`, so a first report — which has no previous offset to
compare against — is neither, preserving the existing "log opens already
scrolled up" behaviour.

Also mark the covering Playwright test slow: it contains four 60s polls
against a 60s budget and ran 55-57s locally, so it timed out on retry rather
than reporting the real failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added backend safe to test Add this label to run secure Github workflows on PRs labels Aug 19, 2026
harshach and others added 24 commits August 19, 2026 10:17
The "log content keeps growing" assertion budgeted 30s for the next batch of
lines, on the assumption that a live run writes at a steady rate. It does not:
the connector logs a burst, then goes silent for as long as its next phase
takes. The longest gap is between the connection test and the first topic
being ingested, where the Kafka consumer joins its group and blocks on an
empty poll without logging anything.

A measured CI run sat silent for 29.8s in that gap and the assertion gave up
0.2s before the next burst arrived — the budget was calibrated on top of the
gap it had to clear, so which side it landed on was a coin flip.

Widen the window to 90s. The test already carries test.slow() (180s) and ran
in 33s, so there is ample room. A stream that connects but delivers nothing
still fails: a healthy run does not go silent for 90s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comment thread openmetadata-ui/src/main/resources/ui/playwright/utils/polling.ts
…lity

# Conflicts:
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/ContextCenterArchive.spec.ts
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Pages/Glossary.spec.ts
…lity

# Conflicts:
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/OntologyExplorerFilters.spec.ts
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/OntologyExplorerIntegration.spec.ts
# Conflicts:
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/AgentLogStreamHandover.spec.ts
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/OntologyExplorerFilters.spec.ts
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/OntologyExplorerIntegration.spec.ts
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Pages/Glossary.spec.ts
#	openmetadata-ui/src/main/resources/ui/playwright/utils/polling.ts
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

⚠️ UI Checkstyle passed — lint findings in changed files

🔍 ESLint findings in this PR's files — 0 error(s), 6 warning(s)

Errors block the build. Warnings do not yet — they are rules whose backlog is still
being worked down, listed so this PR does not add to it. See docs/ui-code-quality-gate.md.

0 error(s), 6 warning(s) across 3 changed file(s).

Count Rule
2 sonarjs/cyclomatic-complexity
1 openmetadata-imports/no-lower-layer-page-imports
1 openmetadata-imports/no-internal-barrel-imports
1 react/no-array-index-key
1 sonarjs/no-duplicate-string
All findings
Location Rule Message
🟡 src/components/Explore/ExploreQuickFilters.tsx:23:1 openmetadata-imports/no-lower-layer-page-imports Pages are route-level composition modules. Move the shared implementation/type to a lower layer instead of importing a page from here.
🟡 src/rest/tasksAPI.ts:26:1 openmetadata-imports/no-internal-barrel-imports Import the internal module directly instead of its index barrel so unrelated siblings do not enter the bundle graph.
🟡 src/utils/FeedUtils.tsx:193:20 react/no-array-index-key Do not use Array index in keys
🟡 src/utils/FeedUtils.tsx:253:3 sonarjs/cyclomatic-complexity {"message":"Function has a complexity of 14 which is greater than 10 authorized.","cost":4,"secondaryLocations":[{"line":253,"column":2,"endLine":253,"endColumn
🟡 src/utils/FeedUtils.tsx:327:14 sonarjs/cyclomatic-complexity {"message":"Function has a complexity of 18 which is greater than 10 authorized.","cost":8,"secondaryLocations":[{"line":327,"column":13,"endLine":327,"endColum
🟡 src/utils/FeedUtils.tsx:365:23 sonarjs/no-duplicate-string Define a constant instead of duplicating this literal 5 times.

Fix locally (fast - only checks files changed in this branch):

make ui-checkstyle-changed

@gitar-bot

gitar-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown
Code Review 👍 Approved with suggestions 1 resolved / 2 findings

Stabilizes Playwright test oracles and shared AUT state across nightly database lanes by resolving timing races and explicit search targets. Consider removing the tautological assertion in the ContextCenterArticles pagination branch.

💡 Quality: Tautological assertion in eager-fetch pagination branch

📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/ContextCenterArticles.spec.ts:665-669

In ContextCenterArticles.spec.ts the if (initialCardCount > ARTICLE_LIST_PAGE_SIZE) branch asserts expect(initialCardCount).toBeGreaterThan(ARTICLE_LIST_PAGE_SIZE), which is guaranteed true by the branch condition itself and verifies nothing about pagination. In this case the test provides no real coverage that additional pages were fetched. Consider asserting a concrete post-condition (e.g. that the card count exceeds one page's worth of items, or that at least one paginated fetch was observed) so the branch still validates the eager-fetch behavior described in the comment.

✅ 1 resolved
Edge Case: Exact-match poll may miss entity beyond first 25 hits

📄 openmetadata-ui/src/main/resources/ui/playwright/utils/polling.ts:42-56
waitForSearchIndexed now scans only the first 25 hits (size=25) for an exact FQN/id/name match. Under a heavily populated AUT index with many similarly-named entities, the newly-created entity could rank below position 25 and never be found, causing a spurious timeout instead of the race it aims to fix. Since the query is q=entityFqn, relevance ranking usually surfaces the exact match near the top, so this is low risk, but consider sorting/filtering server-side (e.g. a term filter on fullyQualifiedName) rather than relying on ranking within a fixed window.

🤖 Prompt for agents
Code Review: Stabilizes Playwright test oracles and shared AUT state across nightly database lanes by resolving timing races and explicit search targets. Consider removing the tautological assertion in the ContextCenterArticles pagination branch.

1. 💡 Quality: Tautological assertion in eager-fetch pagination branch
   Files: openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/ContextCenterArticles.spec.ts:665-669

   In ContextCenterArticles.spec.ts the `if (initialCardCount > ARTICLE_LIST_PAGE_SIZE)` branch asserts `expect(initialCardCount).toBeGreaterThan(ARTICLE_LIST_PAGE_SIZE)`, which is guaranteed true by the branch condition itself and verifies nothing about pagination. In this case the test provides no real coverage that additional pages were fetched. Consider asserting a concrete post-condition (e.g. that the card count exceeds one page's worth of items, or that at least one paginated fetch was observed) so the branch still validates the eager-fetch behavior described in the comment.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants