Skip to content

Strip the XCUIElementType prefix when normalizing mobile node kinds (upstream 1133) - #659

Merged
kblok merged 2 commits into
mainfrom
issue-636-xcuielementtype-prefix
Oct 25, 2026
Merged

kblok merged 2 commits into
mainfrom
issue-636-xcuielementtype-prefix

Conversation

@kblok

@kblok kblok commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

iOS sessions on agent-device's WebDriver runtime (hosted Appium clouds) report XCUIElementTypeButton where the native runner reports Button, so roles resolved to nothing and a shown keyboard was not detected. SnapshotProjection.NormalizeKind now drops ^XCUIElementType(?=[A-Z]) before mapping the type, and DeviceSession.KeyboardShowing routes node types through NormalizeKind. This is reachable through a custom IPageSourceConverter on E2E.TestMu.Mobile.

Ports tester-army/e2e#1133 (commit 3f9a7d8): packages/mobile/src/nodes.ts (normalizeKind), packages/mobile/src/surface.ts (keyboardShowing), and the tests in tests/unit/nodes.test.ts and tests/unit/engine.test.ts.

Closes #636

Tests ported

  • Nodes/SnapshotProjectionTests: Reads_the_XCUIElementType_class_names_a_WebDriver_session_reports_as_the_native_XCTest_types (nodes.test.ts, full).
  • Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests: Reads_an_XCUIElementTypeKeyboard_from_a_WebDriver_session_as_a_shown_keyboard (engine.test.ts, full).

Skipped / partial

  • Skipped: the upstream changeset file (.changeset/...); the port has no changesets.
  • Partial tests: none. Tests with no upstream it: none.

Port-only symbols

  • SnapshotProjection.XcuiElementTypePrefix(): a .NET [GeneratedRegex] for upstream's inline /^XCUIElementType(?=[A-Z])/ (the port's regexes are all source-generated).
  • NormalizeKind stays internal (upstream exports it).

Minor decisions

  • No public API or user-visible docs change, so SKILL.md, docs/ and COMPATIBILITY.md are untouched.
  • Real-model tests not run: the change is a pure string normalization in the mobile snapshot projection, covered by the unit tests, with no agent, prompt, cache or replay path touched. The mobile test project shows only the known Windows CRLF failures (58, in captured-snapshot comparisons); the touched classes pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DH1b4gwq2kmMK94pdZRQZm

Summary by CodeRabbit

  • Bug Fixes
    • Keyboard detection now recognizes both standard and XCUIElementType-prefixed element names, allowing keyboards without a dismiss key to be dismissed with a swipe.
    • Native element names with the prefix are projected consistently with their unprefixed equivalents, including matching element kinds and roles. This also preserves the expected viewport dimensions when processing prefixed element names.

…upstream #1133)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DH1b4gwq2kmMK94pdZRQZm
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between c61a644 and 254e778.

Files selected for processing: 1
  • COMPATIBILITY.md

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

Walkthrough

The change strips the XCUIElementType prefix during mobile node-kind normalization. Keyboard detection now uses the normalized kind. Tests cover projected kinds and roles, viewport dimensions, and keyboard dismissal by swipe.

Changes: Mobile Kind Normalization

Layer / File(s) Summary
Normalize prefixed XCTest kinds
src/E2E.Mobile/SnapshotProjection.cs, tests/E2E.Mobile.Tests/Nodes/SnapshotProjectionTests.cs, COMPATIBILITY.md
NormalizeKind strips the XCUIElementType prefix before existing normalization. Tests compare projected kinds and roles for prefixed and native type names, and verify the prefixed input retains a 390×844 viewport. Compatibility notes describe the normalization and tests.
Use normalized kinds for keyboard detection
src/E2E.Mobile/DeviceSession.Screen.cs, tests/E2E.Mobile.Tests/Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests.cs
KeyboardShowing uses NormalizeKind to match keyboard types. A test verifies that dismissal swipes when the snapshot contains an XCUIElementTypeKeyboard node without an explicit keyboard fact.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Possibly related PRs

  • hardkoded/e2e-dotnet#596: It changes viewport and drag coordinates for keyboard dismissal after rotation. This PR changes keyboard detection that enables the dismissal swipe.

Suggested reviewers: cursoragent

Merge Risk: Minimal · up to 254e7

No actionable merge-blocking risk was identified in the reviewed changes.

Architecture risk: Low · up to 254e7

The change affects 3 systems.

Changed systems: src, tests, COMPATIBILITY.md

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — src (service) was modified; 2 changed files map to changed impact.
  • observed — tests (service) was modified; 2 changed files map to changed impact.
  • observed — COMPATIBILITY.md (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in src/E2E.Mobile/DeviceSession.Screen.cs: KeyboardShowing now uses SnapshotProjection.NormalizeKind to normalize node types before matching keyboard types, instead of lowercasing the raw type.
  • observed — Modified behavior in src/E2E.Mobile/SnapshotProjection.cs: NormalizeKind now strips an XCUIElementType prefix before applying its existing word-boundary, separator, and lowercase normalization; prefixed XCTest element types therefore normalize consistently with unprefixed names.
  • observed — Modified behavior in src/E2E.Mobile/SnapshotProjection.cs: Added the generated regex used by NormalizeKind to recognize XCUIElementType only when followed by an uppercase letter.
  • observed — Modified behavior in tests/E2E.Mobile.Tests/Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests.cs: Added a test that supplies an XCUIElementTypeKeyboard node without an explicit keyboard fact, invokes keyboard dismissal, and asserts that one swipe occurs.

Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

Failed Failed checks: 1 error

Check name Status Explanation Resolution
Upstream Parity Error Fails point 4. The diff changes user-visible behavior in SnapshotProjection.NormalizeKind and DeviceSession.KeyboardShowing, matching the upstream behavior identified in the PR description and `CO… Update skills/e2e-dotnet/SKILL.md and applicable docs/ to document the XCUIElementType normalization and WebDriver keyboard detection using the C# names, and retain the COMPATIBILITY.md entry.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly summarizes the main change: removing the XCUIElementType prefix when normalizing mobile node kinds. The upstream reference does not make it unclear.
Linked Issues check Passed Issue #636 requires XCUIElementType prefix normalization, normalized keyboard detection, and ported tests. SnapshotProjection.NormalizeKind removes the prefix, and DeviceSession.KeyboardShowing …
Out of Scope Changes check Passed The reported source changes and tests implement issue #636. The COMPATIBILITY.md entry documents this same fix. No unrelated changes are identified in the supplied change summary.
Full details: Linked Issues check

Explanation

Issue #636 requires XCUIElementType prefix normalization, normalized keyboard detection, and ported tests. SnapshotProjection.NormalizeKind removes the prefix, and DeviceSession.KeyboardShowing uses that normalization. The reported tests cover prefixed node types and roles, plus XCUIElementTypeKeyboard detection.

Full details: Upstream Parity

Explanation

Fails point 4. The diff changes user-visible behavior in SnapshotProjection.NormalizeKind and DeviceSession.KeyboardShowing, matching the upstream behavior identified in the PR description and COMPATIBILITY.md. The PR adds the compatibility entry, but the changed-file inventory contains no update to skills/e2e-dotnet/SKILL.md or docs/. The PR’s claim that those files have no upstream wording does not meet the check’s stated documentation requirement. Upstream source inspection was unavailable, so this assessment uses the PR text and compatibility entry as directed.

Resolution

Update skills/e2e-dotnet/SKILL.md and applicable docs/ to document the XCUIElementType normalization and WebDriver keyboard detection using the C# names, and retain the COMPATIBILITY.md entry.

Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Linked repositories: skipped tester-army/e2e. Repositories added from the pull request branch configuration require the pull request author (and the user who triggered the review, when different) to have read access to them.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-merge checks failed. Please resolve the failing checks before merging.

…m #1133)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DH1b4gwq2kmMK94pdZRQZm
@kblok

kblok commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

Re "Upstream Parity" point 4 (SKILL.md, docs/ and COMPATIBILITY.md updates absent):

SnapshotProjection.NormalizeKind now maps XCUIElementType... names and DeviceSession.KeyboardShowing now detects WebDriver keyboards. These match upstream nodes.ts (normalizeKind) and surface.ts (keyboardShowing) from 3f9a7d8.

I added the COMPATIBILITY.md entry (in the mobile section, next to the known-viewport and keyboard-drag entries). I did not touch skills/e2e-dotnet/SKILL.md or docs/: upstream commit 3f9a7d8 changes only nodes.ts, surface.ts, the two unit tests, a changeset and the CHANGELOG. Its skills/e2e/references/*.md and docs/**/*.mdx say nothing about XCUIElementType names or WebDriver keyboard detection, and the port adds no public API, so there is nothing to mirror. The COMPATIBILITY.md entry says the same.

@kblok

kblok commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-merge checks failed. Please resolve the failing checks before merging.

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@kblok
kblok merged commit 55a3c65 into main Oct 25, 2026
3 checks passed
@kblok
kblok deleted the issue-636-xcuielementtype-prefix branch October 25, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Strip the XCUIElementType prefix when normalizing mobile node kinds

1 participant