Skip to content

feat(conformance): add error_responses and adapter suites and run adapter and accessibility conformance in all renderers - #3583

Merged
gspencergoog merged 83 commits into
mainfrom
feat/conformance-error-and-renderer-suites
Oct 30, 2026
Merged

gspencergoog merged 83 commits into
mainfrom
feat/conformance-error-and-renderer-suites

Conversation

@gspencergoog

@gspencergoog gspencergoog commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Add shared conformance suites for renderer-to-agent error responses and UI adapters, and expand existing suites under conformance/core/. Run the accessibility and adapter suites across @a2ui/lit, @a2ui/react, @a2ui/angular, a2ui_flutter, and A2UISwiftUI.

Reviewer Focus

  1. Check conformance/core/adapter_conformance.yaml and conformance/core/accessibility.yaml first. They define the declarative UI expectations that each renderer test runner executes mechanically.
  2. Check conformance/core/error_responses.yaml and validate_renderer_error_payload in conformance/tests/test_conformance_yaml.py. Together they check every expected error payload against the published v0.9 and v1.0 wire schemas.
  3. Inspect the runtime adapter changes in dart/a2ui_flutter/lib/src/node_view.dart, typescript/renderers/react/src/A2uiSurface.tsx, typescript/renderers/react/src/web_component_node.tsx, and swift/swiftui/Sources/BasicCatalog/.
  4. Skim the five adapter_conformance test files (~4,000 lines combined) once the YAML definitions look right. Each runner maps the same step operations (process_messages, user_input, user_gesture, programmatic_invoke, unmount_surface) onto its UI framework.

Changes

  1. Extend conformance/conformance_schema.json and conformance/tests/test_conformance_yaml.py with the render_surface action, userActivation flags, and ExpectError fields (path, surfaceId, functionCallId, rendererError).
  2. Add conformance/core/error_responses.yaml (23 error cases and 1 non-strict buffered streaming case) and conformance/core/adapter_conformance.yaml (13 UI adapter cases), and update conformance/core/accessibility.yaml to v1.0 schemas. Expand node_resolution.yaml, validation_result.yaml, actions.yaml, rpc_functions.yaml, composition_constraints.yaml, validator_v0_9.yaml, validator_v1_0.yaml, message_processor_v0_9.yaml, and message_processor_v1_0.yaml. Document the new suites and adapter runner contracts in conformance/README.md and conformance/conformance_harness_implementation_guide.md.
  3. Update @a2ui/web_core (expression_parser.ts) and swift/core (ExpressionParser.swift) to parse ${@index()}, and update dart/a2ui_core (generic_binder.dart, validation_result.dart) so explicit CheckRule.message overrides function defaults on evaluated checks. Update the conformance runners in @a2ui/web_core (conformance_test.mjs, node-resolution.ts, accessibility_conformance.test.ts), dart/a2ui_core (conformance_harness.dart, message_processor_conformance_test.dart, validator_conformance_test.dart), and swift/core (ConformanceTestHelper.swift, ValidatorConformanceTests.swift).
  4. Add conformance tests to @a2ui/lit (a2ui-surface.ts rootNode getter and adapter_conformance.test.ts) and @a2ui/angular (karma.conf.js and adapter_conformance.spec.ts). In @a2ui/react, prepare catalogs on componentsModel.onCreated in A2uiSurface.tsx, dispose detached element controllers and clear _controllerNode in web_component_node.tsx, and add adapter_conformance.test.tsx.
  5. Subscribe NodeView in dart/a2ui_flutter/lib/src/node_view.dart to data-bound accessibility signals (label, description, live, hidden) so Flutter semantics update reactively. Add dart/a2ui_flutter/test/adapter_conformance_test.dart.
  6. Update A2UISwiftUI basic catalog views in A2UITextField.swift and A2UIButton.swift for warning or info validation captions and requiresUserActivation metadata, while keeping Surface and A2UIAccessibilityModifier unchanged. Add AdapterConformanceTests.swift.

Impact & Risks

  1. No breaking API changes. Public surface additions (A2uiSurface.rootNode in @a2ui/lit, ValidationResult.fromEvaluation(explicitMessage:) in a2ui_core, and read-only validation or activation properties on A2UITextField and A2UIButton in A2UISwiftUI) are additive.
  2. The core prerequisites (fix(core): coerce ValidationResult to DynamicBoolean arguments from the schema #3653, feat(web_core): add MessageErrorMapper, JSON Pointer error paths, and conformance parity #3579, feat(python-core): add message_error_mapper, NodeResolver, and conformance parity #3580, feat(dart-core): add MessageErrorMapper, validator error details, and conformance parity #3581, feat(swift-core): add MessageErrorMapper envelope handling, JSON Pointer paths, and conformance runners #3582) and consolidated web renderer changes (test(renderers): add UI adapter and accessibility conformance for Lit, React, and Angular #3584) have merged into main. This branch now merges directly onto main.
  3. A2UISwiftUITests requires SwiftUI on Apple platforms and runs on macOS CI (swift_build_and_test.yml).

Testing

  1. Check conformance YAML schemas and run Python core conformance:
    uv run --directory python/a2ui_core pytest ../../conformance/tests/test_conformance_yaml.py
    uv run --directory python/a2ui_core pytest
  2. Run @a2ui/web_core conformance and the adapter tests for the web renderers:
    yarn --cwd typescript/core/web_core test:conformance
    yarn --cwd typescript/renderers/lit test:unit
    yarn --cwd typescript/renderers/react test:unit
    yarn --cwd typescript/renderers/angular test:unit
  3. Run Dart core conformance and Flutter adapter tests:
    cd dart/a2ui_core && dart analyze --fatal-infos && dart test
    cd dart/a2ui_flutter && flutter analyze && flutter test
  4. Build and run Swift core and SwiftUI conformance suites (SwiftUI tests run on macOS CI):
    swift test --filter A2UIConformanceTests
    swift test --filter A2UISwiftUITests

Review history for consolidated branches: #3584 (web renderers), #3586 (Flutter), #3587 (SwiftUI).

🤖 Agent-generated

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request significantly expands the A2UI conformance test suite by introducing new test files (adapter_conformance.yaml and error_responses.yaml), updating existing test cases to support v1.0 features, and refining the conformance schema. It also updates the Python test runner to validate renderer error payloads against JSON schemas, and updates the Dart and Swift test harnesses to execute the new test suites. Feedback on the Swift test harness identifies a bug in effectiveLabel(for:) where an empty accessibility label string fails to fall back to the child's text.

@gspencergoog
gspencergoog force-pushed the feat/swift-core-conformance-parity branch from b776a4f to d3de786 Compare October 23, 2026 23:56
@gspencergoog
gspencergoog force-pushed the feat/conformance-error-and-renderer-suites branch from c25798f to 2b00d8f Compare October 23, 2026 23:56
@gspencergoog
gspencergoog force-pushed the feat/swift-core-conformance-parity branch from d3de786 to edd62ed Compare October 27, 2026 15:58
@gspencergoog
gspencergoog force-pushed the feat/conformance-error-and-renderer-suites branch from 2b00d8f to 0aef4f0 Compare October 27, 2026 15:58
@gspencergoog
gspencergoog removed this pull request from stack #3588 October 27, 2026 16:00
@gspencergoog
gspencergoog added this pull request to stack #3654 October 27, 2026 16:00
@gspencergoog
gspencergoog force-pushed the feat/conformance-error-and-renderer-suites branch from 0aef4f0 to c882ba7 Compare October 27, 2026 16:02
@gspencergoog
gspencergoog force-pushed the feat/swift-core-conformance-parity branch from 0b813ba to b8470a8 Compare October 27, 2026 16:24
@gspencergoog
gspencergoog force-pushed the feat/conformance-error-and-renderer-suites branch from c882ba7 to 09a3573 Compare October 27, 2026 16:24
@gspencergoog
gspencergoog removed this pull request from stack #3654 October 27, 2026 17:10
@gspencergoog
gspencergoog added this pull request to stack #3655 October 27, 2026 17:12
@gspencergoog gspencergoog changed the title feat(conformance): add error_responses.yaml, adapter_conformance.yaml, and expanded core suites feat(conformance): add error_responses and adapter suites and run adapter and accessibility conformance in all renderers Oct 27, 2026
@gspencergoog
gspencergoog force-pushed the feat/conformance-error-and-renderer-suites branch from 65758c8 to cb214b9 Compare October 27, 2026 17:18
@gspencergoog
gspencergoog force-pushed the feat/swift-core-conformance-parity branch from b8470a8 to 0568ec1 Compare October 27, 2026 17:18
@gspencergoog
gspencergoog marked this pull request as ready for review October 27, 2026 19:32
@gspencergoog
gspencergoog force-pushed the feat/swift-core-conformance-parity branch from 0568ec1 to f34f91a Compare October 27, 2026 19:57
@gspencergoog
gspencergoog force-pushed the feat/conformance-error-and-renderer-suites branch 4 times, most recently from 896e925 to 30b8423 Compare October 27, 2026 20:28
@gspencergoog
gspencergoog force-pushed the feat/swift-core-conformance-parity branch from a646a26 to bcd36f6 Compare October 27, 2026 20:56
@gspencergoog
gspencergoog force-pushed the feat/conformance-error-and-renderer-suites branch from 30b8423 to 1610181 Compare October 27, 2026 20:56
@gspencergoog
gspencergoog force-pushed the feat/swift-core-conformance-parity branch from bcd36f6 to f235f2b Compare October 27, 2026 21:34
@gspencergoog
gspencergoog force-pushed the feat/conformance-error-and-renderer-suites branch from 1610181 to cebcb10 Compare October 27, 2026 21:34
…C tests

- Deduplicate catalog resolution logic and spell out local variable names
- Refactor recursion and validation preconditions to guard statements
- Extract functionCallID and isCallRendererFunction in MessageProcessor entry points
- Spell out envelope processing variable names and optimize Set insertion
- Nest RendererError inside ConformanceExpectError
- Deterministically synchronize outbound RPC calls with continuations instead of Task.sleep
@gspencergoog
gspencergoog force-pushed the feat/swift-core-conformance-parity branch from 0ba09ba to 4e2b76a Compare October 29, 2026 17:38
Base automatically changed from feat/swift-core-conformance-parity to main October 29, 2026 17:52
…ser tests

Use "1.0" instead of "v1.0" in _v10_create and _V10_BOGUS_ROOT fixtures
to match the v1.0 specification and RendererState requirements.
…y' into feat/conformance-error-and-renderer-suites
…ore SDKs

Keep A2UISwiftUI's public API surface unchanged from main by moving view-tree inspection, accessibility heuristics, and Combine subscription tracking into AdapterConformanceTests.

Support @-prefixed system function calls such as ${@index()} in TypeScript and Swift ExpressionParser implementations, and reject bare @Identifiers or mid-path @ characters so adapter_conformance.yaml and renderer harnesses run without test-only catalog wrappers.

Give explicit CheckRule.message precedence on evaluated checks in Dart GenericBinder while preserving data-model messages on path-bound checks, tighten Lit and Flutter validationErrors assertions and axe checks, clear detached _controllerNode references in React, update conformance_schema.json and README.md, and document the new suites in conformance_harness_implementation_guide.md.

@nan-yu nan-yu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed PR #3583 (ef0f8e5f). See inline comments for high and medium severity findings.

🤖 Agent-generated

Comment thread conformance/core/error_responses.yaml Resolved
Comment thread conformance/core/rpc_functions.yaml Resolved
Comment on lines +259 to +317
private func assertModifierMatches(
_ modifier: A2UIAccessibilityModifier,
node: Node,
expected: [String: Any],
context: String
) {
if let expLabel = expected["label"] as? String {
#expect(effectiveLabel(for: node, modifier: modifier) == expLabel, "\(context): label")
}
if let expDesc = expected["description"] as? String {
#expect(modifier.hint == expDesc, "\(context): description mismatch")
}
if let expLive = expected["live"] as? String {
#expect(node.explicitAccessibilityAttributes?.live == expLive, "\(context): live mismatch")
}
if let expHidden = expected["hidden"] as? Bool {
#expect(modifier.isHidden == expHidden, "\(context): hidden mismatch")
}
if let expRole = expected["role"] as? String {
#expect(semanticRole(for: node) == expRole, "\(context): role mismatch")
}
if let expChecked = expected["checked"] as? Bool {
#expect(checkedState(for: node) == expChecked, "\(context): checked mismatch")
}
}

private func effectiveLabel(for node: Node, modifier: A2UIAccessibilityModifier) -> String? {
if let explicit = modifier.label {
return explicit
}
if let inferred = node.accessibilityAttributes?.label, !inferred.isEmpty {
return inferred
}
if let child = node.child(for: "child") {
return child.accessibilityAttributes?.label ?? child.string(for: "text")
}
return nil
}

private func semanticRole(for node: Node) -> String {
switch node.type {
case "Button":
return "button"
case "TextField":
return "textbox"
case "CheckBox":
return "checkbox"
case "ChoicePicker":
let variant = node.string(for: "variant") ?? "mutuallyExclusive"
return variant == "mutuallyExclusive" ? "radiogroup" : "group"
default:
return node.type.lowercased()
}
}

private func checkedState(for node: Node) -> Bool? {
guard node.type == "CheckBox" else { return nil }
return node.bool(for: "value") ?? false
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] SwiftUI accessibility conformance assertions bypass A2UISwiftUI views and use hardcoded Node helpers

assertAccessibilityNodes (lines 224–283) and assertAccessibilityTreeMatches (lines 978–1000) in AdapterConformanceTests.swift do not instantiate or inspect A2UISwiftUI views (Surface, ComponentNodeView, A2UIButton, A2UITextField, A2UICheckBox, A2UIChoicePicker). Instead, the test runner computes roles, checked states, and fallback labels directly from A2UICore.Node in test helpers:

  • semanticRole(for:) (lines 298–312) is a hardcoded switch node.type table in the test file.
  • checkedState(for:) (lines 314–317) reads node.bool(for: "value") directly from the Node.
  • effectiveLabel(for:modifier:) (lines 285–296) re-implements label fallback resolution in test code (node.accessibilityAttributes?.label and node.child(for: "child")), bypassing .a2uiAccessibilityControl(for: node, defaultLabel: labelText) used by A2UITextField, A2UICheckBox, and A2UIChoicePicker.

🤖 Agent-generated

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SwiftUI has no runtime accessibility tree that a unit test can read without a view inspection dependency, so these helpers mirror the label, role, and checked-state rules the A2UISwiftUI views apply through a2uiAccessibilityControl. The earlier revision exposed them on A2UIAccessibilityModifier and review asked for that public surface to go. If you would rather not have mirrored logic in the test, the alternative is to record the accessibility cases as expected failures for SwiftUI; let me know which you prefer.

🤖 Agent-generated

…mance tests

- Strip numeric repeater instance suffixes in AdapterConformanceTests.baseComponentID
  when the base component ID exists in the surface's ComponentsModel.
- Match component-property pointer prefixes in ConformanceTestHelper.rendererErrorPathMatches
  when the expected rendererError path points to the enclosing component entry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants