Repository navigation
feat(conformance): add error_responses and adapter suites and run adapter and accessibility conformance in all renderers - #3583
Conversation
There was a problem hiding this comment.
Code Review
This pull request significantly expands the A2UI conformance test suite by introducing new test files (adapter_conformance.yaml and error_responses.yaml), updating existing test cases to support v1.0 features, and refining the conformance schema. It also updates the Python test runner to validate renderer error payloads against JSON schemas, and updates the Dart and Swift test harnesses to execute the new test suites. Feedback on the Swift test harness identifies a bug in effectiveLabel(for:) where an empty accessibility label string fails to fall back to the child's text.
b776a4f to
d3de786
Compare
c25798f to
2b00d8f
Compare
d3de786 to
edd62ed
Compare
2b00d8f to
0aef4f0
Compare
0aef4f0 to
c882ba7
Compare
0b813ba to
b8470a8
Compare
c882ba7 to
09a3573
Compare
65758c8 to
cb214b9
Compare
b8470a8 to
0568ec1
Compare
0568ec1 to
f34f91a
Compare
896e925 to
30b8423
Compare
a646a26 to
bcd36f6
Compare
30b8423 to
1610181
Compare
bcd36f6 to
f235f2b
Compare
1610181 to
cebcb10
Compare
…C tests - Deduplicate catalog resolution logic and spell out local variable names - Refactor recursion and validation preconditions to guard statements - Extract functionCallID and isCallRendererFunction in MessageProcessor entry points - Spell out envelope processing variable names and optimize Set insertion - Nest RendererError inside ConformanceExpectError - Deterministically synchronize outbound RPC calls with continuations instead of Task.sleep
…ion and makeFunctionResolvedAction
…ssor and conformance tests
0ba09ba to
4e2b76a
Compare
…onformance-error-and-renderer-suites
…and node_resolver
…y' into feat/conformance-error-and-renderer-suites
…ser tests Use "1.0" instead of "v1.0" in _v10_create and _V10_BOGUS_ROOT fixtures to match the v1.0 specification and RendererState requirements.
…y' into feat/conformance-error-and-renderer-suites
…ror-and-renderer-suites
…ore SDKs
Keep A2UISwiftUI's public API surface unchanged from main by moving view-tree inspection, accessibility heuristics, and Combine subscription tracking into AdapterConformanceTests.
Support @-prefixed system function calls such as ${@index()} in TypeScript and Swift ExpressionParser implementations, and reject bare @Identifiers or mid-path @ characters so adapter_conformance.yaml and renderer harnesses run without test-only catalog wrappers.
Give explicit CheckRule.message precedence on evaluated checks in Dart GenericBinder while preserving data-model messages on path-bound checks, tighten Lit and Flutter validationErrors assertions and axe checks, clear detached _controllerNode references in React, update conformance_schema.json and README.md, and document the new suites in conformance_harness_implementation_guide.md.
| private func assertModifierMatches( | ||
| _ modifier: A2UIAccessibilityModifier, | ||
| node: Node, | ||
| expected: [String: Any], | ||
| context: String | ||
| ) { | ||
| if let expLabel = expected["label"] as? String { | ||
| #expect(effectiveLabel(for: node, modifier: modifier) == expLabel, "\(context): label") | ||
| } | ||
| if let expDesc = expected["description"] as? String { | ||
| #expect(modifier.hint == expDesc, "\(context): description mismatch") | ||
| } | ||
| if let expLive = expected["live"] as? String { | ||
| #expect(node.explicitAccessibilityAttributes?.live == expLive, "\(context): live mismatch") | ||
| } | ||
| if let expHidden = expected["hidden"] as? Bool { | ||
| #expect(modifier.isHidden == expHidden, "\(context): hidden mismatch") | ||
| } | ||
| if let expRole = expected["role"] as? String { | ||
| #expect(semanticRole(for: node) == expRole, "\(context): role mismatch") | ||
| } | ||
| if let expChecked = expected["checked"] as? Bool { | ||
| #expect(checkedState(for: node) == expChecked, "\(context): checked mismatch") | ||
| } | ||
| } | ||
|
|
||
| private func effectiveLabel(for node: Node, modifier: A2UIAccessibilityModifier) -> String? { | ||
| if let explicit = modifier.label { | ||
| return explicit | ||
| } | ||
| if let inferred = node.accessibilityAttributes?.label, !inferred.isEmpty { | ||
| return inferred | ||
| } | ||
| if let child = node.child(for: "child") { | ||
| return child.accessibilityAttributes?.label ?? child.string(for: "text") | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| private func semanticRole(for node: Node) -> String { | ||
| switch node.type { | ||
| case "Button": | ||
| return "button" | ||
| case "TextField": | ||
| return "textbox" | ||
| case "CheckBox": | ||
| return "checkbox" | ||
| case "ChoicePicker": | ||
| let variant = node.string(for: "variant") ?? "mutuallyExclusive" | ||
| return variant == "mutuallyExclusive" ? "radiogroup" : "group" | ||
| default: | ||
| return node.type.lowercased() | ||
| } | ||
| } | ||
|
|
||
| private func checkedState(for node: Node) -> Bool? { | ||
| guard node.type == "CheckBox" else { return nil } | ||
| return node.bool(for: "value") ?? false | ||
| } |
There was a problem hiding this comment.
[MEDIUM] SwiftUI accessibility conformance assertions bypass A2UISwiftUI views and use hardcoded Node helpers
assertAccessibilityNodes (lines 224–283) and assertAccessibilityTreeMatches (lines 978–1000) in AdapterConformanceTests.swift do not instantiate or inspect A2UISwiftUI views (Surface, ComponentNodeView, A2UIButton, A2UITextField, A2UICheckBox, A2UIChoicePicker). Instead, the test runner computes roles, checked states, and fallback labels directly from A2UICore.Node in test helpers:
semanticRole(for:)(lines 298–312) is a hardcodedswitch node.typetable in the test file.checkedState(for:)(lines 314–317) readsnode.bool(for: "value")directly from theNode.effectiveLabel(for:modifier:)(lines 285–296) re-implements label fallback resolution in test code (node.accessibilityAttributes?.labelandnode.child(for: "child")), bypassing.a2uiAccessibilityControl(for: node, defaultLabel: labelText)used byA2UITextField,A2UICheckBox, andA2UIChoicePicker.
🤖 Agent-generated
There was a problem hiding this comment.
SwiftUI has no runtime accessibility tree that a unit test can read without a view inspection dependency, so these helpers mirror the label, role, and checked-state rules the A2UISwiftUI views apply through a2uiAccessibilityControl. The earlier revision exposed them on A2UIAccessibilityModifier and review asked for that public surface to go. If you would rather not have mirrored logic in the test, the alternative is to record the accessibility cases as expected failures for SwiftUI; let me know which you prefer.
🤖 Agent-generated
…mance tests - Strip numeric repeater instance suffixes in AdapterConformanceTests.baseComponentID when the base component ID exists in the surface's ComponentsModel. - Match component-property pointer prefixes in ConformanceTestHelper.rendererErrorPathMatches when the expected rendererError path points to the enclosing component entry.
Summary
Add shared conformance suites for renderer-to-agent error responses and UI adapters, and expand existing suites under
conformance/core/. Run the accessibility and adapter suites across@a2ui/lit,@a2ui/react,@a2ui/angular,a2ui_flutter, andA2UISwiftUI.Reviewer Focus
conformance/core/adapter_conformance.yamlandconformance/core/accessibility.yamlfirst. They define the declarative UI expectations that each renderer test runner executes mechanically.conformance/core/error_responses.yamlandvalidate_renderer_error_payloadinconformance/tests/test_conformance_yaml.py. Together they check every expected error payload against the published v0.9 and v1.0 wire schemas.dart/a2ui_flutter/lib/src/node_view.dart,typescript/renderers/react/src/A2uiSurface.tsx,typescript/renderers/react/src/web_component_node.tsx, andswift/swiftui/Sources/BasicCatalog/.adapter_conformancetest files (~4,000lines combined) once the YAML definitions look right. Each runner maps the same step operations (process_messages,user_input,user_gesture,programmatic_invoke,unmount_surface) onto its UI framework.Changes
conformance/conformance_schema.jsonandconformance/tests/test_conformance_yaml.pywith therender_surfaceaction,userActivationflags, andExpectErrorfields (path,surfaceId,functionCallId,rendererError).conformance/core/error_responses.yaml(23 error cases and 1 non-strict buffered streaming case) andconformance/core/adapter_conformance.yaml(13 UI adapter cases), and updateconformance/core/accessibility.yamlto v1.0 schemas. Expandnode_resolution.yaml,validation_result.yaml,actions.yaml,rpc_functions.yaml,composition_constraints.yaml,validator_v0_9.yaml,validator_v1_0.yaml,message_processor_v0_9.yaml, andmessage_processor_v1_0.yaml. Document the new suites and adapter runner contracts inconformance/README.mdandconformance/conformance_harness_implementation_guide.md.@a2ui/web_core(expression_parser.ts) andswift/core(ExpressionParser.swift) to parse${@index()}, and updatedart/a2ui_core(generic_binder.dart,validation_result.dart) so explicitCheckRule.messageoverrides function defaults on evaluated checks. Update the conformance runners in@a2ui/web_core(conformance_test.mjs,node-resolution.ts,accessibility_conformance.test.ts),dart/a2ui_core(conformance_harness.dart,message_processor_conformance_test.dart,validator_conformance_test.dart), andswift/core(ConformanceTestHelper.swift,ValidatorConformanceTests.swift).@a2ui/lit(a2ui-surface.tsrootNodegetter andadapter_conformance.test.ts) and@a2ui/angular(karma.conf.jsandadapter_conformance.spec.ts). In@a2ui/react, prepare catalogs oncomponentsModel.onCreatedinA2uiSurface.tsx, dispose detached element controllers and clear_controllerNodeinweb_component_node.tsx, and addadapter_conformance.test.tsx.NodeViewindart/a2ui_flutter/lib/src/node_view.dartto data-boundaccessibilitysignals (label,description,live,hidden) so Flutter semantics update reactively. Adddart/a2ui_flutter/test/adapter_conformance_test.dart.A2UISwiftUIbasic catalog views inA2UITextField.swiftandA2UIButton.swiftforwarningorinfovalidation captions andrequiresUserActivationmetadata, while keepingSurfaceandA2UIAccessibilityModifierunchanged. AddAdapterConformanceTests.swift.Impact & Risks
A2uiSurface.rootNodein@a2ui/lit,ValidationResult.fromEvaluation(explicitMessage:)ina2ui_core, and read-only validation or activation properties onA2UITextFieldandA2UIButtoninA2UISwiftUI) are additive.main. This branch now merges directly ontomain.A2UISwiftUITestsrequires SwiftUI on Apple platforms and runs on macOS CI (swift_build_and_test.yml).Testing
@a2ui/web_coreconformance and the adapter tests for the web renderers:Review history for consolidated branches: #3584 (web renderers), #3586 (Flutter), #3587 (SwiftUI).
🤖 Agent-generated