Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
72f7668
feat(web_core): add MessageErrorMapper, JSON Pointer error paths, and…
gspencergoog Oct 23, 2026
02dc30d
fix(web_core): use Object.hasOwn for property checks
gspencergoog Oct 23, 2026
7368c1c
fix(web_core): avoid duplicate nested function validation on componen…
gspencergoog Oct 28, 2026
74bd367
fix(web_core): address review feedback on error mapping and conforman…
gspencergoog Oct 28, 2026
5a92773
fix(web_core): format files and align lit explorer form validator exp…
gspencergoog Oct 29, 2026
05bcd94
fix(web_core): resolve action promises and align validation test expe…
gspencergoog Oct 29, 2026
5869a38
feat(python-core): add message_error_mapper, NodeResolver, and confor…
gspencergoog Oct 23, 2026
f102788
fix(a2ui-core): address review feedback and format with pyink
gspencergoog Oct 23, 2026
8933bcb
fix(python-core): satisfy mypy type checking in message_error_mapper …
gspencergoog Oct 27, 2026
1e3bcd9
feat(dart-core): add MessageErrorMapper, validator error details, and…
gspencergoog Oct 23, 2026
46d7277
fix(dart-core): add explicit loop variable type and align flutter reo…
gspencergoog Oct 23, 2026
e6e63fb
style(dart-core): format processor.dart to adhere to line length limit
gspencergoog Oct 28, 2026
af6d17f
docs(dart-core): move MessageErrorMapper entry under Unreleased headi…
gspencergoog Oct 28, 2026
30a69f9
fix(dart-core): support bare '1.0' in MessageErrorMapper and composit…
gspencergoog Oct 29, 2026
624b13c
fix(a2ui-core): address review feedback on NodeResolver and error mapper
gspencergoog Oct 27, 2026
ccae186
feat(swift-core): add MessageErrorMapper envelope handling, JSON Poin…
gspencergoog Oct 23, 2026
46e339e
fix(swift-core): fix OrderedCollections import and type annotations i…
gspencergoog Oct 23, 2026
426e364
fix(swift-core): use A2UIErrorDetail and doubleValue in conformance t…
gspencergoog Oct 27, 2026
25ca90b
fix(swift-core): specify AnyComponentAPI in mock catalogs and compare…
gspencergoog Oct 27, 2026
cbe5567
fix(swift-core): align conformance test helper types and action error…
gspencergoog Oct 27, 2026
07a4d87
fix(swift-core): use intValue on JSONValue in ActionsConformanceTests
gspencergoog Oct 27, 2026
30685af
fix(swift-core): resolve remaining conformance and unit test assertions
gspencergoog Oct 27, 2026
425e882
fix(swift-core): update CatalogConformanceTests for removed peer cata…
gspencergoog Oct 27, 2026
7d58357
fix(swift-core): address review feedback on conformance parity and RP…
gspencergoog Oct 28, 2026
425795a
fix(swift-core): use correct argument labels for makeEventResolvedAct…
gspencergoog Oct 28, 2026
0d45fd6
fix(swift-core): use spelled-out variable names in makeMockRPCFunction
gspencergoog Oct 28, 2026
23f9de7
fix(swift-core): pass open schema instead of nil in ActionTestFunction
gspencergoog Oct 28, 2026
5dbb8ae
fix(swift-core): support bare '1.0' protocol version in message proce…
gspencergoog Oct 29, 2026
df3d6dd
fix(swift-core): prioritize explicit check rule message in NodeResolver
gspencergoog Oct 29, 2026
8022970
Merge remote-tracking branch 'upstream/feat/py-core-conformance-parit…
gspencergoog Oct 29, 2026
86c49ab
Merge remote-tracking branch 'upstream/feat/dart-core-conformance-par…
gspencergoog Oct 29, 2026
13e8dca
Merge remote-tracking branch 'upstream/feat/swift-core-conformance-pa…
gspencergoog Oct 29, 2026
317e4cf
feat(conformance): add error_responses.yaml, adapter_conformance.yaml…
gspencergoog Oct 23, 2026
ef39314
fix(conformance): check non-empty child accessibility label and forma…
gspencergoog Oct 23, 2026
932925f
test(renderers): add UI adapter and accessibility conformance for Lit…
gspencergoog Oct 23, 2026
e11b85e
fix(angular): use Object.hasOwn for property checks, format files, an…
gspencergoog Oct 23, 2026
4fef7a2
test(flutter): add UI adapter and accessibility conformance for a2ui_…
gspencergoog Oct 23, 2026
af3f7be
style(flutter): format node_view.dart
gspencergoog Oct 23, 2026
3b7169a
test(swiftui): add UI adapter and accessibility conformance for A2UIS…
gspencergoog Oct 23, 2026
99dff63
fix(swiftui): address PR #3587 review feedback and adapter conformanc…
gspencergoog Oct 23, 2026
7209829
fix(swiftui): remove dropped clearReportedExpressionErrors call in Su…
gspencergoog Oct 27, 2026
275f483
fix(swiftui): disambiguate A2UICore.Node and Catalog.init in AdapterC…
gspencergoog Oct 27, 2026
b5b1ea9
fix(swift): resolve error_responses catalog version fallback and Swif…
gspencergoog Oct 27, 2026
31fb4a6
test(conformance): align test_v10_unknown_nested_function with v1.0 A…
gspencergoog Oct 28, 2026
3ec8f1c
test(conformance): align error_responses and validation_result expect…
gspencergoog Oct 28, 2026
4b74a69
test(conformance): unify v1.0 on bare '1.0' across conformance suites…
gspencergoog Oct 29, 2026
2620a27
test(dart/core): wire error-message capture and pattern alignment int…
gspencergoog Oct 29, 2026
953b0c7
fix(web_core): remove unused assertExpectedProcessError in conformanc…
gspencergoog Oct 29, 2026
0ba09ba
fix(swift-core): clean up stray conflict markers and duplicated closi…
gspencergoog Oct 29, 2026
eae5a62
Merge branch 'upstream/feat/swift-core-conformance-parity' into feat/…
gspencergoog Oct 29, 2026
7611237
Merge branch 'upstream/feat/web-core-conformance-parity' into feat/co…
gspencergoog Oct 29, 2026
3f5e9fc
feat(swift-core): add MessageErrorMapper envelope handling, JSON Poin…
gspencergoog Oct 23, 2026
2be3f18
fix(swift-core): fix OrderedCollections import and type annotations i…
gspencergoog Oct 23, 2026
9ca83a8
fix(swift-core): use A2UIErrorDetail and doubleValue in conformance t…
gspencergoog Oct 27, 2026
27bc31a
fix(swift-core): specify AnyComponentAPI in mock catalogs and compare…
gspencergoog Oct 27, 2026
1bd4848
fix(swift-core): align conformance test helper types and action error…
gspencergoog Oct 27, 2026
59d84e9
fix(swift-core): use intValue on JSONValue in ActionsConformanceTests
gspencergoog Oct 27, 2026
eafdec6
fix(swift-core): resolve remaining conformance and unit test assertions
gspencergoog Oct 27, 2026
df6b73f
fix(swift-core): update CatalogConformanceTests for removed peer cata…
gspencergoog Oct 27, 2026
6ba73dc
fix(swift-core): address review feedback on conformance parity and RP…
gspencergoog Oct 28, 2026
fc24b08
fix(swift-core): use correct argument labels for makeEventResolvedAct…
gspencergoog Oct 28, 2026
b297c05
fix(swift-core): use spelled-out variable names in makeMockRPCFunction
gspencergoog Oct 28, 2026
dfebe1e
fix(swift-core): pass open schema instead of nil in ActionTestFunction
gspencergoog Oct 28, 2026
8b3124b
fix(swift-core): support bare '1.0' protocol version in message proce…
gspencergoog Oct 29, 2026
e565361
fix(swift-core): prioritize explicit check rule message in NodeResolver
gspencergoog Oct 29, 2026
4e2b76a
fix(swift-core): clean up stray conflict markers and duplicated closi…
gspencergoog Oct 29, 2026
440d3bd
Merge rebased upstream/feat/swift-core-conformance-parity into feat/c…
gspencergoog Oct 29, 2026
d6c4641
feat(python-core): add message_error_mapper, NodeResolver, and confor…
gspencergoog Oct 23, 2026
ba28855
fix(a2ui-core): address review feedback and format with pyink
gspencergoog Oct 23, 2026
05378f5
fix(python-core): satisfy mypy type checking in message_error_mapper …
gspencergoog Oct 27, 2026
76c4fb9
fix(a2ui-core): address review feedback on NodeResolver and error mapper
gspencergoog Oct 27, 2026
69b3fa0
Merge remote-tracking branch 'upstream/feat/py-core-conformance-parit…
gspencergoog Oct 29, 2026
4acb7c3
test(python-agent): fix v1.0 protocol version string in streaming par…
gspencergoog Oct 29, 2026
e4ea4d1
Merge remote-tracking branch 'upstream/feat/py-core-conformance-parit…
gspencergoog Oct 29, 2026
c2b42d8
Merge remote-tracking branch 'upstream/main' into feat/conformance-er…
gspencergoog Oct 29, 2026
a66b0ca
chore: merge main@a23a82c8e and fix Flutter, Swift, and changelog CI
gspencergoog Oct 29, 2026
958e001
chore: merge main@cf92be7b9 into feat/conformance-error-and-renderer-…
gspencergoog Oct 29, 2026
47d8a2e
docs(renderers): update Unreleased changelog entries for PR #3583
gspencergoog Oct 29, 2026
b4b8f5f
chore: merge main@e539cd146 into feat/conformance-error-and-renderer-…
gspencergoog Oct 29, 2026
5a69615
fix(conformance): address review findings across adapter suites and c…
gspencergoog Oct 29, 2026
ef0f8e5
style(angular): format adapter_conformance.spec.ts with root Prettier…
gspencergoog Oct 29, 2026
ce76806
fix(conformance): address PR #3583 review feedback across harnesses a…
gspencergoog Oct 30, 2026
a77214c
fix(swift): resolve repeater template ID and validator path in confor…
gspencergoog Oct 30, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@ let package = Package(
"A2UICore",
"BasicCatalog",
"BasicCatalogSwiftUI",
.product(name: "Yams", package: "Yams"),
],
path: "swift/swiftui/Tests/A2UISwiftUITests"
),
Expand Down
18 changes: 12 additions & 6 deletions conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,14 @@ Test suites are organized by functional domain:
- `core/catalog.yaml`: Covers the catalog API, grouped by entry point. (1) `from_json` cases load a catalog with `Catalog.fromJson`, from an inline `catalog` or a repository-relative `catalogPath`, and check `expect` (`FromJsonExpect` in `conformance_schema.json`). `components` and `functions` list exactly the entries the loaded catalog defines, so a case can check that entries the `anyComponent` and `anyFunction` unions leave out are not loaded. The basic cases set `matchesBuiltinCatalog: true`: the published basic catalog loaded from JSON must be equivalent to the SDK's own basic catalog implementation (same id, version, component and function names, per-component property and required names, and per-function return type, callers and activation). Custom cases cover parsing rules and identifiers. `selfContained: true` requires every `$ref` in the resolved catalog schema to be a local reference that resolves within that document, and `validComponents` / `invalidComponents` list instances that must pass or fail validation. Every published catalog under `specification/<version>/catalogs/` and `catalogs/<name>/<version>/` has a `selfContained` case, and the Python suite fails if one is missing. (2) `to_json` cases compare the whole unbundled document `toJson()` returns with `expectFile`. Each case sets `builtinCatalog: basic` to serialize the SDK's own basic catalog for the case's protocol version and expects the published catalog file. (3) `round_trip` cases load `catalog` or `catalogPath` with `fromJson` (never the SDK's own implementation) and require `toJson()` to equal the input and `fromJson(out).toJson()` to equal `out`. (4) `validation_schema` cases load a published basic catalog, or a small inline catalog that exercises flattening, with `fromJson` and compare `validationSchema`, the successor of `catalogSchema`, with a golden artifact named by `expectFile` under `test_data/validation_schema/`. The goldens use the flattened shape: `allOf` envelopes are flattened, each component gets the `id` envelope, and the common types it references are inlined into `$defs`. An SDK whose output has a different shape lists the case as a known divergence. (5) `derive` cases (`DeriveTest`) load `catalog` or `catalogPath` with `fromJson`, derive a catalog from it with `args`, and check `expect` (`DeriveExpect`): the derived catalog's ID, protocol version, entries, theme and instructions, and the top-level keys its `toJson()` document leaves out. In `args.components` and `args.functions`, a name keeps the source's entry and a map with one key is a new entry. A harness fails a case whose `expect` block uses a key its definition does not list. All comparisons ignore object key order and the order of `enum` and `required` values.
- `core/common_types.yaml`: Compares the common types schema that an SDK generates from its own models (`action: common_types_schema`) with the published `specification/<version>/json/common_types.json`. Both documents are compared as canonical JSON with sorted keys and must be identical. It also checks the rules those types enforce (`action: validate_common_type`): each step's `value` is validated as the named `definition` with the SDK's own model. A step with `expectError` must be rejected, and any other step must be accepted and serialize back to the same JSON.
- `core/agent_to_renderer.yaml`: Compares the `agent_to_renderer` (`server_to_client`) message schema that an SDK generates from its own models (`action: agent_to_renderer_schema`) with the published `specification/<version>/json/server_to_client.json` or `specification/<version>/json/agent_to_renderer.json`. Both documents are compared as canonical JSON with sorted keys and must be identical.
- `core/accessibility.yaml`: Contains test cases for accessibility attributes and checks.
- `core/accessibility.yaml`: Covers 6 v1.0 accessibility attribute and data binding cases (`action: accessibility_check`). Each case checks static and data-bound `label`, `description`, `live`, and `hidden` attributes across Core SDKs and UI renderers.
- `core/adapter_conformance.yaml`: Covers 13 v1.0 UI framework adapter cases (`action: render_surface`). Each case steps through message processing, data updates, user interactions, and surface disposal. It checks rendered view hierarchies, accessibility attributes, validation states, and outbound actions across `@a2ui/lit`, `@a2ui/react`, `@a2ui/angular`, `a2ui_flutter`, and `A2UISwiftUI`.
- `core/error_responses.yaml`: Covers 23 renderer-to-agent `error` cases and 1 non-strict buffered streaming case across v0.9 and v1.0 (`action: validate`). Error cases check the error category, `expectError.rendererError` payload (`code`, `message`, and applicable `surfaceId`, `functionCallId`, or JSON Pointer `path`), and outbound schemas `client_to_server.json` (v0.9) and `renderer_to_agent.json` (v1.0).
- `core/rpc_functions.yaml`: Covers 23 `callRendererFunction` and `rendererFunctionResponse` lifecycle, timeout, cancellation, and error cases (`action: handle_rpc`).
- `core/validator_v0_8.yaml`, `core/validator_v0_9.yaml`, `core/validator_v1_0.yaml`: Contain test cases for schema and structural validation, one suite per protocol version. They cover envelope and component schema checks, duplicate IDs, missing roots, dangling references, cycles, reachability (including list templates and custom v0.8 roots), nesting depth limits, invalid paths, multiple surfaces, and incremental updates. Every case uses `action: validate` in strict mode. The messages in a step are processed in order, so a later message is checked against the components that earlier messages in the same step created. These three suites replace the former `core/validator.yaml`.
- `core/data_model.yaml`: Contains test cases for the reactive data model, verifying JSON Pointer reads and writes, container creation, deletion, and observer notification.
- `core/message_processor_v0_8.yaml`, `core/message_processor_v0_9.yaml`, `core/message_processor_v1_0.yaml`: Contain test cases for the message processor's state machine, one suite per protocol version. A step of a `process_messages` case may set `validationConfig`, the validation rules for that call only (`ExecutionContext.validationConfig`), in place of the rules the processor was created with. A rule the step leaves out is strict. These steps belong to the optional `agent_renderer_app` feature (`blueprints/features/agent_renderer_app.blueprint.md`), and an SDK that does not implement it skips their cases.
- `core/node_resolution.yaml`: Contains node-tree, scoped-binding, notification, identity, action, and destruction scenarios. See [the case format](core/node_resolution.md).
- `core/node_resolution.yaml`: Covers 16 reactive `NodeResolver` and `ComponentNode` cases (`action: resolve_nodes`) executed across TypeScript (`@a2ui/web_core`), Python (`a2ui_core`), Dart (`a2ui_core`), and Swift (`A2UICore`). Scenarios check node tree materialization, scoped template bindings, incremental message streams (`op: process_messages`), cycle and orphan recovery, rule evaluation, action dispatch, and disposal. See [the case format](core/node_resolution.md).
- `core/expressions.yaml`: Contains test cases for the client-side expression parser behind `formatString`, covering literals, data bindings, function calls, nested interpolation, escaped markers and parse errors.
- `core/functions.yaml`: Contains test cases for the basic catalog functions, and for the catalog a function call runs in on a surface. See [Writing cases for `evaluate_function`](#writing-cases-for-evaluate_function).
- `core/message_processor_scaling.yaml`: Timing cases for `processMessages`, with a validation config and without one, which check that a surface arriving in many small messages takes time in proportion to its size. See [Timing cases](#timing-cases).
Expand Down Expand Up @@ -82,11 +85,14 @@ Each language SDK implements test harnesses that:
2. Feed the inputs to the language's specific implementation of the parser/validator/compiler.
3. Assert that the output matches the expected results defined in the YAML.

Refer to the test harnesses across SDKs for worked examples of running conformance suites:
Refer to the test harnesses across Core SDKs and UI framework renderers for worked examples of running conformance suites:

- Python: `python/a2ui_agent/tests/conformance/test_conformance.py`
- Dart: `dart/a2ui_core/test/conformance/expressions_conformance_test.dart`
- TypeScript: `typescript/core/web_core/src/expressions/expression_parser.conformance.test.ts`
- Python Core: `python/a2ui_core/tests/test_conformance.py` and `python/a2ui_core/tests/node_resolution_runner.py`
- Python Agent: `python/a2ui_agent/tests/conformance/test_conformance.py`
- TypeScript Web Core: `typescript/core/web_core/tests/conformance/conformance_test.mjs`, `typescript/core/web_core/tests/conformance/node-resolution.ts`, and `typescript/core/web_core/tests/conformance/accessibility_conformance.test.ts`
- Web Renderers (`@a2ui/lit`, `@a2ui/react`, `@a2ui/angular`): `typescript/renderers/lit/src/tests/adapter_conformance.test.ts`, `typescript/renderers/react/tests/adapter_conformance.test.tsx`, and `typescript/renderers/angular/src/adapter_conformance.spec.ts`
- Dart Core and Flutter Renderer: `dart/a2ui_core/test/conformance/validator_conformance_test.dart`, `dart/a2ui_core/test/conformance/node_resolution_conformance_test.dart`, and `dart/a2ui_flutter/test/adapter_conformance_test.dart`
- Swift Core and SwiftUI Renderer: `swift/core/Tests/A2UIConformanceTests/ValidatorConformanceTests.swift`, `swift/core/Tests/A2UIConformanceTests/NodeResolutionConformanceTests.swift`, and `swift/swiftui/Tests/A2UISwiftUITests/AdapterConformanceTests.swift`

Both Dart and TypeScript locate the suite by walking up from the test file, so they need no configured path. When implementing conformance harnesses for a new language or platform, across the Core SDK, Inference SDK, and framework adapter tiers, see [Implementing conformance harnesses](conformance_harness_implementation_guide.md).

Expand Down
Loading
Loading