From 74f08409d1e16c72d2f9c630195d2160b9cf638d Mon Sep 17 00:00:00 2001
From: Yernat Yestekov <2068106+doublewhy@users.noreply.github.com>
Date: Fri, 9 Oct 2026 02:48:47 -0700
Subject: [PATCH 1/2] docs(sdl): assess VS Code support for SDL authoring

Add a feasibility note for issue 1412 under docs/research. It records
what raes.language_service supports, what a VS Code integration needs,
what a prototype extension outside this repository demonstrates, and
the gaps an editor exposes, including the completion mismatch tracked
in issue 1339. It recommends one language server over the existing
helpers with thin editor clients, starting with diagnostics, reference
completion and go to definition, and gives a verdict on each of the
issue's four alternatives.

Add an evidence record beside it with the probe inputs and the outputs
observed at 35122105, plus the source locations behind the claims that
no probe shows. The note's gap table cites that record; the prototype
link remains only for claims about the editor client.
---
 docs/research/sdl-editor-support/evidence.md  | 162 ++++++++++++
 .../sdl-editor-support/feasibility.md         | 235 ++++++++++++++++++
 2 files changed, 397 insertions(+)
 create mode 100644 docs/research/sdl-editor-support/evidence.md
 create mode 100644 docs/research/sdl-editor-support/feasibility.md

diff --git a/docs/research/sdl-editor-support/evidence.md b/docs/research/sdl-editor-support/evidence.md
new file mode 100644
index 000000000..ca3f672fc
--- /dev/null
+++ b/docs/research/sdl-editor-support/evidence.md
@@ -0,0 +1,162 @@
+# SDL editor support evidence
+
+Observed 2026-10-09 on repository revision
+`35122105b0c1bb648754625d8e3920eafa9bc599`. The
+[feasibility note](feasibility.md) interprets these observations. The probes
+called the language-service helpers in-process, with the working tree at that
+revision and the repository Python environment (CPython 3.14.4, free-threaded).
+Nothing was mocked, and no scenario was launched. The probes establish helper
+and schema behavior on small inputs. They are not a usability, conformance or
+security result.
+
+## Minimal probe inputs
+
+Specimen N is a valid document with one feature and one node:
+
+```yaml
+name: demo
+features:
+  app: {type: service, source: webapp}
+nodes:
+  web:
+    type: compute
+    os: linux
+    resources: {ram: 2 GiB, cpu: 1}
+    features: [app]
+```
+
+Specimen P is a valid document with a participant relationship:
+
+```yaml
+name: audit
+entities:
+  team: {role: red}
+agents:
+  one: {affiliations: [team]}
+  two: {affiliations: [team]}
+propositions:
+  ok:
+    description: declared state
+    subjects: [agents.one]
+    basis: declared_state
+    predicate:
+      kind: boolean
+      property: ready
+      semantic_ref: urn:raes:declared-property:ready
+      operator: equals
+      expected: true
+assertions:
+  done: {proposition: ok, role: postcondition, polarity: positive}
+relationships:
+  pair:
+    type: participant
+    source: agents.one
+    target: agents.two
+    participant: {kind: cooperation}
+```
+
+Specimen M is a valid document in which every value has its long form:
+
+```yaml
+name: demo
+features:
+  app: {type: service, source: {name: webapp, version: '*'}}
+nodes:
+  web: {type: compute, os: linux, resources: {ram: 2 GiB, cpu: 1}, features: {app: client}, roles: {client: {username: www}}}
+```
+
+Each row below names the edit that it applies to a specimen. Lines and columns
+are 1-based, as the helpers report them.
+
+## Language-service observations
+
+| ID | Operation | Observed result | Limit |
+| --- | --- | --- | --- |
+| L1 | `language_completions(P, cursor_path="/relationships/pair/target")`, then `language_diagnostics` of P with each offered `insert_text` as the target | Context `reference:targetable`, with six items: `agents.one`, `agents.two`, `assertions.done`, `entities.team`, `propositions.ok` and `relationships.pair`. Only `two` is valid. `one` fails with "Relationship 'pair' requires distinct participants". The other four fail because the target must "resolve unambiguously to a declared agent". | Finding F5 in the [participant audit](../participant-identity/audit.md) reports the same mismatch, and #1339 tracks it |
+| L2 | Compare each list in `SECTION_FIELD_COMPLETIONS` with the property names that the published schema declares for that section | 18 of the 33 lists differ from the schema. Together they omit 69 schema fields. The `features` list also offers `version`, which the schema does not declare. The table below lists the differences. | The comparison merges every union branch. Seven schema sections have no list. |
+| L3 | `language_completions` on a document whose only feature is `app: {type: service}`, with `cursor_path="/features/app"` and `prefix="v"`; then `language_diagnostics` with `version: '1.0'` added to that feature | Completion offers `version`. Diagnostics report `sdl.model.invalid`, "Extra inputs are not permitted", at `/features/app/version`. | |
+| L4 | `language_completions(text, cursor_path="/nodes/web/features")` on N and on four edits of N | N gives `ok` with `app`. `features:` with no value gives `ok` with `app`. `features: [` gives `invalid` with `sdl.parse`. `fea` in place of the `features` line gives `invalid` with `sdl.parse`. `type: VM` gives `invalid` with `sdl.legacy_node_type_vm`. | A parse failure anywhere in the document removes every item |
+| L5 | `language_diagnostics` on four documents | Adding `colour: blue` under `web` in N gives `sdl.model.invalid` at `/nodes/web/colour`, on line 10, columns 13 to 17. The key starts at column 5. `features: [missing]` in N gives `sdl.semantic` at the `semantic_validation` stage, with no path and no range. `name: demo` followed by `name: again` gives `sdl.mapping_key_conflict` at `/name`, on line 2, columns 1 to 5. An unclosed `web: {type: compute` gives `sdl.parse` with an empty path and one point, line 4, column 1. | The point in the last case is the end of the input |
+| L6 | `language_format` on a document with three comments and the feature `app: {type: Service, source: webapp}` | Status `formatted`. The output has no comments. `type: Service` becomes `type: service`, and `source: webapp` becomes a mapping of `name: webapp` and `version: '*'`. | |
+| L7 | `apply_structured_edit` on N with two added comments, operation `set`, pointer `/nodes/web/os`, value `windows` | Status `edited`. The output has no comments. `source: webapp` expands as in L6, and `features: [app]` becomes the mapping `app: ''`. | |
+| L8 | `parse_sdl_file` and `language_diagnostics` on `contracts/fixtures/sdl/variation-points-v1/composition/root.yaml` | `parse_sdl_file` returns an `ExpandedScenario`. On the same text, `language_diagnostics` returns `sdl.parse`, "SDL imports require file-backed parsing via parse_sdl_file()". Turning off `semantic_validation` gives the same result. | |
+| L9 | File sizes of the tracked `*.sdl.yaml` files, against `_MAX_INPUT_BYTES` | The limit is 65,536 bytes. None of the 40 files is over it. The largest is `examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml`, at 46,073 bytes. | |
+| L10 | `language_completions(N, cursor_path=path)` for `/nodes`, `/nodes/web`, `/nodes/web/resources` and `/nodes/web/os` | Each call returns context `section:nodes` and the same nine fields: `type`, `description`, `os`, `resources`, `features`, `conditions`, `services`, `roles` and `endpoint_persona`. | |
+| L11 | Median of 5 warm calls of `language_diagnostics`, and of `language_completions` at `/nodes`, on the L9 file. Median of 3 fresh interpreters that import `raes` and make one diagnostics call. | At load average 7.2: 172 ms and 159 ms warm, and 1,207 ms cold. At 13.1: 174 ms and 163 ms warm, and 1,297 ms cold. At 22.5: 377 ms and 373 ms warm, and 2,939 ms cold. | An 8-core Apple M3 that other work shared. The timings are indicative. |
+
+L2 counts, for each list, the distinct property names that the schema declares
+for one entry of the section. For a section that is one object, such as
+`realization`, it counts that object's properties. The count follows `$ref` and
+merges `anyOf`, `oneOf` and `allOf` branches. These lists differ:
+
+| Section | Offered | In schema | Schema fields not offered |
+| --- | --- | --- | --- |
+| `accounts` | 7 | 14 | `description`, `disabled`, `groups`, `home`, `mail`, `materialization_profile`, `shell` |
+| `agents` | 7 | 12 | `allowed_subnets`, `authority_anchors`, `description`, `observation_boundaries`, `operating_scope` |
+| `behavior_specifications` | 17 | 18 | `autonomous_execution` |
+| `conditions` | 5 | 10 | `environment`, `name`, `retries`, `start_period`, `timeout` |
+| `content` | 7 | 13 | `description`, `destination`, `sensitive`, `tags`, `text`, `text_from` |
+| `entities` | 4 | 7 | `description`, `events`, `mission` |
+| `events` | 2 | 5 | `description`, `name`, `source` |
+| `evidence_requirements` | 17 | 24 | `capture_requirement_ref`, `capture_spec_ref`, `description`, `field_selectors`, `notes`, `observation_demand`, `output_contract` |
+| `features` | 4 | 7 | `description`, `destination`, `environment`, `name`. The list also offers `version`. |
+| `infrastructure` | 4 | 6 | `acls`, `description` |
+| `injects` | 3 | 6 | `description`, `environment`, `name` |
+| `nodes` | 9 | 16 | `architecture`, `asset_value`, `injects`, `os_distribution`, `os_version`, `runtime`, `source` |
+| `objectives` | 7 | 9 | `description`, `name` |
+| `realization` | 2 | 3 | `constraints` |
+| `relationships` | 11 | 17 | `database_access`, `description`, `forwarding_edge`, `mail_access`, `proxy_upstream`, `service_integration` |
+| `scripts` | 4 | 6 | `description`, `name` |
+| `stories` | 2 | 4 | `description`, `name` |
+| `workflows` | 2 | 5 | `compensation`, `description`, `timeout` |
+
+These schema sections have no list: `action_contracts`, `execution_policy`,
+`forwarding_agents`, `generated_artifacts`, `observation_boundaries`,
+`outcome_interpretation_rules` and `persistent_volumes`.
+
+## Schema observations
+
+S1 to S3 load each document with `yaml.safe_load` and validate it against
+`contracts/schemas/sdl/sdl-authoring-input-v1.json` with the
+`Draft202012Validator` of `jsonschema` 4.26.0. That is how an editor association
+checks raw source.
+
+| ID | Operation | Observed result | Limit |
+| --- | --- | --- | --- |
+| S1 | The 21 `*.sdl.yaml` files under `examples/` and `docs/public/_static/examples/` | `language_diagnostics` accepts all 21. The schema rejects one, `docs/public/_static/examples/first-scenario.sdl.yaml`: its `/nodes/lab-network/type` is `Switch`, and the schema allows only `compute` and `switch`. | |
+| S2 | Six documents that `language_diagnostics` accepts: P without `relationships`, four edits of M, and a document with the node `web` and `infrastructure: {web: 2}` | P without `relationships` has no schema error. The other five have one each. The edits of M are `type: Service` in the feature, `source: webapp`, `features: [app]` in place of the node's `features` and `roles`, and `roles: {client: www}`. | |
+| S3 | Four documents that `language_diagnostics` rejects | `features: {missing: client}` in M (`sdl.semantic`) passes the schema. So does `name: demo` followed by `name: again` (`sdl.mapping_key_conflict`). `colour: blue` in M's node (`sdl.model.invalid`) and the key `Features` in M (`sdl.noncanonical_field`) fail it. | `yaml.safe_load` keeps only the last duplicate key, so the schema never sees the first |
+| S4 | Read the schema's annotations, and request its `$id` with `curl -L` | `x-raes-validates-raw-source` is `false`. `$id` is `https://openrae.github.io/rae/schemas/sdl-authoring-input-v1.json`, and the request returned HTTP 404 on 2026-10-09 at 11:45 UTC. | |
+| S5 | Compare the schema at tag `v6.0.1` with this revision. Run `language_diagnostics` on `name: demo` plus `execution_policy: {}` at both. | At `v6.0.1` the schema has 45 top-level properties and 602 `$defs` entries, and no top-level `execution_policy`. At this revision it has 46 and 606, including `execution_policy`. The document is valid at this revision. With the `v6.0.1` package tree on `PYTHONPATH`, it fails with `sdl.model.invalid`, "Extra inputs are not permitted", at `/execution_policy`. | The `v6.0.1` run used this revision's dependencies, not the wheel's |
+| Y1 | yaml-language-server 1.24.0 from npm, over stdio, with the association that a VS Code `yamlValidation` entry for `*.sdl.yaml` produces. The schema is the prototype's copy of this revision's file, SHA-256 `162121460f19a555f4e7c469a3e4574fe4ed5ecabe628836bf1880ede5f1aba0`. | 20 of the 21 S1 files get no diagnostics. `first-scenario.sdl.yaml` gets "Value is not accepted" on `type`. Each of the five accepted shorthand and case forms in S2 gets one diagnostic, and the unresolved reference in S3 gets none. On an empty line inside `/nodes/web`, where only `type` is present, completion offers 15 fields. Hover on `type` shows the `NodeType` description. | Red Hat's YAML extension 1.24.0 uses this server version. VS Code itself was not started. |
+
+## Inspected seams
+
+These source locations support the claims that a probe does not show. All
+package paths are relative to `implementations/python/packages/`.
+
+| Concern | Source and inspected symbol |
+| --- | --- |
+| Field lists | `raes/_language_metadata.py`: `SECTION_FIELD_COMPLETIONS`, a hand-kept tuple of field names for each section |
+| Completion depth | `raes/language_service.py`: `language_completions`. A path of one section name, and any longer path that does not end in a reference field, both read `SECTION_FIELD_COMPLETIONS[pointer[0]]`. |
+| Reference completion | `raes/_language_metadata.py`: `REFERENCE_COMPLETION_TARGETS`; `raes/language_service.py`: `_completion_target_section`, `_reference_completion_items` |
+| Completion on text that does not parse | `raes/language_service.py`: `_load_completion_data` returns the parse diagnostic instead of items |
+| Semantic diagnostics | `raes/language_service.py`: `language_diagnostics` turns each message in `SDLValidationError.errors` into an `sdl.semantic` record without a path or range |
+| Document context | Each helper takes text only. `language_diagnostics` calls `parse_sdl` without a path. |
+| Handled exceptions | `raes/language_service.py`: `language_diagnostics` catches `SDLParseError` and `SDLValidationError`. `_load_completion_data`, `_try_declaration_index` and `language_format` catch `SDLParseError`. `_declaration_index_from_data` catches the pydantic `ValidationError`. `apply_structured_edit` catches `SDLParseError` and `ValueError`. `raes/_language_references.py`: `find_references` catches `SDLParseError`. Any other exception reaches the caller. |
+| Input limit | `raes/language_service.py`: `_MAX_INPUT_BYTES`, `_size_error` |
+| Formatting | `raes/language_service.py`: `language_format`, whose docstring reads "Migrate recognized legacy spellings and return canonical SDL YAML"; `raes/formatting.py`: `format_sdl_source` |
+| Module cache | `raes/module_registry/__init__.py`: `_oci_cache_dir` returns `.raes/module-cache` under the base directory |
+| Schema for an installed `raes` | `raes_contracts/corpus.py`: `corpus_family_root`. The wheel build bundles the contract corpus at `raes_contracts/_corpus`, as the comment above `[tool.hatch.build.targets.wheel.hooks.custom]` in `implementations/python/pyproject.toml` describes. |
+| Python range | `implementations/python/pyproject.toml` at tag `v6.0.1`: `requires-python = ">=3.11,<3.15"` |
+
+## Reproduce
+
+The prototype's
+[evidence scripts](https://github.com/doublewhy/vscode-raes-sdl/tree/feac96e0d5262d914c11deecb209426851915763/evidence)
+automate L1 to L11, S1 to S3 and Y1. Each script takes the path of a RAES
+checkout and runs under that checkout's Python environment. The Y1 script also
+takes the directory where `yaml-language-server@1.24.0` is installed. At this
+revision the scripts printed the results above, and L11 records three runs of
+the latency script. S4 and S5 used `curl`, `git show` and `git archive` on tag
+`v6.0.1`.
diff --git a/docs/research/sdl-editor-support/feasibility.md b/docs/research/sdl-editor-support/feasibility.md
new file mode 100644
index 000000000..222518a9e
--- /dev/null
+++ b/docs/research/sdl-editor-support/feasibility.md
@@ -0,0 +1,235 @@
+# SDL editor support feasibility
+
+Date: 2026-10-09
+
+This note answers issue #1412: should RAES support SDL authoring in VS Code, and what scope
+would justify the work? It is research evidence, not contract authority.
+
+The observations use repository revision `35122105b0c1bb648754625d8e3920eafa9bc599`. The
+[evidence record](evidence.md) holds the probe inputs and outputs, and IDs such as L4 name its
+rows. Claims about the editor client rest on a prototype extension at
+[doublewhy/vscode-raes-sdl](https://github.com/doublewhy/vscode-raes-sdl/tree/feac96e0d5262d914c11deecb209426851915763).
+The prototype stays outside this repository because the issue asks for an exploration, and an
+extension here would be executable adoption.
+
+## Recommendation
+
+Build one language server over the existing helpers and keep each editor client thin. A VS Code
+extension is worth having as that thin client. It does not need its own SDL logic.
+
+The smallest useful feature set is:
+
+1. File association for `*.sdl.yaml`.
+2. Diagnostics from `language_diagnostics`, debounced while the author types.
+3. Reference completion and go to definition from `language_completions` and
+   `language_references`.
+
+Leave out formatting, edit actions, and schema diagnostics until the gaps below are closed.
+Formatting and structured edits drop comments. Schema diagnostics disagree with RAES validation.
+
+Reusable components already exist. The five helpers carry the semantics. The published schema
+serves completion and hover. `raes_contracts.corpus` locates the schema that matches an installed
+`raes`. From the prototype, the stdio server and its cursor mapper are reusable starting points.
+
+These helper changes would remove most of the work that each client now repeats:
+
+1. Accept a line and column, or return a map from positions to paths.
+2. Give semantic diagnostics a path and a source range.
+3. Derive field completion from the model at every depth, not from hand-kept lists.
+4. Offer only references that the field's purpose accepts, as #1339 requires.
+5. Accept an optional document path, so local imports can resolve.
+6. Return a diagnostic for every input instead of raising.
+
+### Alternatives
+
+The issue lists four alternatives:
+
+1. **YAML tooling with the schema association and explicit CLI validation: rejected as the
+   main path.** The association gives completion and hover, but its diagnostics disagree with
+   RAES (S2, S3 and Y1). `raes semantic validate` reports RAES results only when the author runs
+   it. Both remain useful without an extension.
+2. **A highlighting and snippet extension: rejected.** Highlighting adds little over YAML, and
+   neither feature gives diagnostics or references.
+3. **Direct VS Code language-feature providers over the helpers: rejected.** They still need a
+   Python process to run the helpers, and they serve only VS Code. Other editors and Monaco would
+   each need their own providers.
+4. **One language server with thin editor clients: selected.** One server serves every LSP
+   client, and the editor-specific code stays small.
+
+## What the language service supports
+
+[`raes.language_service`](../../../implementations/python/packages/raes/language_service.py)
+has five helpers that return plain dictionaries. The MCP server wraps them as the
+`language_service` tool family. This repository ships no language server.
+
+| Helper | Input | Result | Editor use |
+| --- | --- | --- | --- |
+| `language_completions` | Text, a JSON-pointer-like `cursor_path`, a prefix | Top-level keys, section fields, or reference candidates | Completion |
+| `language_references` | Text, a bare or qualified symbol | Definitions and occurrences with 1-based ranges | Go to definition, find references |
+| `language_diagnostics` | Text, a `semantic_validation` flag | Parse, structural, and semantic records | Diagnostics |
+| `language_format` | Text | Canonical YAML with migration advisories | Formatting |
+| `apply_structured_edit` | Text, an operation, a JSON pointer, a value | Edited YAML with diagnostics | Code actions |
+
+The helpers reuse the repository parser and semantic validator. Each one refuses input over
+64 KiB. The [language-service guide](../../explain/sdl/language-service.md) describes the
+payloads.
+
+## What the editor integration needs
+
+A VS Code integration needs these parts:
+
+- A language contribution that maps `*.sdl.yaml` to a language identifier.
+- A client that starts the server. With `vscode-languageclient`, the prototype's client is 41
+  lines.
+- A server that turns cursor positions into helper paths and helper ranges into LSP ranges. It
+  also debounces diagnostics and copes with documents that do not parse yet.
+- A setting that picks the Python interpreter. The `raes` version installed there decides the
+  semantics.
+- A Workspace Trust declaration, because that setting names a program to run.
+- A package built with `vsce`. A VS Code package cannot declare Python dependencies, so authors
+  install `raes` with `pip`.
+
+## What the prototype demonstrates
+
+The prototype has a TextMate grammar, a schema association, and a stdio language server. The
+server uses only the Python standard library and `raes`. Sixteen tests drive the server over
+stdio with the messages that the VS Code client sends. Nine test the cursor mapper, and six
+tokenize SDL with VS Code's YAML grammar. All 31 passed locally against revision `35122105`,
+and in [CI run 37926735623](https://github.com/doublewhy/vscode-raes-sdl/actions/runs/37926735623)
+with `raes` 6.0.1 from PyPI. In that run, `vsce package` built a 571 KB package. Nothing is
+published to the Marketplace, and no test starts VS Code itself.
+
+| Feature | Result |
+| --- | --- |
+| Diagnostics | Parse and structural errors keep their RAES ranges. Semantic errors have no location, so they appear on line 1 with a note. |
+| Completion | Works on documents that parse. Otherwise the server answers from the last version that parsed and marks the items. |
+| Go to definition | Works inside one file through `language_references`. |
+| Highlighting | Top-level SDL keys and `${variable}` placeholders get their own scopes. A test checks the key list against the schema. |
+| Schema association | Completion and hover work in yaml-language-server 1.24.0. Its diagnostics disagree with RAES. |
+| Formatting | Not offered, because the helper drops comments. |
+
+The server forwards reference candidates unfiltered, so the #1339 mismatch stays visible. It
+drops field names only after `key:` or inside a flow sequence, where no mapping key can go. That
+is a YAML rule, not an SDL rule. When a helper raises, the server reports one diagnostic and
+keeps running. Two tests check this with helpers that raise on a marker line.
+
+## Gaps
+
+| Area | Observation | Evidence |
+| --- | --- | --- |
+| Reference completion | For a participant relationship `target`, completion offers six declarations. Validation accepts one, the agent that is not the source. This is the #1339 mismatch. | L1; finding F5 in the [participant identity audit](../participant-identity/audit.md) |
+| Field lists | The hand-kept lists miss 69 schema fields across 18 of 33 sections. The `features` list offers `version`, which validation rejects. | L2 and L3; `SECTION_FIELD_COMPLETIONS` in [`_language_metadata.py`](../../../implementations/python/packages/raes/_language_metadata.py) |
+| Completion depth | Field completion uses only the section name. `/nodes`, `/nodes/web`, and `/nodes/web/resources` get the same nine fields. | L10; the field branches of `language_completions` in [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) |
+| Cursor paths | The helpers take a JSON pointer, so each client derives one from the text. The prototype's mapper has 204 lines and skips multi-line flow collections, block scalars, anchors, tags, and complex keys. | Prototype [`server/sdl_cursor.py`](https://github.com/doublewhy/vscode-raes-sdl/blob/feac96e0d5262d914c11deecb209426851915763/server/sdl_cursor.py) |
+| Source ranges | Semantic diagnostics have no path or range. An unknown field is ranged on its value, not its key. A YAML syntax error is a single point. | L5 |
+| Incomplete documents | Completion needs a document that parses. An unclosed `[` or a key without its colon gives a parse error and no items. | L4 |
+| Imported modules | The helpers take no file path, so a document with `imports` is invalid even without semantic validation. `parse_sdl_file` accepts the same fixture. | L8 |
+| Workspace context | The helpers see one document. Nothing indexes the workspace or follows references across files. | [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) |
+| Robustness | Each helper turns the parser's and validator's own errors into results. Any other exception reaches the caller, so a server must guard every call. | Handled exceptions under [inspected seams](evidence.md#inspected-seams) |
+| Large scenarios | The largest repository example is 46 KB, under the 64 KiB limit. Each call parses the whole text. On that file a warm call took 160 to 175 ms at load average 7 to 13, and about 375 ms at load average 22. A cold start took 1.2 to 2.9 s. The machine was shared, so the timings are indicative. | L9 and L11 |
+| Formatting and edits | Both drop comments and expand shorthand. Formatting also lowers enum case and migrates recognized legacy spellings. | L6 and L7; the `language_format` docstring |
+| Schema validation | The schema describes the normalized object, so `x-raes-validates-raw-source` is `false`. It flags accepted shorthand and case variants and misses semantic errors. | S2 to S4 and Y1 |
+| Schema versions | The schema's `$id` returned HTTP 404. The schema on `dev` declares `execution_policy`, which `raes` 6.0.1 rejects. | S4 and S5 |
+
+The schema check flags one of the 21 accepted examples (S1): the quickstart's
+`first-scenario.sdl.yaml` uses `type: Switch`. Any fix for imports must also handle registry
+imports. When `parse_sdl` gets a path, an OCI import fetches the module over the network. It also
+writes `.raes/module-cache` next to the document (`raes.module_registry`).
+
+## Answers to the issue's questions
+
+### Which features are most useful
+
+Diagnostics come first. They need no cursor mapping and reuse the full validator. Reference
+completion and go to definition come next. They help most on reference-heavy sections, but
+completion inherits the #1339 mismatch. Schema hover shows field descriptions. Highlighting adds
+little over plain YAML. Snippets and a document outline were not prototyped. Formatting should
+wait until comments survive it.
+
+### Can YAML support and the schema provide enough value
+
+They are enough for completion and hover, not for diagnostics. Inside a node, schema completion
+offered the 15 node fields not yet present (Y1). The helper's list for nodes has 9 fields in all
+(L10). Hover showed the schema's descriptions. But schema diagnostics flagged all five accepted
+shorthand and case forms that the probe tried. They also missed an unresolved reference. A
+dedicated extension is warranted only to add RAES diagnostics and reference features.
+
+Red Hat's YAML extension serves only the languages in its own document selector ([R2](#references)).
+So an extension must choose:
+
+- Its own language identifier, with SDL highlighting but no schema completion.
+- The `yaml` identifier, with schema completion but plain YAML highlighting.
+
+The prototype supports both.
+
+### Would an LSP adapter enable reuse
+
+Yes, for desktop editors. Any LSP client can start the same stdio command. VS Code needs the thin
+extension described above. Other editors that speak LSP should need only their own settings for
+the command and the file pattern. They were not tested.
+
+Monaco is different. A browser page cannot start a Python process. A Monaco client would need a
+server reached over a WebSocket, for example through
+[monaco-languageclient](https://github.com/TypeFox/monaco-languageclient). Running `raes` in
+the browser was not evaluated. [monaco-yaml](https://github.com/remcohaszing/monaco-yaml) gives
+Monaco the schema features, as the YAML extension does in VS Code. A VS Code extension does not
+run inside Monaco.
+
+### What gaps exist
+
+See [Gaps](#gaps). The cursor-to-path mapping, source ranges, incomplete documents, imports,
+workspace context, and large scenarios each have a row there. So does the #1339 completion
+mismatch.
+
+### How packaging and versions should work
+
+- **File association.** The repository names 40 SDL files `*.sdl.yaml` (L9). VS Code prefers the
+  longest matching extension, so `.sdl.yaml` wins over `.yaml` ([R1](#references)). Files such as
+  the composition fixtures end in `.yaml` and need a `files.associations` entry.
+- **Version selection.** The server runs in an interpreter that the author picks, and it reports
+  the `raes` version. A schema bundled in the extension can drift from that version, as S5 shows.
+  The `raes` wheel bundles its own copy, reachable through
+  `raes_contracts.corpus.corpus_family_root`, so a server could serve the matching schema.
+- **Local and offline use.** The helpers work on the buffer text in-process. They read no other
+  document and use no network. Resolving imports would change that, so it should stay opt-in.
+  Red Hat's YAML extension downloads the SchemaStore catalog by default
+  (`yaml.schemaStore.enable`, [R3](#references)).
+- **Packaging.** `vsce package` works without bundling Python. Authors install `raes` from PyPI.
+  Version 6.0.1 requires Python 3.11 to 3.14. In Restricted Mode, VS Code ignores a workspace
+  value of the interpreter setting if the extension lists it in `restrictedConfigurations`
+  ([R4](#references)).
+
+## Compatibility notes
+
+- **SDL semantics stay in RAES.** The prototype server adds no SDL rules. The grammar's key list
+  mirrors the schema's top-level properties, and a test checks that they match.
+- **No launch and no fetch.** The helpers parse and validate text. Nothing runs a scenario or
+  downloads content.
+- **Formatting.** `language_format` drops comments, expands shorthand, normalizes enum case, and
+  migrates legacy spellings. Format on save would rewrite authored text, so formatting should be
+  an explicit command at most.
+- **Monaco.** Shared services are the helpers, the published schema, and one language server.
+  Editor adapters are the VS Code client, a Monaco client, and other editors' settings.
+
+## Method
+
+The probes call the helpers on small documents and on the repository examples. They compare the
+results with the published schema and with yaml-language-server 1.24.0. Version 1.24.0 is the
+server that Red Hat's YAML extension 1.24.0 uses. Nothing was mocked. The
+[evidence record](evidence.md) gives each probe's input and output, the source locations behind
+the other claims, and the scripts that reproduce the probes. The prototype's tests and CI run
+cover only the editor client.
+
+## References
+
+- R1: VS Code 1.140.0,
+  [`languagesAssociations.ts`](https://github.com/microsoft/vscode/blob/1.140.0/src/vs/editor/common/services/languagesAssociations.ts),
+  the longest extension match.
+- R2: Red Hat YAML extension 1.24.0,
+  [`src/extension.ts`](https://github.com/redhat-developer/vscode-yaml/blob/1.24.0/src/extension.ts),
+  the client document selector and the `yamlValidation` association.
+- R3: Red Hat YAML extension 1.24.0,
+  [`package.json`](https://github.com/redhat-developer/vscode-yaml/blob/1.24.0/package.json),
+  the `yaml.schemaStore.enable` default.
+- R4: VS Code, [Workspace Trust extension guide](https://code.visualstudio.com/api/extension-guides/workspace-trust).
+- VS Code, [language server extension guide](https://code.visualstudio.com/api/language-extensions/language-server-extension-guide).

From 1ebc0bfa9ef25ca48f9973813c4305cd15ae0090 Mon Sep 17 00:00:00 2001
From: Yernat Yestekov <2068106+doublewhy@users.noreply.github.com>
Date: Sat, 10 Oct 2026 09:39:44 -0700
Subject: [PATCH 2/2] docs(sdl): correct import trust and format error claims

The note said that parse_sdl with a path fetches an OCI import. It
refuses the import before any request unless raes-trust.yaml beside the
document lists the registry, and it writes .raes/module-cache only after
the registry's signature policy passes. Say so, add an Import trust seam
row, and note in the Workspace Trust bullet that the trust file is
workspace content.

The Robustness row said that each helper turns the parser's and
validator's errors into results. language_format lets
SDLValidationError reach the caller for a materialized document,
because parse_sdl runs the semantic validator for it. Record that as
probe L12 and complete the handled-exceptions seam row.

Also list the nine schema sections without a completion list, give
verdicts on snippets and the document outline, and say that the probe
scripts live outside this repository.
---
 docs/research/sdl-editor-support/evidence.md  | 16 +++++++-----
 .../sdl-editor-support/feasibility.md         | 26 +++++++++++++------
 2 files changed, 28 insertions(+), 14 deletions(-)

diff --git a/docs/research/sdl-editor-support/evidence.md b/docs/research/sdl-editor-support/evidence.md
index ca3f672fc..e53871a70 100644
--- a/docs/research/sdl-editor-support/evidence.md
+++ b/docs/research/sdl-editor-support/evidence.md
@@ -1,6 +1,6 @@
 # SDL editor support evidence
 
-Observed 2026-10-09 on repository revision
+Observed 2026-10-09, and L12 on 2026-10-10, on repository revision
 `35122105b0c1bb648754625d8e3920eafa9bc599`. The
 [feasibility note](feasibility.md) interprets these observations. The probes
 called the language-service helpers in-process, with the working tree at that
@@ -73,7 +73,7 @@ are 1-based, as the helpers report them.
 | ID | Operation | Observed result | Limit |
 | --- | --- | --- | --- |
 | L1 | `language_completions(P, cursor_path="/relationships/pair/target")`, then `language_diagnostics` of P with each offered `insert_text` as the target | Context `reference:targetable`, with six items: `agents.one`, `agents.two`, `assertions.done`, `entities.team`, `propositions.ok` and `relationships.pair`. Only `two` is valid. `one` fails with "Relationship 'pair' requires distinct participants". The other four fail because the target must "resolve unambiguously to a declared agent". | Finding F5 in the [participant audit](../participant-identity/audit.md) reports the same mismatch, and #1339 tracks it |
-| L2 | Compare each list in `SECTION_FIELD_COMPLETIONS` with the property names that the published schema declares for that section | 18 of the 33 lists differ from the schema. Together they omit 69 schema fields. The `features` list also offers `version`, which the schema does not declare. The table below lists the differences. | The comparison merges every union branch. Seven schema sections have no list. |
+| L2 | Compare each list in `SECTION_FIELD_COMPLETIONS` with the property names that the published schema declares for that section | 18 of the 33 lists differ from the schema. Together they omit 69 schema fields. The `features` list also offers `version`, which the schema does not declare. The table below lists the differences. | The comparison merges every union branch. Nine schema sections have no list. |
 | L3 | `language_completions` on a document whose only feature is `app: {type: service}`, with `cursor_path="/features/app"` and `prefix="v"`; then `language_diagnostics` with `version: '1.0'` added to that feature | Completion offers `version`. Diagnostics report `sdl.model.invalid`, "Extra inputs are not permitted", at `/features/app/version`. | |
 | L4 | `language_completions(text, cursor_path="/nodes/web/features")` on N and on four edits of N | N gives `ok` with `app`. `features:` with no value gives `ok` with `app`. `features: [` gives `invalid` with `sdl.parse`. `fea` in place of the `features` line gives `invalid` with `sdl.parse`. `type: VM` gives `invalid` with `sdl.legacy_node_type_vm`. | A parse failure anywhere in the document removes every item |
 | L5 | `language_diagnostics` on four documents | Adding `colour: blue` under `web` in N gives `sdl.model.invalid` at `/nodes/web/colour`, on line 10, columns 13 to 17. The key starts at column 5. `features: [missing]` in N gives `sdl.semantic` at the `semantic_validation` stage, with no path and no range. `name: demo` followed by `name: again` gives `sdl.mapping_key_conflict` at `/name`, on line 2, columns 1 to 5. An unclosed `web: {type: compute` gives `sdl.parse` with an empty path and one point, line 4, column 1. | The point in the last case is the end of the input |
@@ -83,6 +83,7 @@ are 1-based, as the helpers report them.
 | L9 | File sizes of the tracked `*.sdl.yaml` files, against `_MAX_INPUT_BYTES` | The limit is 65,536 bytes. None of the 40 files is over it. The largest is `examples/scenarios/hospital-ransomware-surgery-day.sdl.yaml`, at 46,073 bytes. | |
 | L10 | `language_completions(N, cursor_path=path)` for `/nodes`, `/nodes/web`, `/nodes/web/resources` and `/nodes/web/os` | Each call returns context `section:nodes` and the same nine fields: `type`, `description`, `os`, `resources`, `features`, `conditions`, `services`, `roles` and `endpoint_persona`. | |
 | L11 | Median of 5 warm calls of `language_diagnostics`, and of `language_completions` at `/nodes`, on the L9 file. Median of 3 fresh interpreters that import `raes` and make one diagnostics call. | At load average 7.2: 172 ms and 159 ms warm, and 1,207 ms cold. At 13.1: 174 ms and 163 ms warm, and 1,297 ms cold. At 22.5: 377 ms and 373 ms warm, and 2,939 ms cold. | An 8-core Apple M3 that other work shared. The timings are indicative. |
+| L12 | The five helpers on two documents: `contracts/fixtures/sdl/materialized-scenario-v1/valid/reference.json` with N's node `web` added, and N. In both, the node's `features` is `[missing]`. Completion uses `cursor_path="/nodes/web/features"`, references use the symbol `web`, and the edit is L7's. | On the fixture, `language_diagnostics` returns `invalid` with `sdl.semantic`. `language_format` raises `SDLValidationError`, "Node 'web' references undefined feature 'missing'". Completion and references return `ok`, and the edit returns `edited_with_diagnostics` with `sdl.semantic`. The fixture's JSON text and its YAML dump behave the same. On N nothing raises, and `language_format` returns `formatted_with_diagnostics` with `sdl.semantic`. | |
 
 L2 counts, for each list, the distinct property names that the schema declares
 for one entry of the section. For a section that is one object, such as
@@ -111,8 +112,9 @@ merges `anyOf`, `oneOf` and `allOf` branches. These lists differ:
 | `workflows` | 2 | 5 | `compensation`, `description`, `timeout` |
 
 These schema sections have no list: `action_contracts`, `execution_policy`,
-`forwarding_agents`, `generated_artifacts`, `observation_boundaries`,
-`outcome_interpretation_rules` and `persistent_volumes`.
+`forwarding_agents`, `generated_artifacts`, `imports`, `module`,
+`observation_boundaries`, `outcome_interpretation_rules` and
+`persistent_volumes`. The schema's other four top-level properties are scalars.
 
 ## Schema observations
 
@@ -143,10 +145,11 @@ package paths are relative to `implementations/python/packages/`.
 | Completion on text that does not parse | `raes/language_service.py`: `_load_completion_data` returns the parse diagnostic instead of items |
 | Semantic diagnostics | `raes/language_service.py`: `language_diagnostics` turns each message in `SDLValidationError.errors` into an `sdl.semantic` record without a path or range |
 | Document context | Each helper takes text only. `language_diagnostics` calls `parse_sdl` without a path. |
-| Handled exceptions | `raes/language_service.py`: `language_diagnostics` catches `SDLParseError` and `SDLValidationError`. `_load_completion_data`, `_try_declaration_index` and `language_format` catch `SDLParseError`. `_declaration_index_from_data` catches the pydantic `ValidationError`. `apply_structured_edit` catches `SDLParseError` and `ValueError`. `raes/_language_references.py`: `find_references` catches `SDLParseError`. Any other exception reaches the caller. |
+| Handled exceptions | `raes/language_service.py`: `language_diagnostics` catches `SDLParseError` and `SDLValidationError`. `_load_completion_data`, `_try_declaration_index` and `language_format` catch `SDLParseError`. `_declaration_index_from_data` catches the pydantic `ValidationError`. `apply_structured_edit` catches `SDLParseError` and `ValueError`, and `_split_pointer_or_empty` catches `ValueError`. `raes/_language_references.py`: `_compose_yaml`, which `find_references` calls, catches `SDLParseError`. `_qualified_symbol_section`, `_is_variation_member_symbol` and `_bare_symbol` catch `TypeError` and `ValueError` from symbol parsing. `format_sdl_source` (`raes/formatting.py`) re-parses its output with `parse_sdl(..., skip_semantic_validation=True)`, and `parse_sdl` (`raes/parser.py`) still runs `SemanticValidator` for a `MaterializedScenario`, so `language_format` lets `SDLValidationError` reach the caller. Any other exception reaches the caller. |
 | Input limit | `raes/language_service.py`: `_MAX_INPUT_BYTES`, `_size_error` |
 | Formatting | `raes/language_service.py`: `language_format`, whose docstring reads "Migrate recognized legacy spellings and return canonical SDL YAML"; `raes/formatting.py`: `format_sdl_source` |
 | Module cache | `raes/module_registry/__init__.py`: `_oci_cache_dir` returns `.raes/module-cache` under the base directory |
+| Import trust | `raes/composition/_expand.py`: `expand_sdl_modules` calls `load_lockfile` and `load_trust_policy` on the document's directory. `raes/module_registry/models.py`: `load_trust_policy` returns `TrustPolicy()`, with no registries, when `raes-trust.yaml` is missing; `RegistryTrustPolicy.require_signatures` defaults to `True`. `raes/module_registry/resolution.py`: `_resolve_oci_import` raises "is not allowed by trust policy" for an unlisted registry before any request. For a listed registry it then fetches the manifest, config and bundle, checks signatures if the registry requires them, and only then calls `_extract_bundle_to_cache`. The integration test `test_untrusted_and_unsigned_oci_imports_fail_closed` in `implementations/python/tests/test_sdl_module_registry.py` asserts both refusals. |
 | Schema for an installed `raes` | `raes_contracts/corpus.py`: `corpus_family_root`. The wheel build bundles the contract corpus at `raes_contracts/_corpus`, as the comment above `[tool.hatch.build.targets.wheel.hooks.custom]` in `implementations/python/pyproject.toml` describes. |
 | Python range | `implementations/python/pyproject.toml` at tag `v6.0.1`: `requires-python = ">=3.11,<3.15"` |
 
@@ -159,4 +162,5 @@ checkout and runs under that checkout's Python environment. The Y1 script also
 takes the directory where `yaml-language-server@1.24.0` is installed. At this
 revision the scripts printed the results above, and L11 records three runs of
 the latency script. S4 and S5 used `curl`, `git show` and `git archive` on tag
-`v6.0.1`.
+`v6.0.1`. L12 called the helpers directly, in-process, on the inputs that its
+row names.
diff --git a/docs/research/sdl-editor-support/feasibility.md b/docs/research/sdl-editor-support/feasibility.md
index 222518a9e..b165c4420 100644
--- a/docs/research/sdl-editor-support/feasibility.md
+++ b/docs/research/sdl-editor-support/feasibility.md
@@ -85,7 +85,9 @@ A VS Code integration needs these parts:
   also debounces diagnostics and copes with documents that do not parse yet.
 - A setting that picks the Python interpreter. The `raes` version installed there decides the
   semantics.
-- A Workspace Trust declaration, because that setting names a program to run.
+- A Workspace Trust declaration, because that setting names a program to run. If the server
+  resolves imports, the workspace's `raes-trust.yaml` also decides which registries an import may
+  fetch from.
 - A package built with `vsce`. A VS Code package cannot declare Python dependencies, so authors
   install `raes` with `pip`.
 
@@ -125,7 +127,7 @@ keeps running. Two tests check this with helpers that raise on a marker line.
 | Incomplete documents | Completion needs a document that parses. An unclosed `[` or a key without its colon gives a parse error and no items. | L4 |
 | Imported modules | The helpers take no file path, so a document with `imports` is invalid even without semantic validation. `parse_sdl_file` accepts the same fixture. | L8 |
 | Workspace context | The helpers see one document. Nothing indexes the workspace or follows references across files. | [`language_service.py`](../../../implementations/python/packages/raes/language_service.py) |
-| Robustness | Each helper turns the parser's and validator's own errors into results. Any other exception reaches the caller, so a server must guard every call. | Handled exceptions under [inspected seams](evidence.md#inspected-seams) |
+| Robustness | Every helper turns `SDLParseError` into a result. `language_diagnostics` also turns `SDLValidationError` into one, and `apply_structured_edit` and `language_format` check their output with `language_diagnostics`. But `language_format` lets `SDLValidationError` reach the caller for a document with `materialization_provenance`. For such a document, `parse_sdl` runs the semantic validator even though `format_sdl_source` skips it. Other exceptions reach the caller too, unless a catch listed under inspected seams handles them, so a server must guard every call. | L12; handled exceptions under [inspected seams](evidence.md#inspected-seams) |
 | Large scenarios | The largest repository example is 46 KB, under the 64 KiB limit. Each call parses the whole text. On that file a warm call took 160 to 175 ms at load average 7 to 13, and about 375 ms at load average 22. A cold start took 1.2 to 2.9 s. The machine was shared, so the timings are indicative. | L9 and L11 |
 | Formatting and edits | Both drop comments and expand shorthand. Formatting also lowers enum case and migrates recognized legacy spellings. | L6 and L7; the `language_format` docstring |
 | Schema validation | The schema describes the normalized object, so `x-raes-validates-raw-source` is `false`. It flags accepted shorthand and case variants and misses semantic errors. | S2 to S4 and Y1 |
@@ -133,8 +135,12 @@ keeps running. Two tests check this with helpers that raise on a marker line.
 
 The schema check flags one of the 21 accepted examples (S1): the quickstart's
 `first-scenario.sdl.yaml` uses `type: Switch`. Any fix for imports must also handle registry
-imports. When `parse_sdl` gets a path, an OCI import fetches the module over the network. It also
-writes `.raes/module-cache` next to the document (`raes.module_registry`).
+imports. When `parse_sdl` gets a path and the document has `imports`, it reads `raes.lock.json`
+and `raes-trust.yaml` from the document's directory. It refuses an OCI import before any request
+unless that trust file lists the import's registry. For a listed registry it fetches the module.
+Once the registry's signature policy passes, it writes `.raes/module-cache` next to the document.
+The trust file is workspace content, so a path-aware server should resolve imports only in a
+trusted workspace (Import trust under [inspected seams](evidence.md#inspected-seams)).
 
 ## Answers to the issue's questions
 
@@ -143,7 +149,10 @@ writes `.raes/module-cache` next to the document (`raes.module_registry`).
 Diagnostics come first. They need no cursor mapping and reuse the full validator. Reference
 completion and go to definition come next. They help most on reference-heavy sections, but
 completion inherits the #1339 mismatch. Schema hover shows field descriptions. Highlighting adds
-little over plain YAML. Snippets and a document outline were not prototyped. Formatting should
+little over plain YAML. Snippets and a document outline were not prototyped. Snippets give no
+diagnostics or references (alternative 2), so they rank low. None of the five helpers returns the
+document's declarations with their ranges, so an outline needs new helper or server work. It
+would help in long scenarios, but it ranks after diagnostics and references. Formatting should
 wait until comments survive it.
 
 ### Can YAML support and the schema provide enough value
@@ -216,9 +225,10 @@ mismatch.
 The probes call the helpers on small documents and on the repository examples. They compare the
 results with the published schema and with yaml-language-server 1.24.0. Version 1.24.0 is the
 server that Red Hat's YAML extension 1.24.0 uses. Nothing was mocked. The
-[evidence record](evidence.md) gives each probe's input and output, the source locations behind
-the other claims, and the scripts that reproduce the probes. The prototype's tests and CI run
-cover only the editor client.
+[evidence record](evidence.md) gives each probe's input and output and the source locations
+behind the other claims. It also links to the scripts that reproduce most probes. They live in
+the prototype's repository, not in this one. The prototype's tests and CI run cover only the
+editor client.
 
 ## References