[Draft prototype] Known-tool policy catalog (Policy Store design)

.github/copilot-instructions.md

@@ -21,6 +21,7 @@ MXC (Microsoft eXecution Container) is a cross-platform sandboxed code execution
 - Use `#[cfg(target_os = "...")]` and existing Cargo feature gates for platform-specific code.
 - Preserve the distinction between run-to-completion, streaming, and state-aware lifecycle APIs.
 - Unsupported policy must fail closed. Do not accept a field that the selected backend cannot enforce.
+- The `mxc-sdk` `policy_store` module (prototype, pending API review) is the single tool-requirements resolver and owns the bundled catalog data in `src/mxc-sdk/policy_store/`. `mxc_ffi` exposes it to the Node and C# SDKs; do not reimplement resolution per language or move it into `mxc_engine`. See [`docs/policy-store/README.md`](../docs/policy-store/README.md).
 
 See:
 

docs/policy-store/README.md

@@ -0,0 +1,194 @@
+# MXC Policy Store (prototype)
+
+> **PROTOTYPE, pending API review.** The API names and shapes here are proposed
+> and may change before sign-off. The policy store is not part of MXC 1.0; a
+> later MXC SDK release is targeted.
+
+The policy store resolves known tools (for example `git`, `node`, `npm`) to
+command-free **`ContainerRequirements`** from a reviewed catalog bundled
+statically in the MXC SDKs. The result is a best-effort floor, not a
+guarantee, and it never grants access by itself: the caller reviews and
+constrains it, adds its own command (object spread in Node,
+`ContainerRequest::from_requirements` in Rust,
+`ContainerRequest.FromRequirements` in C#), and passes the request to the
+SDK's existing v1 container APIs. It is complementary to Learning Mode, not a
+replacement. Lookup never creates a container, contacts a network service,
+writes state, or downloads data.
+
+- The spec is in [microsoft/mxc#1309](https://github.com/microsoft/mxc/pull/1309),
+  head `5c2ba8e`: `docs/mxc-policy-store-api.md` is the caller-facing API
+  contract (types, behavior, errors, examples), and
+  `docs/mxc-policy-store.md` the catalog design (data model, packaging,
+  source layout, contribution).
+- [design.md](design.md) records the prototype's implementation decisions and
+  its known gaps against that spec.
+- [`catalog/views/`](../../src/mxc-sdk/policy_store/catalog/views/) holds the
+  generated reviewer view and exact-request view for each revision.
+
+## Where it lives
+
+| Layer | Location | API |
+|---|---|---|
+| Resolver and V1 data | `src/mxc-sdk/src/policy_store/` (internal `mxc-sdk` module) and `src/mxc-sdk/policy_store/` (catalog, schemas, conformance) | Single implementation |
+| Rust SDK | `mxc_sdk::v1` | `resolve_tool_requirements`, `resolve_tool_requirements_with_diagnostics`, `get_catalog_info`, `list_catalog_entries`; inputs and diagnostics in `mxc_sdk::v1::tool_requirements` |
+| C ABI | `src/ffi/mxc_ffi/src/policy_store.rs` | `mxc_resolve_tool_requirements_json`, `mxc_resolve_tool_requirements_with_diagnostics_json`, `mxc_policy_catalog_info_json`, `mxc_list_policy_catalog_entries_json`, `mxc_policy_store_result_free` |
+| Node SDK | `sdk/node/src/v1/tool-requirements.ts` (`@microsoft/mxc-sdk/v1`) | `resolveToolRequirements`, `resolveToolRequirementsWithDiagnostics` (both return a `Promise`), `getCatalogInfo`, `listCatalogEntries` |
+| C# SDK | `sdk/dotnet/Microsoft.Mxc.Sdk/V1/MxcContainer.ToolRequirements.cs` | `MxcContainer.ResolveToolRequirements`, `ResolveToolRequirementsWithDiagnostics` (plus `Async` forms), `GetCatalogInfo`, `ListCatalogEntries` |
+
+There is one resolver. The catalog is compiled into `mxc-sdk`, so the data
+ships inside `mxc_ffi` and every SDK that loads it. The Rust SDK returns
+`ContainerRequirements` built from the existing v1 section types. The Node and
+C# SDKs send a JSON lookup request (`{"tools", "context"?}`) through
+`mxc_ffi` and map the result onto their own v1 section types. The SDK owns
+the wire version: the catalog authors access only (`default.requirements`,
+never a `version` or `command`). Resolution stays outside `mxc_engine`: it never
+selects a backend or launches a container.
+
+## Example (Node)
+
+```ts
+import {
+  ContainerRequest,
+  resolveToolRequirementsWithDiagnostics,
+} from '@microsoft/mxc-sdk/v1';
+
+const { requirements, diagnostics } = await resolveToolRequirementsWithDiagnostics(
+  { invocationName: 'git', packageUrl: 'pkg:generic/git', detectedVersion: '2.45.1', intent: 'push' },
+  { projectRoot: '/work/repo', symbols: { git_prefix: '/usr', ssh_prefix: '/usr' } },
+);
+// diagnostics.tools[0] => { inputIndex: 0, contributes: true, status: 'matched_version', selection: { entryId: 'tool:git', ... } }
+// diagnostics.resolvedDependencies[0].entryId === 'tool:ssh'
+if (requirements) {
+  const request: ContainerRequest = { ...requirements, command: 'git push' };
+}
+```
+
+## Entry model
+
+Each catalog entry has one unversioned **default** (`requirements` holding
+container access sections, its dependencies, and named **intents** such as
+`fetch` and `push`) and additive **overlays**: `platformVariants` for a
+platform or architecture, and `versionVariants` keyed by a purl `vers` range
+in the entry's required `versionScheme` (`npm`, `semver`, `pypi`, `nuget`, or
+`intdot`). Overlays use `policyAdditions`, `intentAdditions` (extending
+intents the default declares), and `newIntents`; they can never remove or
+narrow anything. The effective policy is the default plus the platform
+overlay plus at most one version overlay, and an intent then selects from it.
+
+## Resolution
+
+One call resolves one input or an array in a single pass. Each input is a
+tool candidate (`invocationName`, optional `packageUrl`, `detectedVersion`,
+and `intent`), or a bare name.
+
+- A `purl` match (on type, namespace, and name) is strong; an
+  `invocation-name` match is weak and needs `allowWeakIdentityFallback`.
+  Names compare case-insensitively on Windows and macOS and exactly on Linux.
+  Purl components other than type/namespace/name are ignored with a
+  `purl_components_ignored` warning; an unparseable purl is `purl_invalid`.
+- Per-input statuses:
+
+  | Situation | Status | Contributes |
+  |---|---|---|
+  | No version | `matched_default` | default (+ platform) |
+  | Version in a range | `matched_version` | default + that overlay |
+  | Valid version in no range | `version_out_of_range` | default (+ platform) |
+  | Unparseable version | `version_unparseable` | nothing |
+  | Intent not defined | `intent_unsupported` | nothing |
+  | No eligible entry | `tool_unmatched` | nothing |
+
+  No intent selects the base plus every intent. Other inputs still resolve.
+- `diagnostics.tools[i]` is `{ inputIndex, contributes, status, selection? }`.
+  `contributes` is true only when that input added access to the returned
+  requirements, so every flag is false when no requirements are returned.
+  `selection` (entry, match kind, version selection, intent selection) is
+  present once an entry is chosen. The Node and C# SDKs reject a result whose
+  shape breaks these rules with `backend_error`; unfamiliar status strings
+  are passed through.
+- `projectRoot` and `symbols.project_root` both bind `${project_root}`. The
+  same non-empty value in both is accepted; different values fail with
+  `invalid_context` before lookup. A caller-supplied symbol is reported as
+  a `symbol_resolved` warning with `source: "caller"`.
+- Dependencies contribute their default plus platform base additions; a
+  reference naming intents adds those intents. `resolvedDependencies` is
+  de-duplicated and attributes each dependency to every requesting input
+  (`inputIndexes`).
+- Composition satisfies every contributing pair, and never more: paths are
+  substituted, normalized, and de-duplicated per layer; read-write supersedes
+  read-only; overlapping catalog denies are removed with structured
+  warnings. `ui` and `timeoutMs` come from at most one entry: omission
+  elsewhere adds nothing, and two distinct entries supplying either fails
+  with `composition_conflict`, even when the values are equal. Alias paths
+  are preserved as written. Paths whose filesystem
+  object identity cannot be established fail closed for that pair. The
+  composed result is validated as an exact MXC 1.0.0 request before it is
+  returned.
+- Warnings are structured objects with a stable `code` and code-specific
+  fields (for example `entryId`, `detectedVersion`, `ignoredComponents`).
+
+Verifying that the executable really carries the strong identity passed in
+`packageUrl`, and that `detectedVersion` is accurate, is the caller's job.
+The caller's own restrictions always win.
+
+## Failures
+
+Failures reuse MXC's error codes, with a stable sub-reason: `details.reason`
+on the Node `MxcError` and `MxcException.Reason` in C#. The Rust `Error`
+carries the code only.
+
+| Code | Reason | Meaning |
+|---|---|---|
+| `policy_validation` | `invalid_catalog` | Catalog data violates the contract, or a resolved field does not fit the v1 section types. |
+| `policy_validation` | `composition_conflict` | The contributing pairs cannot be composed without granting broader access. |
+| `policy_validation` | `ambiguous_match` | An input matches more than one entry at the same rank. |
+| `malformed_request` | `invalid_context` | The caller's inputs or context are invalid. |
+| `unsupported_containment` | `unsupported_host` | The host platform, or a needed native architecture, cannot be determined. |
+| `backend_error` | `integrity` | Bundled catalog data cannot be read or declares the wrong revision. |
+| `backend_error` | `revision_unavailable` | An explicitly requested revision is not bundled. |
+
+Absence is not a failure: the resolve call yields no requirements.
+
+## Changing the catalog
+
+Authors edit one JSON file per tool under
+`src/mxc-sdk/policy_store/catalog/entries/`. Each file holds that tool's full
+entry: identity, default, intents, every platform and version variant,
+dependencies, and provenance. Optional category subdirectories are
+organizational only; a file's path never affects identity, lookup, or
+composition. `revisions/<catalogRevision>.json` and `views/` are generated,
+immutable published snapshots: never edit them by hand, and never rewrite one
+that is already published.
+
+1. Add a new `{ catalogRevision, file }` (`<YYYY-MM-DD.N>`,
+   `revisions/<YYYY-MM-DD.N>.json`) to `catalog/manifest.json` and point
+   `defaultRevision` at it. The data is compiled into the native library and
+   inherits MXC package signing.
+2. Edit the entry files. Bump `entryRevision` for every entry that changes.
+   Every entry needs a `versionScheme` and one `default`; overlays are
+   additive only, and version ranges in one entry must not overlap.
+3. Generate the revision snapshot from the entries with
+   `MXC_POLICY_STORE_UPDATE_REVISION=1`, then the views with
+   `MXC_POLICY_STORE_UPDATE_VIEWS=1` (both run
+   `cargo test -p mxc-sdk --test policy_store_catalog_validation`), and review
+   them. The assembler orders entries by `entryId` and rejects duplicate
+   `entryId` values and dependencies on entries no file defines.
+4. Add or update a case in `conformance/fixtures/bundled-catalog.json`. Every
+   entry in the default revision needs one. To record actual outcomes for
+   review, run the conformance test with
+   `MXC_POLICY_STORE_UPDATE_FIXTURES=<output-dir>`, then merge them by hand.
+5. Run the checks (from `src/`). They fail if a generated snapshot or view is
+   stale against its inputs:
+
+   ```text
+   cargo test -p mxc-sdk --lib policy_store
+   cargo test -p mxc-sdk --test policy_store_library --test policy_store_conformance --test policy_store_catalog_validation --test policy_store_vectors --test policy_store_sdk
+   cargo test -p mxc_ffi --test policy_store
+   ```
+
+   Set `MXC_POLICY_STORE_BASE_REF=origin/main` to also check that no
+   published revision changed.
+
+The JSON Schemas in `src/mxc-sdk/policy_store/schema/` describe the catalog
+shape for editors. The Rust validation in
+`tests/policy_store_catalog_validation.rs` covers the same rules and more,
+including every materialized effective policy as an exact MXC 1.0.0 request.

docs/policy-store/design.md

@@ -0,0 +1,72 @@
+# Policy Store prototype: implementation notes
+
+> **PROTOTYPE, pending API review.** Not part of MXC 1.0; a later MXC SDK
+> release is targeted. API names and shapes may change before sign-off.
+
+The spec is in [microsoft/mxc#1309](https://github.com/microsoft/mxc/pull/1309),
+head `5c2ba8e`: `docs/mxc-policy-store-api.md` is the caller-facing API
+contract and `docs/mxc-policy-store.md` the catalog design. This prototype
+tracks both. The spec is the source of truth; this
+page is not a copy of it. It records only how the prototype implements the
+spec, the decisions it made where the spec leaves room, and its known gaps.
+The [README](README.md) describes the API and the catalog workflow.
+
+## Shape
+
+- One resolver, an internal `policy_store` module of `mxc-sdk`
+  (`src/mxc-sdk/src/policy_store/`), with the V1 catalog compiled in from
+  `src/mxc-sdk/policy_store/catalog/`. There is no separate crate, library,
+  or CLI.
+- Catalog sources follow design §6.3: one editable file per tool in
+  `catalog/entries/`. `assemble_revision` (a checked generator, like the
+  reviewer views) builds the default revision snapshot in `entryId` order and
+  rejects duplicate IDs and dangling dependencies; a cargo test fails when the
+  checked-in snapshot is stale. The static, manifest-selected embedding is
+  unchanged.
+- Rust: `mxc_sdk::v1::{resolve_tool_requirements,
+  resolve_tool_requirements_with_diagnostics, get_catalog_info,
+  list_catalog_entries}` return `ContainerRequirements` built from the v1
+  section types. Lookup types live in `mxc_sdk::v1::tool_requirements`.
+- Node and .NET call the same resolver through panic-contained `mxc_ffi`
+  exports. Node resolution is promise-based (`koffi` async calls, so the
+  event loop is not blocked); catalog inspection is synchronous. .NET exposes
+  `MxcContainer.ResolveToolRequirements` and
+  `ResolveToolRequirementsWithDiagnostics` with `Async` forms.
+- Catalog revision `2026-10-06.1` (`sdkContractVersion` `1.0.0`). Every
+  effective policy is materialized at build-test time, bound to fixture
+  symbols and a validation-only command, and validated as an exact MXC 1.0.0
+  `ContainerRequest`; `catalog/views/<revision>.requests.json` records those
+  requests and `<revision>.md` is the reviewer view.
+
+## Decisions where the spec leaves room
+
+| Topic | Prototype behavior |
+|---|---|
+| purl parsing | An `@` before the last `/` is an unencoded npm scope, not a version (`pkg:npm/@scope/name`). Matching compares type, namespace, and name only; other components produce `purl_components_ignored`. |
+| purl name normalization | No per-type normalization beyond the spec's comparison rule; open in the spec. |
+| Empty `packageUrl` | `malformed_request` / `invalid_context`, like other invalid inputs. |
+| Filesystem identity on a foreign platform | A lookup for a platform other than the host cannot examine object identity, so affected pairs fail closed. Tests resolve for the host platform against existing temporary directories. |
+| Identity read failures | A discovery read failure maps to `unsupported_containment` / `unsupported_host`. |
+| Read-only alias promotion | Reported as `readonly_superseded`. Alias paths are kept as written. |
+| Lookup-time exact validation | A composed result that fails exact MXC 1.0.0 validation maps to `policy_validation` / `composition_conflict`. |
+| `architecture_fallback` | Also emitted for dependencies and for the default selection. |
+| `npm_cache` symbol | Discovery is limited; callers pass it in `symbols`. |
+| Error reason | Node `details.reason` and .NET `MxcException.Reason` carry the stable reason. The Rust `Error` carries the code only. |
+| FFI request | The `{"tools", "context"?}` lookup request is catalog-lookup input, not policy ingress, so it is not an exact versioned contract. |
+| .NET runner contract | The lookup is on `MxcContainer` per the spec but is not part of `IContainerRunner`, which models container execution. |
+
+## Known gaps
+
+- Final API names are pending API review.
+- The exact policy validator and version mapping, and purl normalization and
+  equality, follow the spec's open items and may change.
+- The JSON Schemas in `policy_store/schema/` are editor aids; the Rust
+  catalog-validation tests are authoritative.
+
+## Tests
+
+From `src/`: `cargo test -p mxc-sdk --lib policy_store`, the
+`policy_store_*` integration tests in `src/mxc-sdk/tests/`, and
+`cargo test -p mxc_ffi --test policy_store`. Node:
+`sdk/node/tests/unit/policy-store.test.ts`. .NET:
+`sdk/dotnet/Microsoft.Mxc.Sdk.Tests/V1/ToolRequirementsTests.cs`.

sdk/dotnet/Microsoft.Mxc.Sdk.AotSmokeTest/Program.cs

@@ -361,4 +361,49 @@ static void CheckRoundTrip<T>(T value, Func<T, bool> check, string name)
 Check(probe.Probes.BaseContainerApiPresent, "probe facts parsed");
 Check(probe.Probes.UiCapabilities.CanBlockClipboardRead, "probe ui capabilities parsed");
 
+// 11. Prototype policy store DTOs (pending API review): the lookup request
+//     writer, the diagnostics result with absent and retained selection and
+//     true/false contribution flags, rejected missing/non-boolean flags, and
+//     the inspection metadata roots.
+var lookupRequest = MxcContainer.ToolRequirementsRequest(
+    new ToolCandidate[] { "git", new("npm") { PackageUrl = "pkg:npm/npm", Intent = "install" } },
+    new ResolveContext { ProjectRoot = "C:\\repo", Symbols = new Dictionary<string, string> { ["npm_cache"] = "C:\\c" } });
+Check(lookupRequest[^1] == 0, "lookup request is NUL-terminated");
+const string resolutionJson = """
+{"requirements":{"filesystem":{"readwritePaths":["C:\\repo"]},"timeoutMs":5},
+ "diagnostics":{"catalogRevision":"r","tools":[
+  {"inputIndex":0,"contributes":true,"status":"matched_default","selection":{"entryId":"tool:git","entryRevision":1,
+   "matchedIdentities":[{"kind":"invocation-name","strength":"weak"}],"versionSelection":{"status":"matched_default"},
+   "intentSelection":{"mode":"all","selected":["fetch"]}}},
+  {"inputIndex":1,"contributes":false,"status":"version_unparseable","selection":{"entryId":"tool:npm","entryRevision":1,
+   "matchedIdentities":[{"kind":"purl","strength":"strong"}],"versionSelection":{"status":"version_unparseable","detectedVersion":"x"}}},
+  {"inputIndex":2,"contributes":false,"status":"tool_unmatched"}],
+  "resolvedDependencies":[],
+  "warnings":[{"code":"tool_unmatched","inputIndex":2,"invocationName":"nope","message":"m"},
+   {"code":"symbol_resolved","inputIndexes":[0],"entryIds":["tool:git"],"symbol":"project_root","value":"C:\\repo","source":"caller","message":"m"}]}}
+""";
+var lookup = MxcContainer.ParseToolRequirementsResolution(resolutionJson, requireDiagnostics: true);
+Check(lookup.Requirements?.TimeoutMs == 5, "policy store requirements parsed");
+Check(lookup.Diagnostics.Tools[0] is { Contributes: true, Selection.EntryId: "tool:git" }, "contributing selection parsed");
+Check(lookup.Diagnostics.Tools[1] is { Contributes: false, Selection.EntryId: "tool:npm" }, "failed selection retained");
+Check(lookup.Diagnostics.Tools[2] is { Contributes: false, Selection: null }, "absent selection parsed");
+Check(lookup.Diagnostics.Warnings.Count == 2, "structured warnings parsed");
+foreach (var flag in new[] { "", "\"contributes\":\"true\"," })
+{
+    var malformed = $$$"""{"diagnostics":{"catalogRevision":"r","tools":[{"inputIndex":0,{{{flag}}}"status":"tool_unmatched"}],"resolvedDependencies":[],"warnings":[]}}""";
+    try
+    {
+        MxcContainer.ParseToolRequirementsResolution(malformed, requireDiagnostics: true);
+        Check(false, "missing or non-boolean contributes is rejected");
+    }
+    catch (MxcException error) when (error.Code == ErrorCode.BackendError)
+    {
+    }
+}
+var catalogInfo = MxcJson.Deserialize<CatalogInfo>(
+    """{"catalogSchemaVersion":"1","catalogRevision":"r","sdkContractVersion":"1.0.0"}""",
+    MxcJson.PolicyStoreOptions);
+Check(catalogInfo?.SdkContractVersion == "1.0.0", "catalog info parsed");
+CheckRoundTrip(catalogInfo!, x => x.CatalogRevision == "r", "catalog info round-trips");
+
 Console.WriteLine("AOT smoke test passed: all JSON paths are reflection-free.");