Add verbose Learning Mode denial logging

.github/copilot-instructions.md

@@ -280,7 +280,7 @@ The workspace is organized into six top-level directories under `src/`:
 - The lower-level execution surface lives in `wxc_common::sandbox_process`: the `SandboxBackend` trait (`validate` + `spawn(request, logger, StdioMode) -> Box<dyn SandboxProcess>` + a `diagnose_exit` hook) and the generic `Runner<B>` adapter that bridges any `SandboxBackend` to the run-to-completion `ScriptRunner` (via `spawn(StdioMode::Inherit)` then `wait()`). `SandboxProcess::output_metadata()` carries backend-produced structured outputs after terminal teardown without writing to process-global stdio. `StdioMode::Pipes` hands the caller live stdin/stdout/stderr (what the `mxc-sdk` streaming path uses); `StdioMode::Inherit` lets the child inherit the host's stdio (what the executor binaries use, preserving the TTY under a pty). `SandboxBackend` is implemented for Seatbelt, Bubblewrap, Windows ProcessContainer, and WSLC (on `wslc_common::WSLContainerRunner` itself, which shares one container lifecycle — `start_container` — between its streaming `SandboxBackend` and run-to-completion `ScriptRunner` impls, differing only in where the WSLC SDK's output callbacks write).
 - `mxc_ffi` (`ffi/mxc_ffi`, `crate-type = ["cdylib", "staticlib", "lib"]`) is a flat, panic-safe **C ABI over `mxc-sdk`** for language bindings. `mxc_run(policyJson, command, out)` runs a sandbox to completion, filling a `#[repr(C)] MxcRunResult` (status + exit_code + timed_out + owned stdout/stderr/error/output-metadata C strings); every entry point is `catch_unwind`-wrapped so a panic becomes a status code, never an unwind. Its `build.rs` runs **csbindgen** to generate the C# P/Invoke (`sdk/dotnet/Microsoft.Mxc.Sdk/Native/NativeMethods.g.cs`), gated behind the crate's **`dotnetsdk`** feature (off by default, so the whole-workspace backend build matrix doesn't compile csbindgen). The generated file is **not committed** (gitignored); the C# csproj regenerates it at build time and `scripts/check-dotnet-bindings-codegen.js` runs the codegen in CI and asserts the expected entry points are produced. The C ABI is **not a stable external contract** (native + binding are co-versioned and generated together; see the crate docs). It exposes three surfaces: **run-to-completion** (`mxc_run`), **streaming** (`mxc_spawn` → opaque `MxcSandbox` handle; `mxc_stream_read`/`write`/`flush`, `mxc_sandbox_take_stdin`/`stdout`/`stderr`, `mxc_sandbox_id`/`try_wait`/`wait`/`kill`/`output_metadata_json`/`free`, in `src/streaming.rs`), and the **state-aware lifecycle** (`mxc_state_aware` for the envelope phases + `mxc_state_aware_exec` returning a live streaming handle, in `src/state_aware.rs`). All three `.rs` files are csbindgen inputs in `build.rs`; the `MXC_STATUS_*` space already reserves the state-aware phase codes.
 - `mxc_pty` is the shared pty bridge used by the LXC backend (`lxc_common::lxc_bindings::attach_run`) so the inner shell sees a real TTY and host stdio is streamed live. (Seatbelt and Bubblewrap no longer use it: they spawn directly and let the child inherit the host's stdio — a TTY when the executor binary runs under a pty — via `SandboxBackend::spawn(StdioMode::Inherit)`.)
-- `learning_mode_core` is the **cross-platform learning-mode / captureDenials model + output emitter**: `DeniedResource` (+ `ResourceType`/`AccessType`), `DenialSummary`, the `DenialAnalyzer` decode trait, and `emit` — which writes the on-disk denials deliverable as a **single JSON document** `{ "denials": [...], "summary": {...} }` (`write_document` / `DenialsDocument`) and defines the serializable `DenialsOutputPointer`. It carries no OS-specific code (must not depend on any `backends/*` crate); the Windows ETL decoder implementing `DenialAnalyzer` lives in `backends/learning_mode/windows`. When `processContainer.captureDenials` is set, native PSEC/V2 seals and decodes a managed ETL locally, while guarded WPR relogs its host-wide source into a process-scoped ETL before analysis; both routes write the same canonical JSON through shared output plumbing and return neutral `wxc_common` metadata. Explicit `retainEtl` preserves the native sealed trace or the guarded process-scoped relogged trace after a terminal wait; abandonment discards it. `wxc-exec` serializes the metadata as the one-line stderr pointer at the CLI boundary; Rust/C#/FFI callers receive it programmatically. Each denial's `resource` field holds the file path or the AppContainer capability name; capability denials resolve their capability SID to a friendly name via `backends/learning_mode/windows`'s `capability_names` (well-known `S-1-15-3-…` SID → policy name; custom hashed SIDs fall back to the SID string).
+- `learning_mode_core` is the **cross-platform learning-mode / captureDenials model + output emitter**: `DeniedResource` (+ `ResourceType`/`AccessType`), `DenialSummary`, the `DenialAnalyzer` decode trait, canonical `DenialsDocument` emission, bounded sensitive-value-redacted `VerboseLoggingDocument` emission, and transactional paired-file write/relocation helpers. Every successful analysis writes the canonical policy artifact plus its deterministic `*.verbose.json` sibling; relocation reports destination commit separately from source-cleanup warnings so callers keep metadata truthful. The crate also defines the serializable `DenialsOutputPointer` and carries no OS-specific code (it must not depend on any `backends/*` crate). The Windows ETL decoder implementing `DenialAnalyzer` lives in `backends/learning_mode/windows`. When `processContainer.captureDenials` is set, native PSEC/V2 seals and decodes a managed ETL locally, while guarded WPR relogs its host-wide source into a process-scoped ETL before analysis; both routes write the same output pair through shared plumbing and return neutral `wxc_common` metadata. Explicit `retainEtl` preserves the native sealed trace or the guarded process-scoped relogged trace after a terminal wait; abandonment discards it. `wxc-exec` serializes the metadata as the one-line stderr pointer at the CLI boundary; Rust/C#/FFI callers receive it programmatically. Each canonical denial's `resource` field holds the file path, named object, UI category, or AppContainer capability name; capability denials resolve their capability SID to a friendly name via `backends/learning_mode/windows`'s `capability_names` (well-known `S-1-15-3-…` SID → policy name; custom hashed SIDs fall back to the SID string).
 - `mxc_build_common` is a build-time helper crate — all Windows binary crates use it in their `build.rs` to embed VersionInfo (ProductName, FileDescription, copyright, version+commit). When adding a new Windows binary crate, add `mxc_build_common` as a build-dependency and call `mxc_build_common::embed_version_info()` from `build.rs`
 - `nanvix_build_common` is a **build-only** helper crate (never linked into the runtime): it stages NanVix binaries next to the executable and resolves the `NANVIX_BIN` prefetch directory. The `nanvix_binaries`, `wxc`, and `lxc` build scripts consume it as a `[build-dependencies]` entry. Runtime constants it needs (binary/snapshot filenames) stay in `nanvix_common`. Keep build-only file-staging logic here, not in `nanvix_common` (which is a runtime dependency of `nanvix_runner`).
 - Platform-specific modules use `#[cfg(target_os = "windows")]` / `#[cfg(target_os = "linux")]`

docs/learning-mode/capabilities.md

@@ -105,9 +105,10 @@ stays enforced:
    hosts use native capture without PLM or UAC, while legacy or incompatible
    tiers use the session-scoped guarded-WPR fallback and elevate only its
    fixed-operation guardian. The CLI consumes the returned JSON and ETL paths,
-   relocates them to `denials.json` and `trace.etl`, and generates the source
-   snapshot and `Adjusted_*.json` from canonical denials without decoding ETL
-   again. Truncated analysis skips the adjusted config.
+   relocates the policy output, its verbose logging sibling, and the trace to
+   `denials.json`, `denials.verbose.json`, and `trace.etl`, and generates the
+   source snapshot and `Adjusted_*.json` from the policy denials without decoding
+   ETL again. Truncated analysis skips the adjusted config.
 
    ```
    wxc-exec --audit --config <config>
@@ -171,8 +172,8 @@ ungranted access is handled while it is recorded:
 ### Output file the caller consumes
 
 After the sandboxed workload exits, MXC decodes the captured denials and writes
-them to a **single JSON file** — the deliverable a host application reads to
-regenerate its sandbox policy:
+the policy JSON deliverable a host application reads to regenerate its
+sandbox policy:
 
 ```json
 {
@@ -203,14 +204,18 @@ regenerate its sandbox policy:
 - `denials` is already de-duplicated per `(resource, accessType)`, so
   `summary.totalDenials` equals `denials.length`.
 - Analysis retains at most 10,000 unique denials and processes at most
-  1,000,000 ETW events. Reaching either bound stops further analysis and sets
-  `summary.deniedResourcesTruncated` to `true`.
+  1,000,000 ETW events. Reaching the unique-denial bound stops adding policy
+  entries but continues bounded diagnostic accounting; reaching either bound
+  sets `summary.deniedResourcesTruncated` to `true`.
 - `resource` is the user-visible identifier for the denied resource,
-  interpreted by `resourceType`: a canonical `C:\…` path for `file`, the
+  interpreted by `resourceType`: an absolute `C:\…` path for `file`, the
   AppContainer **capability name** (e.g. `internetClient`) for `capability`,
-  and the raw resource identifier otherwise. Well-known capability SIDs are
+  and the raw resource identifier otherwise. Named Section, SymbolicLink, and
+  Timer access checks are emitted as `other`. Well-known capability SIDs are
   resolved to their policy name; custom (hashed) capability SIDs that can't be
-  reversed fall back to the `S-1-15-3-…` SID string.
+  reversed fall back to the `S-1-15-3-…` SID string. Event 28 is
+  schema-discriminated: UI-shaped `Category`/`Detail` payloads emit `ui`
+  resources instead of treating the package SID as a capability.
 - `resourceType` is one of `file`, `ui`, `network`, `capability`, `other`;
   `accessType` is one of `read`, `write`, `execute`, `unknown`. Capability
   denials are recorded under `block`; current `allow` traces expose capability
@@ -219,6 +224,107 @@ regenerate its sandbox policy:
 - `filetime` is a decimal string containing the Windows `FILETIME` value, so
   JavaScript consumers retain all 64 bits without numeric precision loss.
 
+### Verbose logging event signatures
+
+Every successful decode also writes a deterministic sibling file:
+`denials.<run-id>.json` produces `denials.<run-id>.verbose.json`. This verbose
+logging artifact is a bounded, sensitive-value-redacted superset containing
+policy denial occurrences plus diagnostic outcomes omitted from the policy file:
+
+```json
+{
+  "version": 1,
+  "signatures": [
+    {
+      "signature": {
+        "provider": "kernelGeneral",
+        "providerGuid": "{A68CA8B7-004F-D7B6-A698-07E2DE0F1F5D}",
+        "eventId": 14,
+        "reason": "canonicalDenial",
+        "pid": 4321,
+        "accessType": "read",
+        "resourceType": "file",
+        "properties": [
+          ["PackageSid", "S-1-15-3-1"],
+          ["resource", "<REDACTED>"]
+        ]
+      },
+      "count": 37
+    }
+  ],
+  "summary": {
+    "totalOccurrences": 37,
+    "overflowOccurrences": 0,
+    "canonicalOverflowOccurrences": 0,
+    "aggregateGroupsTruncated": false,
+    "processedEventsTruncated": false,
+    "canonicalDenialLimitReached": false
+  }
+}
+```
+
+Signatures are keyed by symbolic provider category, provider GUID,
+provider-scoped event ID, closed exclusion reason, PID, and sorted sanitized
+properties. SIDs, capability names, GUIDs, PIDs/process identifiers, and
+non-file resource values are retained. Complete file paths are replaced with
+`<REDACTED>`; standalone user/account names remain replaced with
+`<redacted-user>`.
+Exact header timestamps and timestamp-like properties are omitted so otherwise
+identical events deduplicate, and free-form decoder errors are never serialized.
+
+Every valid denial candidate is classified as `canonicalDenial` in the verbose
+file, including its first occurrence, later duplicates, and candidates observed
+after the policy file's unique-denial bound. Those occurrences deduplicate under the
+same signature and increment its count. `accessType` and `resourceType` are
+included when denial extraction determined them; diagnostic outcomes without
+those classifications omit the fields.
+
+Candidates excluded from the policy output retain a closed diagnostic
+reason and their sanitized event properties:
+
+- `unusableResourcePath` means a File access-check resource could not be
+  converted to a safe absolute DOS or UNC path. For example,
+  `\Device\MountPointManager` is useful Devices-namespace evidence, but it is
+  not a directly authorable filesystem grant.
+- `unsupportedObjectType` means the event names a resource outside the
+  supported policy model. Observed Section, SymbolicLink, and Timer checks are
+  retained as `other` resources; remaining examples include
+  `\BaseNamedObjects` as a Directory, ALPC Ports such as
+  `ubpmtaskhostchannel`, and RPC Interface GUIDs.
+
+Property values longer than 256 characters retain bounded prefix and suffix
+context plus a SHA-256 digest of the complete sanitized value. This keeps long
+named-object resources individually identifiable when they share a prefix
+without exceeding the per-property bound. Redaction occurs before the digest is computed, so neither retained context nor
+a digest is derived from a sensitive value.
+
+Unknown event IDs from known Learning Mode providers are classified as
+`unsupportedEventSchema`; the real ETL path retains their provider GUID and
+PID without attempting an unsupported TDH payload decode.
+
+Per-event TDH failures use closed diagnostic reasons:
+`eventPayloadMalformed` means the payload conflicts with its declared schema,
+`decoderLimitReached` means a nesting/element/work safety bound stopped
+decoding, and `unsupportedPropertyEncoding` means the decoder cannot consume
+that property shape. When TDH exposes it, the schema-declared name is retained
+as the bounded `EventName` signature property. Free-form decoder errors are
+never serialized. Failure to obtain the event schema remains a fatal analysis
+error rather than being represented as a verbose logging signature.
+
+To keep diagnostics bounded, verbose logging retains at most 4,096 distinct
+signatures, 24 sorted properties per signature, and 256 characters per property
+value. `overflowOccurrences` and `aggregateGroupsTruncated` indicate that
+additional diagnostic groups were omitted. `canonicalOverflowOccurrences`
+counts omitted policy-denial occurrences, while `processedEventsTruncated`
+indicates that the 1,000,000-event limit prevented complete accounting. The
+policy file itself is never reduced to make room for verbose logging.
+
+The policy and verbose logging files fail together: MXC stages both and reports
+capture failure unless both final artifacts are committed. The verbose logging path
+is intentionally absent from stderr pointers and Rust, Node, C#, and FFI output
+metadata; callers derive it from the policy output path using the naming rule
+above.
+
 **Locating the file.** Set `captureDenials.outputPath` to name the file
 explicitly (its parent directory must already exist). MXC inserts a unique
 per-run identifier (process id plus random suffix) into the file stem
@@ -232,8 +338,8 @@ so CLI callers can locate the deliverable without scanning the filesystem:
 {"type":"captureDenials","outputPath":"C:\\logs\\denials.4321_0123456789abcdef0123456789abcdef.json","exitCode":0,"totalDenials":2,"deniedResourcesTruncated":false}
 ```
 
-The pointer echoes the file's `summary`; the authoritative record is the file
-itself. In-process Rust callers receive the same information through
+The pointer echoes the policy file's `summary`; that file is the authoritative
+record of denials. In-process Rust callers receive the same summary information through
 `Output::output_metadata` or `Sandbox::output_metadata()` after waiting. The
 C# SDK exposes it through `RunResult.OutputMetadata` and
 `MxcSandboxProcess.OutputMetadata`.

src/Cargo.toml

@@ -90,6 +90,7 @@ plm = { path = "host/plm" }
 quick-xml = "0.41"
 sandbox_spec = { path = "core/generated/base_container_specification" }
 seatbelt_common = { path = "backends/seatbelt/common" }
+same-file = "1"
 serde = { version = "1", features = ["derive"] }
 serde_json = "1"
 serde_path_to_error = "0.1"