feat(skippy): wire next-generation KV cache and publisher defaults

.agents/skills/manage-ci/references/current-inventory.md

@@ -225,7 +225,7 @@ it after the protected-main runner-contract update is active.
 | `ci-website-lane.yml` | Console and website graph; reusable from PRs and dispatchable for main/manual |
 | `ci-linux-lane.yml` | Linux host/runtime/product/Rust/SDK/smoke graph with one platform-local UI producer |
 | `ci-macos-lane.yml` | macOS host/runtime/product/platform/Swift/Metal graph with one platform-local UI producer |
-| `ci-windows-lane.yml` | Windows host/runtime/product/platform graph with one platform-local UI producer |
+| `ci-windows-lane.yml` | Windows host/runtime/product/platform graph plus the CPU durable-L3 product qualification, with one platform-local UI producer |
 | `ci-quality-slice.yml` | Contracts, format, Clippy and generated CLI inventory freshness; additive protected authority sentinel |
 | `ci-web-slice.yml` | Console quality, console Playwright E2E, public website build, and CLI explorer browser validation |
 | `ci-ui-artifact-slice.yml` | Immutable console distribution producer; release callers prepare one source/version-bound UI with complete file checksums, shared by all hosts and SDK resources |
@@ -235,7 +235,7 @@ it after the protected-main runner-contract update is active.
 | `ci-{linux,macos,windows}-runtime-slice.yml` | Platform-pure native runtime producers. The Linux CPU row also runs the native runtime-event gate against the runtime it just built and uploads its evidence. |
 | `ci-{linux,macos,windows}-product-slice.yml` | Platform-pure composition-only product consumers |
 | `ci-platform-checks-slice.yml` | macOS portable/unit, Windows portable, and Windows log-store privacy ACL checks |
-| `ci-linux-product-smoke-slice.yml`, `ci-macos-product-smoke-slice.yml` | Platform-local callers of the typed CPU/CUDA/Vulkan (`gpu-nvidia` self-hosted), conditional ROCm (`gpu-amd`), and Metal product-integration suite plus model-download. The suite stages the registry-pinned SmolLM2 Q8 and IBM Granite 4.0 H Q4 pair once, runs dense standalone/SDK/restart, then dense passive-client split routing and strict recurrent `KvRecurrent` validation. Each split phase persists strict-whitelist seed/worker node, mesh, and peer identity plus stage/model snapshots, then atomically reconciles exact two-observer, topology/run/model/package/manifest, two-stage contiguous-cut and bind-address, ready-status, and served-model agreement. A capped five-minute wall-clock deadline with parallel, bounded endpoint capture finalizes failure evidence before workflow cancellation; the status projection excludes invite tokens, nested fields, and unrelated paths. Product reconciliation independently verifies both evidence files, records their paths and SHA-256 digests in `phase-results.json`, rejects missing or modified evidence, and uploads every JSON snapshot/evidence file with logs on success or failure. Linux CUDA packages admit only the reviewed cudart, cuBLAS, cuBLASLt, and nvJitLink families for the declared CUDA major, retain NVIDIA object bytes, and include the toolkit distribution license. The Linux CUDA smoke verifies that closure with `LD_LIBRARY_PATH` unset; cudart and cuBLAS are not installed by apt, and the NVIDIA driver remains host-owned. Before inference, it records CUDA visibility variables, host driver-library resolution and NVIDIA device nodes, then runs the packaged benchmark's device-count probe without benchmark allocations, using inherited and strict packaged-library resolution. ROCm skips unless `MESH_ROCM_INFERENCE_RUNNER_ENABLED` is exactly `true`; accelerator product-integration rows remain outside the checked plan pending live qualification. |
+| `ci-{linux,macos,windows}-product-smoke-slice.yml` | Platform-local callers of the typed CPU/CUDA/Vulkan (`gpu-nvidia` self-hosted), conditional ROCm (`gpu-amd`), Metal, and Windows CPU product-integration suite plus model-download. The suite stages the registry-pinned SmolLM2 Q8 and IBM Granite 4.0 H Q4 pair once, runs dense standalone/SDK/restart, then dense passive-client split routing and strict recurrent `KvRecurrent` validation. Each split phase persists strict-whitelist seed/worker node, mesh, and peer identity plus stage/model snapshots, then atomically reconciles exact two-observer, topology/run/model/package/manifest, two-stage contiguous-cut and bind-address, ready-status, and served-model agreement. A capped five-minute wall-clock deadline with parallel, bounded endpoint capture finalizes failure evidence before workflow cancellation; the status projection excludes invite tokens, nested fields, and unrelated paths. Product reconciliation independently verifies both evidence files, records their paths and SHA-256 digests in `phase-results.json`, rejects missing or modified evidence, and uploads every JSON snapshot/evidence file with logs on success or failure. Linux CUDA packages admit only the reviewed cudart, cuBLAS, cuBLASLt, and nvJitLink families for the declared CUDA major, retain NVIDIA object bytes, and include the toolkit distribution license. The Linux CUDA smoke verifies that closure with `LD_LIBRARY_PATH` unset; cudart and cuBLAS are not installed by apt, and the NVIDIA driver remains host-owned. Before inference, it records CUDA visibility variables, host driver-library resolution and NVIDIA device nodes, then runs the packaged benchmark's device-count probe without benchmark allocations, using inherited and strict packaged-library resolution. A separately reconciled durable-L3 phase preserves per-node roots and identities across full process restarts for both model families, requires active CLI-sourced disk configuration, persisted inventory, a post-restart L3 fill with cached tokens and exact output, then verifies status and clear. Windows CPU runs the durable phase alone on `windows-2022`; Linux CPU and macOS Metal run the complete suite. ROCm skips unless `MESH_ROCM_INFERENCE_RUNNER_ENABLED` is exactly `true`; accelerator product-integration rows remain outside the checked plan pending live qualification. |
 | `ci-linux-sdk-slice.yml`, `ci-macos-sdk-slice.yml` | Platform-local Rust/Kotlin/Swift smoke consumers; SDK producers are independent top-level calls and each smoke receives the lane-local immutable UI artifact |
 | `ci-runner-contract-slice.yml` | Provider/cache/plan trust and main runner-image checks |
 | `native-sdk-artifact.yml` | Typed native SDK producer |
@@ -390,7 +390,7 @@ does not grant; GitHub rejects at run creation with a **zero-job
 `actionlint` cannot see it. Containerizing surfaced this because
 `packages: read` (needed to pull the private GHCR runner images) has to be
 granted at *every* hop, and
-`ci-linux-product-smoke-slice.yml` / `ci-macos-product-smoke-slice.yml` sat at
+the platform product-smoke slices sat at
 `contents: read` between granted parents and requesting children.
 `scripts/tests/test_ci_workflow_permission_contract.py` walks every local
 `uses: ./.github/workflows/X.yml` edge and asserts the caller's effective
@@ -797,7 +797,7 @@ Current image references and historical null evidence remain unchanged.
 
 The `product-smoke` catalog role covers both the legacy `smoke.yml` job and
 the typed `product-integration-smoke.yml` job. The latter uses the same pinned
-CPU image only for Linux CPU; accelerator and macOS paths retain their existing
+CPU image only for Linux CPU; accelerator, macOS, and Windows paths retain their
 container opt-outs. The inventory has 9 images, 32 roles and 33 literal workflow image bindings.
 
 ### Qualified lean UI consumers

.github/actions/plan-ci/action.yml

@@ -284,7 +284,7 @@ runs:
                 rust_tests: .matrices.rust_tests,
                 hosts: [.matrices.hosts[] | select(.platform == "linux")],
                 runtime_products: [.matrices.runtime_products[] | select(.platform == "linux")],
-                smoke: [.matrices.smoke[] | select(.id != "metal-model-load")],
+                smoke: [.matrices.smoke[] | select(.id != "metal-model-load" and .id != "product-integration-metal" and .id != "product-integration-windows-cpu")],
                 sdk: [.matrices.sdk[] | select(.platform == "linux")]
               }
             }
@@ -309,7 +309,7 @@ runs:
                 hosts: [.matrices.hosts[] | select(.platform == "macos")],
                 runtime_products: [.matrices.runtime_products[] | select(.platform == "macos")],
                 platform_checks: [.matrices.platform_checks[] | select(.platform == "macos")],
-                smoke: [.matrices.smoke[] | select(.id == "metal-model-load")],
+                smoke: [.matrices.smoke[] | select(.id == "metal-model-load" or .id == "product-integration-metal")],
                 sdk: [.matrices.sdk[] | select(.platform == "macos")]
               }
             }
@@ -326,7 +326,8 @@ runs:
               matrices: {
                 hosts: [.matrices.hosts[] | select(.platform == "windows")],
                 runtime_products: [.matrices.runtime_products[] | select(.platform == "windows")],
-                platform_checks: [.matrices.platform_checks[] | select(.platform == "windows")]
+                platform_checks: [.matrices.platform_checks[] | select(.platform == "windows")],
+                smoke: [.matrices.smoke[] | select(.id == "product-integration-windows-cpu")]
               }
             }
             | .required = ([.matrices[] | length] | add) > 0

.github/actions/restore-product-integration-inputs/action.yml

@@ -11,6 +11,10 @@ inputs:
   staged_binary_path:
     required: true
     description: Stable executable path used by the product suite.
+  binary_name:
+    required: false
+    default: mesh-llm
+    description: Host executable filename inside the composed product.
   model_cadence:
     required: true
     description: Authorized registry cadence for this invocation.
@@ -50,6 +54,7 @@ runs:
       with:
         artifact_name: ${{ inputs.artifact_name }}
         artifact_path: ${{ inputs.artifact_path }}
+        binary_name: ${{ inputs.binary_name }}
         staged_binary_path: ${{ inputs.staged_binary_path }}
 
     - name: Resolve pinned product-integration fixtures

.github/workflows/ci-linux-lane.yml

@@ -179,7 +179,7 @@ jobs:
       source_sha: ${{ inputs.source_sha }}
       smoke_matrix: ${{ toJson(fromJson(inputs.lane_plan_json).matrices.smoke) }}
       binary_target: target/release/mesh-llm
-      timeout_minutes: 30
+      timeout_minutes: 45
     secrets:
       HF_TOKEN: ${{ inputs.original_event_name == 'push' && secrets.HF_TOKEN || '' }}
 

.github/workflows/ci-macos-lane.yml

@@ -170,7 +170,7 @@ jobs:
       architecture: ${{ fromJson(inputs.lane_plan_json).matrices.runtime_products[0].architecture }}
       smoke_matrix: ${{ toJson(fromJson(inputs.lane_plan_json).matrices.smoke) }}
       binary_target: target/release/mesh-llm
-      timeout_minutes: 30
+      timeout_minutes: 45
 
   summary:
     name: CI / macOS

.github/workflows/ci-windows-lane.yml

@@ -36,6 +36,9 @@ on:
         required: false
         default: "[]"
         type: string
+    secrets:
+      HF_TOKEN:
+        required: false
   workflow_dispatch:
     inputs: *lane_inputs
 permissions:
@@ -98,6 +101,18 @@ jobs:
       max_parallel: ${{ fromJson(inputs.lane_plan_json).budgets.windows_max_parallel }}
       fail_fast: ${{ inputs.original_event_name == 'pull_request' }}
 
+  product_smoke:
+    name: Windows product smoke
+    needs: [runtime_product]
+    if: ${{ !cancelled() && needs.runtime_product.result == 'success' && fromJson(inputs.lane_plan_json).matrices.smoke[0] != null }}
+    uses: ./.github/workflows/ci-windows-product-smoke-slice.yml
+    with:
+      source_sha: ${{ inputs.source_sha }}
+      smoke_matrix: ${{ toJson(fromJson(inputs.lane_plan_json).matrices.smoke) }}
+      timeout_minutes: 45
+    secrets:
+      HF_TOKEN: ${{ secrets.HF_TOKEN }}
+
   platform_checks:
     name: Windows checks
     if: ${{ fromJson(inputs.lane_plan_json).matrices.platform_checks[0] != null }}
@@ -113,7 +128,7 @@ jobs:
 
   summary:
     name: CI / Windows
-    needs: [ui_artifact, hosts, native_runtimes, runtime_product, platform_checks]
+    needs: [ui_artifact, hosts, native_runtimes, runtime_product, product_smoke, platform_checks]
     if: ${{ !cancelled() }}
     runs-on: ubuntu-24.04
     permissions:

.github/workflows/ci-windows-product-smoke-slice.yml

@@ -0,0 +1,41 @@
+name: CI Windows Product Smoke Slice
+
+on:
+  workflow_call:
+    inputs:
+      source_sha:
+        description: Immutable product source revision.
+        required: true
+        type: string
+      smoke_matrix:
+        description: JSON Windows smoke rows from the versioned CI plan.
+        required: true
+        type: string
+      timeout_minutes:
+        description: Bounded smoke timeout.
+        required: false
+        default: 45
+        type: number
+    secrets:
+      HF_TOKEN:
+        required: false
+
+permissions:
+  contents: read
+  packages: read
+
+jobs:
+  product_integration_cpu:
+    name: Windows CPU durable L3 qualification
+    if: ${{ contains(fromJson(inputs.smoke_matrix).*.id, 'product-integration-windows-cpu') }}
+    uses: ./.github/workflows/product-integration-smoke.yml
+    with:
+      source_sha: ${{ inputs.source_sha }}
+      artifact_name: ci-product-windows-amd64-cpu
+      artifact_path: ci-artifacts/windows
+      staged_binary_path: ci-artifacts/windows/mesh-llm.exe
+      platform: windows
+      backend: cpu
+      timeout_minutes: ${{ inputs.timeout_minutes }}
+    secrets:
+      HF_TOKEN: ${{ secrets.HF_TOKEN }}

.github/workflows/main_windows.yml

@@ -78,6 +78,8 @@ jobs:
       source_sha: ${{ needs.plan.outputs.source_sha }}
       original_event_name: push
       supersession_key: main-${{ github.sha }}
+    secrets:
+      HF_TOKEN: ${{ secrets.HF_TOKEN }}
 
   required:
     name: Main / Windows

.github/workflows/product-integration-smoke.yml

@@ -16,7 +16,7 @@ on:
         required: true
         type: string
       platform:
-        description: "Typed operating-system selector: linux or macos."
+        description: "Typed operating-system selector: linux, macos, or windows."
         required: true
         type: string
       backend:
@@ -39,7 +39,7 @@ jobs:
   product_integration:
     name: Product integration (${{ inputs.platform }}/${{ inputs.backend }})
     if: ${{ inputs.backend != 'rocm' || vars.MESH_ROCM_INFERENCE_RUNNER_ENABLED == 'true' }}
-    runs-on: ${{ inputs.platform == 'macos' && 'macos-15' || (inputs.backend == 'cuda' || inputs.backend == 'vulkan') && fromJSON('["self-hosted","Linux","X64","amd64","gpu-nvidia","mesh-llm-amd64","mesh-llm"]') || inputs.backend == 'rocm' && fromJSON('["self-hosted","Linux","X64","amd64","gpu-amd","mesh-llm-amd64","mesh-llm"]') || 'ubuntu-24.04' }}
+    runs-on: ${{ inputs.platform == 'windows' && 'windows-2022' || inputs.platform == 'macos' && 'macos-15' || (inputs.backend == 'cuda' || inputs.backend == 'vulkan') && fromJSON('["self-hosted","Linux","X64","amd64","gpu-nvidia","mesh-llm-amd64","mesh-llm"]') || inputs.backend == 'rocm' && fromJSON('["self-hosted","Linux","X64","amd64","gpu-amd","mesh-llm-amd64","mesh-llm"]') || 'ubuntu-24.04' }}
     timeout-minutes: ${{ inputs.timeout_minutes }}
     container:
       image: ${{ inputs.platform == 'linux' && inputs.backend == 'cpu' && 'ghcr.io/mesh-llm/mesh-llm-cuda-runner@sha256:f499b79bc52dc7492d57397fdbec9f890c6f6bb1d8c1fcde9c1c97d45c0541a7' || '' }}
@@ -58,7 +58,7 @@ jobs:
         run: |
           set -euo pipefail
           case "${PLATFORM}/${BACKEND}" in
-            linux/cpu|linux/cuda|linux/vulkan|linux/rocm|macos/metal) ;;
+            linux/cpu|linux/cuda|linux/vulkan|linux/rocm|macos/metal|windows/cpu) ;;
             *) echo "unsupported product integration platform/backend: ${PLATFORM}/${BACKEND}" >&2; exit 2 ;;
           esac
 
@@ -121,13 +121,15 @@ jobs:
         with:
           artifact_name: ${{ inputs.artifact_name }}
           artifact_path: ${{ inputs.artifact_path }}
+          binary_name: ${{ inputs.platform == 'windows' && 'mesh-llm.exe' || 'mesh-llm' }}
           staged_binary_path: ${{ inputs.staged_binary_path }}
           model_cadence: ${{ (github.event_name == 'pull_request' || github.event_name == 'pull_request_target' || github.event.inputs.original_event_name == 'pull_request' || github.event.inputs.original_event_name == 'pull_request_target') && 'pull-request' || 'main' }}
           save_model_cache: ${{ github.ref == 'refs/heads/main' && github.event.inputs.original_event_name != 'pull_request' && github.event.inputs.original_event_name != 'pull_request_target' }}
 
       - name: Run typed product integration suite
         env:
           MESH_PRODUCT_INTEGRATION_PHASE_ROOT: ${{ runner.temp }}/product-integration-${{ inputs.platform }}-${{ inputs.backend }}
+          MESH_PRODUCT_INTEGRATION_DURABLE_ONLY: ${{ inputs.platform == 'windows' && '1' || '0' }}
         run: |
           scripts/ci-product-integration-smoke.sh \
             "${{ inputs.staged_binary_path }}" \
@@ -150,5 +152,7 @@ jobs:
             ${{ runner.temp }}/product-integration-${{ inputs.platform }}-${{ inputs.backend }}/phase-results.json
             ${{ runner.temp }}/product-integration-${{ inputs.platform }}-${{ inputs.backend }}/*/split-evidence.json
             ${{ runner.temp }}/product-integration-${{ inputs.platform }}-${{ inputs.backend }}/*/split-evidence-snapshots/*.json
+            ${{ runner.temp }}/product-integration-${{ inputs.platform }}-${{ inputs.backend }}/*/durable-l3-evidence.json
+            ${{ runner.temp }}/product-integration-${{ inputs.platform }}-${{ inputs.backend }}/*/durable-*/*.json
             ${{ runner.temp }}/product-integration-${{ inputs.platform }}-${{ inputs.backend }}/*/*.log
           if-no-files-found: error

.omo/specs/pr-ci-optimization.md

@@ -161,10 +161,11 @@ credentials may differ.
 - ci-{linux,macos,windows}-product-slice.yml: platform-local composition after
   matching host and runtime producers succeed.
 - ci-platform-checks-slice.yml: macOS portable/unit and Windows checks.
-- ci-linux-product-smoke-slice.yml and ci-macos-product-smoke-slice.yml:
-  platform-local inference, backend, two-node, Metal and model-download
-  consumers using only composed artifacts. One Linux KV caching smoke job runs
-  a fixed dense SmolLM2 leg followed by a recurrent Qwen3.5 leg; both must pass.
+- ci-{linux,macos,windows}-product-smoke-slice.yml: platform-local inference,
+  backend, two-node, Metal, Windows CPU and model-download consumers using only
+  composed artifacts. Product integration includes a digest-bound durable-L3
+  phase for dense and recurrent models across a full process restart; Windows
+  runs that phase alone on a real product executor.
 - ci-linux-sdk-slice.yml and ci-macos-sdk-slice.yml: platform-local
   Rust/Kotlin/Swift consumers. Swift and Kotlin SDK artifacts are independent
   producers that start from the plan and static ABI respectively, before