|
| 1 | +# Design — Mac bridge: cloud-agent access to the self-hosted `kakeya-mac-m4` |
| 2 | + |
| 3 | +- **Status**: M1 implemented (git-bus transport); M2/M3 designed |
| 4 | +- **Relates to**: ADR 0009 (multi-host plane), PR #105 (CapabilityService), |
| 5 | + PR #109 evidence gate (`inference_engine/bench/k3_report_gate.py`), |
| 6 | + [`docs/ops/mac-m4-runner-setup.md`](../ops/mac-m4-runner-setup.md) |
| 7 | +- **Implementation**: [`inference_engine/bridge/`](../../inference_engine/bridge/), |
| 8 | + [`scripts/mac_bridge/`](../../scripts/mac_bridge/), |
| 9 | + [`.github/workflows/mac-bridge.yaml`](../../.github/workflows/mac-bridge.yaml) |
| 10 | + |
| 11 | +## 1. Problem |
| 12 | + |
| 13 | +Kakeya development now happens substantially through cloud agents running |
| 14 | +on **Linux x86 VMs with no Metal**. Everything MLX-dependent — the MLX |
| 15 | +verifier, `mlx.distributed`, the K3 Mac harness, the PR #109 evidence-gate |
| 16 | +reruns — needs Apple Silicon. The project owns exactly one such machine: |
| 17 | +the Mac mini registered as the self-hosted runner |
| 18 | +`[self-hosted, macOS, ARM64, kakeya-mac-m4]`, sitting behind NAT with |
| 19 | +**outbound-only** connectivity (the Actions runner long-polls GitHub). |
| 20 | + |
| 21 | +Constraints that shape the design: |
| 22 | + |
| 23 | +- **C1 — No inbound path to the Mac.** No public IP, no port forwarding. |
| 24 | + Any transport must be initiated from the Mac side or relayed. |
| 25 | +- **C2 — Cloud agents are ephemeral and git-native.** They reliably have: |
| 26 | + a repo checkout, git push permission, and read-only `gh`. They do NOT |
| 27 | + reliably have: VPN keys, SSH keys to the Mac, or workflow-dispatch |
| 28 | + permission. |
| 29 | +- **C3 — The Mac executes whatever lands on it.** A bridge that forwards |
| 30 | + arbitrary shell from an internet-facing queue to a desk machine is a |
| 31 | + remote-shell backdoor. Command surface must be an allowlist. |
| 32 | +- **C4 — Evidence discipline.** Results coming back from the Mac must |
| 33 | + flow through the PR #109 evidence gate, not around it. |
| 34 | + |
| 35 | +## 2. Architecture: three transports, one capability model |
| 36 | + |
| 37 | +``` |
| 38 | +M1 (this PR) M2 (queued) M3 (queued) |
| 39 | +┌─────────────────┐ ┌──────────────────┐ ┌──────────────────────┐ |
| 40 | +│ git-bus │ │ tailnet SSH │ │ Kakeya fleet member │ |
| 41 | +│ │ │ │ │ │ |
| 42 | +│ agent ──push──► │ │ agent ──SSH──► │ │ agent ──gRPC──► │ |
| 43 | +│ mac-bridge/* │ │ Mac (tailscaled)│ │ CapabilityService │ |
| 44 | +│ branch+manifest│ │ interactive REPL│ │ ProposerService │ |
| 45 | +│ Mac runner: │ │ lldb / py-spy / │ │ (ADR 0009 plane, │ |
| 46 | +│ run preset, │ │ mlx debugging │ │ PR #105, over the │ |
| 47 | +│ commit results │ │ │ │ M2 tailnet) │ |
| 48 | +│ back to branch │ │ │ │ │ |
| 49 | +└─────────────────┘ └──────────────────┘ └──────────────────────┘ |
| 50 | + async, batch, interactive, programmatic, |
| 51 | + zero new secrets needs TS authkey inference-native |
| 52 | +``` |
| 53 | + |
| 54 | +### 2.1 M1 — git-bus (implemented) |
| 55 | + |
| 56 | +The only transport that satisfies C1+C2 with **zero new infrastructure**: |
| 57 | +git is the RPC bus, the Actions runner is the executor, the branch is the |
| 58 | +session. |
| 59 | + |
| 60 | +Protocol: |
| 61 | + |
| 62 | +1. **Request.** The agent runs `scripts/mac_bridge/request_run.py |
| 63 | + --preset <name> [--param k=v ...] [--ref <workload-ref>]`. The client: |
| 64 | + - branches `mac-bridge/<preset>-<nonce>` from the workload ref, |
| 65 | + - overlays the bridge files if the ref predates them (workflow + |
| 66 | + executor must exist on the pushed branch — `on: push` workflows |
| 67 | + execute the pushed commit's definition), |
| 68 | + - writes `.mac-bridge/request.json` (the manifest), commits, pushes. |
| 69 | +2. **Execute.** `.github/workflows/mac-bridge.yaml` triggers on |
| 70 | + `push: branches: ['mac-bridge/**']`, runs on `kakeya-mac-m4`, |
| 71 | + serialized via a `mac-bridge` concurrency group (one Mac). It calls |
| 72 | + `scripts/mac_bridge/run_preset.py --manifest .mac-bridge/request.json`, |
| 73 | + which validates the manifest against the **preset allowlist** |
| 74 | + (`inference_engine/bridge/manifest.py`) and executes the preset's |
| 75 | + fixed argv list — no shell interpolation of any user-controlled |
| 76 | + string (C3). |
| 77 | +3. **Respond.** The runner commits `.mac-bridge/logs/` + any new |
| 78 | + `results/research/*.json` back to the same branch and pushes; it also |
| 79 | + uploads them as workflow artifacts. K3 acceptance reports are passed |
| 80 | + through `scripts/validate_k3_reports.py` **on the Mac** so a |
| 81 | + non-conforming report fails the bridge run itself (C4). |
| 82 | +4. **Fetch.** The agent polls with read-only `gh run list/view` (or plain |
| 83 | + `git fetch` until the result commit appears) via |
| 84 | + `scripts/mac_bridge/fetch_results.py`. |
| 85 | + |
| 86 | +Latency profile: ~10 s dispatch + queue + workload runtime. Right for |
| 87 | +test/eval/bench cycles (minutes-scale), wrong for interactive debugging — |
| 88 | +that is M2's job, not a reason to widen M1's command surface. |
| 89 | + |
| 90 | +### 2.2 Preset allowlist (M1 command surface) |
| 91 | + |
| 92 | +| Preset | What runs on the Mac | Typical use | |
| 93 | +| --- | --- | --- | |
| 94 | +| `mlx-env-probe` | `backends.mlx.env.probe_environment()` + `distributed.mlx_ring.probe_ring_environment()` (when present on the ref) | "is Metal/mlx healthy, which versions" | |
| 95 | +| `mlx-backend-tests` | `pytest tests/backends/mlx/ -q` | real-mlx truth for the fake-mlx Linux suites | |
| 96 | +| `integration-tests` | `pytest -m integration tests/integration/ -q` | the v0.3 GA gate on demand | |
| 97 | +| `k3-step1-incremental` | hardened Mac harness `--incremental` (n/gen/ctx bounded params) | PR #109 Step-1 decode-only evidence | |
| 98 | +| `k3-step2-fused` | hardened Mac harness `--fused-specdecode` | PR #109 Step-2 `blocks>0` evidence | |
| 99 | +| `k3-native-baseline` | hardened Mac harness `--native-baseline-bypass` | labelled oracle baseline | |
| 100 | +| `k3-evidence-gate` | `scripts/validate_k3_reports.py results/research` | re-validate committed reports on-device | |
| 101 | +| `pytest-path` | `pytest <path> -q` with the path validated against a repo-relative allowlisted-prefix rule (`tests/`) | targeted debugging of one test file | |
| 102 | + |
| 103 | +Parameters are **typed and bounded** (`n_samples ≤ 50`, |
| 104 | +`max_new_tokens ≤ 512`, `block_size ≤ 16`, paths must resolve under |
| 105 | +`tests/`); anything else is rejected at manifest validation, before any |
| 106 | +process starts. Machine-local facts (verifier/model paths) come from the |
| 107 | +runner's environment (`KAKEYA_MAC_VERIFIER_PATH`, …), never from the |
| 108 | +manifest. |
| 109 | + |
| 110 | +### 2.3 M2 — tailnet SSH (designed, needs one secret + one install) |
| 111 | + |
| 112 | +For interactive MLX debugging (lldb, py-spy, Metal captures, REPL): |
| 113 | + |
| 114 | +- Mac: `brew install tailscale`, join the tailnet with `--ssh` |
| 115 | + (Tailscale SSH; respects tailnet ACLs), tag `tag:kakeya-mac`. |
| 116 | +- Cloud agent: `TAILSCALE_AUTHKEY` (ephemeral, pre-authorized, |
| 117 | + tag-scoped key) added in Cursor Dashboard → Cloud Agents → Secrets; |
| 118 | + `scripts/mac_bridge/connect_tailscale.sh` brings up `tailscaled` in |
| 119 | + userspace-networking mode and opens `ssh kakeya@kakeya-mac-m4`. |
| 120 | +- ACL: the agent-side tag may reach `tag:kakeya-mac:22` only; the Mac |
| 121 | + initiates nothing toward agents. Ephemeral nodes garbage-collect when |
| 122 | + the agent VM dies. |
| 123 | + |
| 124 | +This is the same outbound-only trust shape as the Actions runner (C1), |
| 125 | +with per-session ephemeral identity. It is deliberately **not** part of |
| 126 | +M1: it requires a secret a fresh clone does not have. |
| 127 | + |
| 128 | +### 2.4 M3 — fleet membership (evaluation in §4) |
| 129 | + |
| 130 | +With the tailnet up, the Mac's Kakeya runtime serves the ADR 0009 gRPC |
| 131 | +plane (`CapabilityService` + `ProposerService`, PR #105) and the cloud |
| 132 | +agent joins as a fleet peer — capability gossip, placement, and remote |
| 133 | +block proposal over the same wire contract used between Mac minis on a |
| 134 | +desk LAN. |
| 135 | + |
| 136 | +## 3. Security model (M1) |
| 137 | + |
| 138 | +- **Command surface**: presets only; fixed argv; no manifest string ever |
| 139 | + reaches a shell. `pytest-path` constrains to repo-relative `tests/`. |
| 140 | +- **Trigger surface**: anyone with push permission to `mac-bridge/**` — |
| 141 | + identical to the existing surface (any PR labelled `needs-mac-m4` |
| 142 | + already executes its code on the runner via `integration.yaml`). The |
| 143 | + bridge does not widen who can run code on the Mac; it widens *what can |
| 144 | + be conveniently requested* while **narrowing** it to an allowlist. |
| 145 | +- **Result integrity**: results are commits on the request branch — |
| 146 | + reviewable, attributable, and evidence-gated before merge anywhere. |
| 147 | +- **Resource protection**: `concurrency: mac-bridge` serializes the |
| 148 | + single Mac; per-preset `timeout-minutes`; runs are cancellable from |
| 149 | + the Actions UI. |
| 150 | + |
| 151 | +## 4. Evaluation — folding the bridge into Kakeya distributed inference |
| 152 | + |
| 153 | +The question: should "cloud agent ⇄ kakeya-mac-m4" become a first-class |
| 154 | +part of the engine's distributed-inference feature (ADR 0009 / PR #105), |
| 155 | +rather than repo tooling? |
| 156 | + |
| 157 | +### 4.1 What maps cleanly |
| 158 | + |
| 159 | +| Bridge concept | ADR 0009 plane concept | |
| 160 | +| --- | --- | |
| 161 | +| Mac runner with presets | `NodeCapability` with `CAPABILITY_ROLE_TOOL` entries (the enum slot already exists in `distributed.proto`) — e.g. `tool:mlx-eval`, `tool:integration-tests` | |
| 162 | +| preset manifest | `ProposeBlock`-style typed request messages (one RPC per tool capability) | |
| 163 | +| git-bus branch session | durable async job with attributable artifacts — the property worth **keeping** even after gRPC exists | |
| 164 | +| evidence gate on results | the same gate, already shared library code | |
| 165 | + |
| 166 | +The capability model was designed for exactly this shape: the Mac |
| 167 | +advertises what it can do; placement picks it; the work request is typed |
| 168 | +and the accept/reject of its *output* happens on the consumer side. A |
| 169 | +`remote-executor` tool role is a natural, small extension of PR #105 — |
| 170 | +the registry, gossip, TTL, and placement code need **zero changes**; |
| 171 | +only a new `ModelCapability(role=TOOL)` convention plus one service. |
| 172 | + |
| 173 | +### 4.2 What does not map: WAN data-plane spec decode |
| 174 | + |
| 175 | +The latency budget kills token-level speculative decoding across the |
| 176 | +cloud↔desk boundary, and the integration should say so explicitly: |
| 177 | + |
| 178 | +- LAN (two Mac minis, ADR 0009's target): `ProposeBlock` RTT ~0.3–1 ms |
| 179 | + against block compute of tens of ms → negligible overhead. ✔ |
| 180 | +- WAN (cloud agent ⇄ home/office Mac through a relay): RTT 30–150 ms, |
| 181 | + *per block*. A Gemma-4-26B 4-bit verifier on M4 verifies an 8-token |
| 182 | + block in roughly 50–100 ms — the network would add 30–300 % overhead |
| 183 | + per block, and any acceptance-rate gain is consumed by transport. |
| 184 | + Drafts are latency-critical; **proposer and verifier must share a |
| 185 | + LAN** (or a Thunderbolt ring, per ADR 0009 §2). ✘ |
| 186 | +- WAN-tolerant flows: capability gossip (seconds-scale TTLs), placement, |
| 187 | + eval/test/bench jobs, artifact return — all fine. ✔ |
| 188 | + |
| 189 | +So the correct integration boundary is: **WAN = control plane + tool |
| 190 | +plane; LAN = data plane.** This is the same hybrid conclusion as ADR |
| 191 | +0009, extended one tier outward. |
| 192 | + |
| 193 | +### 4.3 Recommendation |
| 194 | + |
| 195 | +1. **Keep M1 in-repo now** (this PR): it unblocks PR #109's required Mac |
| 196 | + reruns and all future agent-driven MLX work, with no new secrets. |
| 197 | +2. **M2 next**: one Tailscale authkey secret + one Mac install; gives |
| 198 | + interactive debugging and the channel M3 needs. Low effort, high |
| 199 | + leverage. |
| 200 | +3. **M3 as a v0.5 roadmap item, scoped**: add a `remote-executor` TOOL |
| 201 | + capability + a small `ToolService` to the ADR 0009 plane so fleet |
| 202 | + nodes (including the Mac) advertise *evaluation* capabilities the |
| 203 | + same way they advertise verifier/proposer roles. Explicitly do |
| 204 | + **not** route spec-decode draft traffic over WAN; placement should |
| 205 | + treat `ring_address`/RTT class as a hard constraint for data-plane |
| 206 | + pairings (a one-line addition to `plan_spec_decode_placement`'s |
| 207 | + candidate filter when WAN nodes appear). |
| 208 | +4. mTLS + node identity (already queued for v0.5 GA) becomes a |
| 209 | + prerequisite for M3 leaving the tailnet's closed world. |
| 210 | + |
| 211 | +## 5. Operating the bridge (cheat sheet) |
| 212 | + |
| 213 | +```bash |
| 214 | +# From a cloud agent (or any clone with push rights): |
| 215 | +python3 scripts/mac_bridge/request_run.py --preset mlx-env-probe |
| 216 | +python3 scripts/mac_bridge/request_run.py --preset k3-step2-fused \ |
| 217 | + --ref AgentMemory/v04-mlx-port-incremental-decode-2815 \ |
| 218 | + --param n_samples=5 --param max_new_tokens=64 --param block_size=4 |
| 219 | +python3 scripts/mac_bridge/request_run.py --preset pytest-path \ |
| 220 | + --param path=tests/backends/mlx/test_fused_specdecode.py |
| 221 | + |
| 222 | +# Poll + fetch results (read-only gh + git): |
| 223 | +python3 scripts/mac_bridge/fetch_results.py --branch mac-bridge/<name> |
| 224 | +``` |
| 225 | + |
| 226 | +On the Mac runner, machine-local configuration lives in the runner env |
| 227 | +(see `docs/ops/mac-m4-runner-setup.md` §bridge): `KAKEYA_MAC_VERIFIER_PATH`, |
| 228 | +`KAKEYA_MAC_DRAFTER_ID`, `KAKEYA_MAC_FTHETA_DIR`. |
0 commit comments