Cupcake Render Gateway integration: shared queue with Palette, gateway-mode gpu slot (supersedes #86 direct mode)

[taskmasterpeace]

Work order: integrate the Cupcake Render Gateway (shared queue) into Directors Desktop

Supersedes the direct-ComfyUI mode of #86 (that plumbing knowledge stays valid for benchmarks/emergencies). Robert's requirement: DD and Directors Palette both generate video on cupcake through ONE queue both apps can see, fully integrated and tested.

DD-specific scope: the 'Directors Desktop' section + test battery below. The gateway itself is built on cupcake (repo of record there: ~/machine-king-ai-os/docs/render-gateway-spec.md).

---

Cupcake Render Gateway — canonical spec (v1, 2026-08-07)

Goal: Directors Palette (web) AND Directors Desktop (Electron) generate video on cupcake,
through ONE shared queue that both apps can see, fully integrated in each app's native UI.

This document is the source of truth. Per-app work orders: Directors-Desktop issue (gateway
integration; supersedes the direct-mode parts of #86) and directors-palette-v2 issue.

Shape (follows the proven dramatis-hub pattern: zero-dep node, filesystem state, key tenancy)

Palette cloud ──Cloudflare Tunnel + Access token──▶ ┌──────────────────────────┐
Palette browser (LAN) ──────────────────────────▶   │ mk-gateway :8700 (cupcake)│
Directors Desktop ─────────LAN/tailnet──────────▶   │  keys.json tenancy        │──▶ ComfyUI :8188
AI agents / scripts ────────────────────────────▶   │  jobs/{queued,running,done}│    (LTX fp8 · H3 int8)
                                                    └──────────────────────────┘

• One global FIFO-with-priorities queue; GPU work serialized (one render at a time).
• State = filesystem (/srv/ai/gateway/jobs/{queued,running,done}/<job_id>.json + artifacts
  pointer into /srv/ai/outputs/comfy/). No database. Survives restart by re-scanning dirs.
• Auth = Authorization: Bearer <key> → sha256 → tenant in keys.json
  (tenants: dd, palette, local; localhost-no-key = local, the hub law).
• Worker drives ComfyUI with the PROVEN graphs — port them from DD's
  ltx_comfy_client_impl._build_graph / h3_local_client_impl._build_graph verbatim
  (they embed every paid-for fix). Reference images arrive via upload, staged into ComfyUI input.

API

Route	Purpose
GET /	agents.md — the gateway teaches its callers (hub law)
GET /v1/health	engines, warmth (hot/cold per engine), queue depth, disk level (reads /srv/ai/logs/disk-status.json)
POST /v1/video	submit → {job_id, position, eta_s}
POST /v1/uploads	reference image → {upload_id} (used in submit)
GET /v1/jobs	the shared queue view — tenant sees own jobs in full + others as {position, engine, eta_s, tenant_label} (no prompts leaked across tenants)
GET /v1/jobs/:id	status: queued(position) / running(phase,pct) / done(result_url) / error(msg)
GET /v1/jobs/:id/result	the mp4 (stream)
DELETE /v1/jobs/:id	cancel (dequeue, or /interrupt if running; own-tenant only)

POST /v1/video body:

{
  "engine": "ltx" | "h3",            // callers may omit → routed by rule below
  "prompt": "...", "negative_prompt": "...",
  "width": 832, "height": 480,        // /32 (LTX) or proven H3 tiers
  "seconds": 10, "seed": 123,         // seed omitted → random (frozen-seed lesson)
  "reference_upload_id": "...",       // optional → LTX first-frame / H3 omni-ref
  "lora": {"name": "...", "strength": 1.0},   // LTX only, v1
  "priority": "interactive" | "batch",         // interactive jumps batch, never preempts running
  "idempotency_key": "..."           // resubmit-safe
}

ETAs: seed the table from measured benchmarks (LTX 480p: 52/93/153s for 5/10/15s; H3 480p:
~300/760s; 720p from the matrix battery), then learn: rolling per-(engine,tier,seconds-bucket)
average of actual walls, warm/cold buckets — DD's model-timing v2 pattern, server-side.
Engine routing default when engine omitted: reference image present → h3; else ltx.

Queue visibility (the requirement both apps share)

GET /v1/jobs returns for every queued/running job: position, engine, eta_s,
tenant_label ("Desktop", "Palette", "Robert"), and is_yours. Both apps poll (2–5s) or
consume GET /v1/events (SSE, nice-to-have v1.1). A Palette user SEES "2 jobs ahead (Desktop)";
a DD user SEES "1 ahead (Palette)". Prompts/thumbnails never cross tenants.

Per-app integration

Directors Desktop (owner: DD repo issue)

• Settings → General: Render node = Off / Gateway URL (default http://192.168.1.249:8700).
• When on: gpu-slot jobs (ltx-comfy, h3-local) submit to the gateway instead of local
  ComfyUI; job model/params map 1:1; result downloaded into outputs_dir → Gallery unchanged.
• Queue strip + Playground queue show gateway position/eta and foreign-job count chips.
• Warmth pill: "cupcake · N in queue". Cancel maps to DELETE. Local 4090 path remains the
  fallback when the field is Off (or gateway unreachable → offer local).
• Remote render node: route ltx-comfy + h3-local to cupcake (GX10) via remoteComfyUrl #86's direct-ComfyUI mode is superseded by gateway mode (keep the stub-driver knowledge for
  emergencies/benchmarks).

Directors Palette v2 (owner: palette repo issue)

• New provider in the video provider abstraction: cupcake (server-side route, mirrors the
  fal/replicate provider shape). Env: CUPCAKE_GATEWAY_URL (tunnel hostname) +
  CUPCAKE_GATEWAY_KEY (tenant palette) + Cloudflare Access service token headers.
• Points: deduct server-side BEFORE submit (existing points law); price cupcake renders at
  a nominal internal rate (e.g. 1–3 pts) or 0 for Robert's account — decision flagged to Robert.
• Shot Creator / Playground video calls gain "Render on cupcake" model option; job status UI
  shows queue position; result ingested like any provider output.
• Cloud→home path: cloudflared tunnel on cupcake exposing ONLY :8700 as
  render.machinekinglabs.com, Cloudflare Access service-token policy (Palette's server is
  the only allowed caller). No router ports. LAN browser sessions may hit the LAN URL directly.

Non-goals v1

Parallel rendering (memory allows it; queue-level change later) · image/LLM routes (video
only; the API is shaped so /v1/image can join) · public self-serve signup · billing.

Test battery ("fully integrated and tested" = this list green)

Gateway: unit (queue order, priorities, tenancy filter, idempotency, cancel-in-queue,
cancel-running, restart-rescan) + live (one LTX + one H3 render end-to-end, ref upload → h3
identity output exists).
DD: suites green (pytest/vitest/pyright/tsc) + live: submit from Playground with gateway on →
mp4 in Gallery; queue strip shows a foreign job injected via curl; cancel works; gateway
unreachable → clean fallback offer.
Palette: provider unit tests + live: points deducted → job visible with position → mp4 lands
in gallery; Access-token rejected without header (verify from outside the tunnel).
Cross-app: submit from BOTH apps simultaneously → both see each other's position; order honored.

Rollout order

1. Gateway v1 on cupcake (localhost + LAN) + local tenant → agents/scripts can use it day one.
2. DD integration (LAN — no tunnel needed).
3. Tunnel + Access + Palette integration.
4. Learned ETAs + SSE + (maybe) parallel small-jobs.

[taskmasterpeace]

Decision (Robert 2026-08-07): the cupcake tier is 480p only, 16:9 + 9:16 only, flat 1pt short / 2pt long in Palette. For DD there is no points system — DD renders on cupcake at full capability (480p/720p, all lengths); the 480p cap is a Palette-tier business rule, not a gateway limit. Gateway spec updated with the decision section.

[taskmasterpeace]

Benchmark matrix complete (2026-08-07) — ETA seed data for the gateway

Full grid measured on cupcake via DD's unmodified clients (warm, quality configs). Wall seconds:

Engine	480p 5s	480p 10s	480p 15s	720p 5s	720p 10s	720p 15s
LTX-2.3 fp8	52	93	153	140	253	460
H3 int8	296	760	1354	850	2306	4392

For the gateway ETA table (/v1/health + submit-time estimates), seed with:
LTX {480p:[52,93,153], 720p:[140,253,460]}, H3 {480p:[296,760,1354], 720p:[850,2306,4392]} for [5s,10s,15s] warm; then replace with rolling actuals per (engine,tier,length) bucket.

Routing implications for defaults:

• LTX is 6–9.5× faster than H3, and the gap widens with resolution + length (9.5× at 720p/15s: 7.7min vs 73min).
• LTX 720p is fine for batch; H3 720p is impractical (73min/15s) — the gateway should default H3 to 480p and warn/deny H3 720p batch.
• Both engines scale ~3× from 480p→720p (predictable extrapolation).

Full analysis + learnings: benchmarks/video/h3-vs-ltx-2026-08-07.md on cupcake (commit 2cb279d).

[taskmasterpeace]

Gateway v1 is LIVE on cupcake (rollout step 1 ✅)

mk-gateway is built, deployed as a systemd service (mk-gateway.service, active + reboot-proof), and serving on http://192.168.1.249:8700. Source + doc committed to the box repo (~/machine-king-ai-os/services/mk-gateway/, commit 5594be7).

Design decision: built in Python stdlib (not node) because the box has no node runtime, and — more importantly — it lets the worker reuse DD's own ltx_comfy_client / h3_local_client verbatim (vendored onto the box), so every paid-for graph fix comes along instead of being hand-ported. Still zero-dep, filesystem state, key tenancy — the dramatis-hub shape.

Verified live:

• GET /v1/health → {engines:[ltx,h3], warmth, queue_depth, disk_pct} ✅
• GET / → agents.md ✅
• POST /v1/video (tenant dd, Bearer key) → {job_id, position, eta_s:67, engine:ltx} — routing (ref→h3 else ltx), ETA seed table, position all working ✅
• Worker picked the job, wrote the per-engine profile marker (ComfyUI systemd service never killed — confirmed), drove the vendored LTX client, submitted the prompt to ComfyUI ✅
• Tenancy: keys.json sha256, localhost=local ✅

The proof render is currently queued behind a foreign H3 job someone submitted directly to ComfyUI :8188 (minimax_h3 int8, ~760s) — my LTX job (A golden chess king…) is correctly pending behind it. That's real queue contention, not a gateway bug, and it surfaces a known v1 limitation: the gateway assumes it's the sole ComfyUI submitter. A background watcher will confirm the mp4 lands when the H3 clears.

Remaining #87 scope:

• DD integration (Settings render_node off/gateway-URL; gpu-slot jobs → gateway; queue strip position/eta + foreign chips; warmth pill; cancel→DELETE; unreachable→local fallback) — starting now
• Tunnel + Cloudflare Access + Palette cupcake provider (points deduction)
• Learned ETAs (wired: rolling avg per bucket, seeds from benchmarks) + SSE

Tenant keys minted (dd, palette) — stored as sha256 in keys.json; plaintext handed to Robert for the DD/Palette config.

[taskmasterpeace]

✅ End-to-end LTX render through the gateway — CONFIRMED

The queued LTX job (behind the foreign H3) completed:

• result_path: /srv/ai/outputs/comfy/video/ltx_render_00010_.mp4
• Fetched via GET /v1/jobs/{id}/result (Bearer dd key) → 350,919 bytes of real mp4 ✅

The full path is proven: submit → tenant auth → queue → worker → per-engine profile marker → vendored DD LTX client → ComfyUI → done → mp4 streamed back through the gateway. That closes the test-battery item "one LTX render end-to-end."

(Note: wall_s was 508s because it sat behind the foreign H3 job; that polluted the first learned-ETA sample, so I reset learned.json — the benchmark seed table stays authoritative until clean samples accrue. In normal sole-submitter operation, wall ≈ render time.)

Rollout step 1 (gateway v1) is done and proven. Moving to step 2 (DD integration).

[taskmasterpeace]

Status 2026-08-07 night — gateway v2 live; DD integration BUILT (branch, unpushed); Palette side shipped

Gateway v2 deployed on cupcake (source now tracked in the box repo services/mk-gateway/): h3_variant: stock|turbo|sol (turbo = 6-step turbo-LoRA graph, sol = SolAttnPatch+H3FirstBlockCache from the proven workflow), first_frame_upload_id = TRUE image-to-video on both engines, tenant auth now required on GET /v1/jobs/:id and /result (v1 leaked cross-tenant), foreign queue rows redacted to {position, engine, eta_s, tenant_label, status}, seed omitted → random (returned in response), per-(engine,variant,resolution) ETA seeds + learning, crash-proof worker, upload ownership checks. Smoke: 22/22 + three real i2v renders (LTX 92s · turbo 129s · sol 206s, 5s/480p). Root cause fixed along the way: stub/ComfyUI/input was a plain dir (only output was symlinked) so every image-staged render 400'd at LoadImage — now a real symlink.

DD integration is BUILT on branch feat/cupcake-gateway-video (commit 10f2711, local/unpushed) per this issue's spec: Settings → General render-node toggle + URL + write-only key, gpu-slot jobs (ltx-comfy/h3-local) submit to the gateway (position/ETA in the job cards), foreign-job chips, cancel → DELETE, unreachable → clean failure + "Retry on this PC" (forceLocal), MCP queue_status parity. Suites: pytest 772 / pyright 0 / vitest 315 / both tsc clean; live submit+cancel verified against the running gateway (no GPU burned). First real render from the DD UI is the remaining QA step.

Cross-app requirement met on the Palette side too: three admin-only cupcake models shipped and live-verified (real points deducted, mp4s in the gallery) — see directors-palette-v2#219. The only deployed-v1 blocker is the Cloudflare Tunnel (bring-up script staged on the box).