Skip to content

DESIGN.md

Latest commit

 

History

History
246 lines (223 loc) · 15.1 KB

DESIGN.md

File metadata and controls

246 lines (223 loc) · 15.1 KB
Raw

mesh-console — design & agent onboarding

A macOS desktop app that makes peer-to-peer AI (mesh-llm) + an agentic chat (goose) usable by non-technical people. This file is the working design record: architecture, verified library facts, gotchas, and roadmap. Keep it updated as the code changes. Sibling project: ../mesh-app (the power-tool/reference Tauri app this project cites; its DESIGN.md holds deeper mesh/goose API notes).

1. Shape

┌────────────────────────── Mesh.app (Tauri v2) ──────────────────────────┐
│  WebView → http://127.0.0.1:4640  (NO Tauri IPC — plain HTTP only)      │
│                                                                         │
│  mesh-console backend (axum, src-tauri/src/)                            │
│  ├── /app/*     lifecycle: state, diagnose, host, join, invite,         │
│  │              shutdown, reset · /app/events = SSE (phase, downloads)  │
│  ├── /app/chat  embedded goose agent turn, streamed as SSE frames       │
│  ├── /api/* ┐   streaming reverse proxy to the embedded node            │
│  ├── /v1/*  ┘   (management :3131 / OpenAI :9337)                       │
│  └── /          React UI (ui/dist embedded via rust-embed)              │
│                                                                         │
│  embedded goose Agent (developer + computercontroller + fetch tools)    │
│  └── OpenAI provider → the node's /v1 on loopback                       │
│                                                                         │
│  embedded mesh-llm MeshNode (mesh-llm-sdk, host-runtime daemon)         │
│  └── iroh QUIC mesh ⇄ peers                                             │
└─────────────────────────────────────────────────────────────────────────┘

Why no Tauri IPC: tauri-driver has no macOS support, so the entire app surface lives behind localhost HTTP. Playwright drives the identical frontend + backend in a plain browser; Tauri is ~30 lines (main.rs) that opens a native window at the URL and shuts the node down on exit. This is the defining architectural difference from ../mesh-app (idiomatic Tauri invoke/events, untestable UI layer).

2. Layout

Path What
src-tauri/src/main.rs Tauri shell: start backend thread, open window
src-tauri/src/bin/mesh-consoled.rs headless daemon (same backend, no window) — dev loop + Playwright
src-tauri/src/server.rs axum router: /app/* endpoints + static UI
src-tauri/src/node.rs MeshNode lifecycle: host/join, downloads, invite
src-tauri/src/agent.rs goose Agent: session, provider, extensions, reply→SSE frames
src-tauri/src/diagnose.rs hardware scan → model fit ranking + recommendation
src-tauri/src/events.rs AppEvent enum + ConsoleSink (bridges mesh-llm's global OutputSink → broadcast → SSE)
src-tauri/src/state.rs AppState, Phase (idle → hosting/joining → running), ports
src-tauri/src/proxy.rs streaming reverse proxy to node ports
ui/src/screens/ wizard: Welcome → PowerSetup/JoinFlow → Progress → MeshLive
ui/src/components/ Chat, InvitePanel, MeshViz, ui primitives
justfile just run (build UI + open app), backend, ui-dev, test, check

~1.6k lines Rust, ~2.4k lines UI.

3. Dependencies (the load-bearing facts)

  • mesh-llm: git deps on Mesh-LLM/mesh-llm, UNPINNED (tracks main via Cargo.lock; currently v0.72.1 @ b4b33ef8). Crates: mesh-llm-sdk (serving), mesh-llm-host-runtime (default-features off → native runtime only, no embedded web console; feature dynamic-native-runtime), mesh-llm-node, mesh-llm-events, mesh-llm-system (hardware detection), mesh-llm-client (model catalog, auto_model_pack).
  • goose: aaif-goose/goose (the canonical goose repo, not a private fork) PINNED rev = "c82c431c" = its main HEAD as of 2026-07-03 (goose 1.41.0), rustls-tls, plus goose-mcp for bundled MCP servers. Bump the rev deliberately and re-run cargo test — the provider/session surface churns. Gotcha: goose's builtin extension registry starts EMPTY for embedders — agent.rs calls register_builtin_extensions(goose_mcp::BUILTIN_EXTENSIONS).
  • rmcp: Cargo.lock keeps rmcp AND rmcp-macros at 1.7.0. goose 1.41.0 still does not compile against rmcp 1.8 (InitializeResult/peer_info() signature change — re-verified 2026-07-03), and rmcp-macros must match rmcp exactly. Re-test before bumping either; don't let cargo update float them.
  • hf-hub: [patch.crates-io] → git branch Mesh-LLM/hf-hub#mesh-console/disable-xet-env (fork base + one commit honoring HF_HUB_DISABLE_XET; app sets it by default). REQUIRED for big models: stock hf-hub's xet path stalls on some networks — removing this broke gemma layer downloads (2026-07-02) and was restored same day, as a git dep so no sibling checkout is needed. Verified: 2.5GB Qwen3-4B hosted
    • answering chat, zero xet in logs.
  • npm: ui/.npmrc pins registry=https://registry.npmjs.org/ (public). The lockfile was rewritten from Block artifactory URLs — if installs ever fail on global.block-artifacts.com, the global ~/.npmrc is bleeding through; the per-project file must win.

4. How the pieces work

Navigation and mesh switching

  • The running node is independent from the visible page. Home and Chat share a top navigation bar; opening Home does not call shutdown, and the connection pill returns directly to the live chat.
  • The SDK hosts one embedded MeshNode at a time. Choosing another public or private mesh while connected therefore opens a confirmation. Confirmation performs one orderly shutdown followed by the requested launch; cancellation leaves the current mesh untouched. Local Goose chats survive mesh switches.

Node lifecycle (node.rs)

  • Host: MeshNode::builder().serve().model(...) with a NativeRuntimeInstallOptions { progress: callback } so runtime download emits AppEvent::DownloadProgress. Model download is explicit and BEFORE node start: download_model_ref_with_progress_details(model, true) — byte progress flows through mesh-llm's global OutputSink → ConsoleSink → SSE.
  • Join: share: bool decides .serve() vs .client(); token via .join_token(...). Invite token read back from node.invite_token().
  • Phases: idle → hosting/joining (with download events) → running.

Agent (agent.rs)

  • Multiple local chats that survive restarts. goose persists each conversation to SQLite under GOOSE_PATH_ROOT; desktop-owned hidden sessions are marked with project id mesh-console, and mesh-console-session stores the last active id. /app/sessions lists/creates chats, id-scoped activation and history routes switch between them, and /app/chat receives an explicit session id. A legacy pointer-only session is marked and migrated on first list. Goose remains the canonical transcript store; the WebView only keeps a convenience copy of the selected id.
  • Agent::new(), update_provider(OpenAI provider → node /v1, ModelConfig::new(model)). Extensions: developer + skills (goose-mcp builtins seeded via register_builtin_extensions).
  • A mutex serializes turns; each /app/chat POST drives one reply() stream, translating AgentEvents into SSE Frames (text deltas, tool activity).
  • Resume on launch: GET /app/history flattens the persisted transcript into UI messages (shape_history, unit-tested — same role/content rules as the live translator: assistant text/thinking + tool chips; tool-output user messages dropped). The Chat repaints from it on mount.
  • New chat + switching: the left chat rail creates a new Goose session and preserves earlier sessions for selection. Chats can be archived by right-click or the row archive action, moved into a collapsible History section, and restored later; Goose's archived_at remains the source of truth. Only one embedded Agent is loaded at a time; switching while a turn streams is rejected as busy. The legacy /app/new_chat endpoint remains as a compatibility route that creates a new session. Distinct from /app/reset (leave-mesh / error recovery).
  • Auto-compaction: goose summarizes older turns once the conversation fills a fraction of the model's context window. We fix that fraction at 0.4 (goose default 0.8) via GOOSE_AUTO_COMPACT_THRESHOLD in init_process_defaults — small mesh models have modest context and the one long-lived session accretes history. It's env-only: the threshold isn't part of the reply/SessionConfig API (goose reads it via get_param, env over config file; ≥1.0 would disable it).
  • Desktop chat always sends the virtual model auto; concrete model selection is intentionally absent while mesh routing policy is being developed.

Diagnose (diagnose.rs)

  • mesh_llm_system::hardware scan (chip, VRAM rating) + MODEL_CATALOG fit ranking (fit_code: model GB vs VRAM GB) → recommended model + full ranked catalog. Unit-tested against upstream thresholds. Not macOS-specific in this crate — mesh-llm-system has macOS + Linux paths.

5. Testing

  • just test — Rust unit tests (diagnose fit ranking etc.).
  • ui: npm run test:e2e — Playwright "mocked": drives the real UI against a mocked backend in a plain browser.
  • npm run test:e2e:real / scripts/run-real-e2e.sh — Playwright "real": the full stack, real node.
  • just check = fmt + lint + test + e2e.
  • This works precisely because the app boundary is HTTP, not Tauri IPC.

6. Gotchas / sharp edges

  • rmcp 1.7.0 lock pin (see §3) — the single most common accidental break.
  • goose builtin registry empty for embedders — without register_builtin_extensions, ExtensionConfig::Builtin names fail.
  • Quit must _exit, not fall through to AppKit terminate: (issue #8). The embedded ggml/Metal runtime aborts (ggml_metal_rsets_free → SIGABRT) inside its C++ static destructors when libc exit() runs them at process teardown. main.rs's RunEvent::Exit handler does a clean node shutdown then calls libc::_exit(0), which skips the destructor phase entirely. Don't remove that hard-exit.
  • Assistant markdown needs remark-gfm + .prose-mesh CSS (issue #7). react-markdown alone won't parse GFM tables (renders raw pipes), and the emitted tags have no default styling. Both live in Chat.tsx / styles.css.
  • MoA floor: the mesh's virtual model "mesh" (Mixture-of-Agents) 503s with <2 real models; "auto" works with 1+. (Validated in ../mesh-app/src-tauri/tests/model_selection.rs.) Relevant when porting the model-ladder heuristic.
  • mesh-llm is unpinned: a cargo update can move it; the runtime download is version-coupled (release artifacts + skippy ABI must match — mesh-app's DESIGN.md §3/§7 has the full story if downloads 404 or ABI-mismatch).
  • Installed app vs source runs: /Applications/Mesh.app, local bundles, and just run/just backend share the native runtime cache and can leave stale helpers. If model startup fails with a macOS dlopen Team ID / library validation error, verify the launched app has com.apple.security.cs.disable-library-validation and kill old mesh-console/mesh-consoled processes. Full runbook: docs/development.md.
  • ui/.npmrc vs global ~/.npmrc (Block artifactory) — see §3.

6.5 Theming & invite links (2026-07-07)

  • Three themes + system: dark (absolute black #000, the shipped default), light, and vinyl — the retro-1977 sunset palette from docs/index.html (browns/amber/paper, Righteous display font, faint diagonal stripes + film grain via html.vinyl body::before/::after). All tokens are CSS custom properties in ui/src/styles.css; the theme is a class on <html> (lib/theme.ts, pre-paint mirror in ui/index.html). --font-display powers the font-display utility for headline moments; vinyl swaps it to Righteous (Google Fonts, loaded in index.html).
  • No emojis / text glyphs in the UI — iconography is lucide-react plus the MeshMark brand SVG (components/MeshMark.tsx, optional pulse).
  • Chat: assistant turns render with an avatar-style MeshMark, an ActivityTrace panel (collapsible chain of thought with a live last-thought tail while streaming, plus per-tool rows with spinner/check/fail icons), a blinking stream-caret, and animate-message-in entrances. Markdown via react-markdown + remark-gfm + .prose-mesh (see §6).
  • Invite links: mesh:// is a registered URL scheme (tauri-plugin-deep-link, config in tauri.conf.json). Accepted shapes: mesh://join/<token>, mesh://join#<token>, mesh://join?token=<token> (extract_invite_token in main.rs, unit-tested). Tokens are parked in AppState.pending_invite and broadcast as an invite_link node event; the frontend drains GET /app/pending_invite (one-shot) on boot and listens for the SSE event, landing on the join flow prefilled. The shareable link is https://mesh-llm.github.io/desktop-app/join/#<token> (inviteLink() in InvitePanel.tsx) — docs/join/index.html is the static landing page (retro-styled, reads the token from location.hash only, auto-attempts the mesh:// bounce, offers copy + download fallback).

7. Roadmap (agreed with micn, 2026-07-02)

  1. Public npm (done, committed).
  2. Third startup option: join PUBLIC mesh (done: JoinRequest.public → auto_join_public_mesh(); Welcome card "Try the public mesh" one-click joins with share=true serving the tiny DEFAULT_MODEL — no model decision; Playwright-covered).
  3. Back navigation in the wizard — user can change their mind at any step.
  4. Downplay model choice — recommended by default; picker behind an "Advanced" disclosure.
  5. Fastest start — consider mesh-app's tiny-model default (unsloth/Qwen3-0.6B-GGUF:Q4_K_M, ~500MB) and/or join-public-as-client first (zero download, chat in seconds) while a local model downloads in background.
  6. Mesh visualisation — MeshViz: who's online / contributing (peer roster in status payload has labels, models, VRAM), plus a nudge to invite/share when solo.
  7. Port from mesh-app (partially done: smart model default now applies the ladder — public or ≥3 real models → "mesh"/MoA, else "auto"; picker offers Auto/Mixture/concrete ids). Still to port: invite-message paste parsing (extract_invite_token), the validated owner-allowlist gating (dormant there, tests/gating.rs).
  8. Open-sourcing: sources are clean (no secrets/internal refs); needs a LICENSE file + org decision (remote is squareup/mesh-console).
  9. Cross-platform: no cfg(target_os) in this repo; Linux is near-term feasible (mesh-llm ships Linux runtimes); Windows depends on mesh-llm runtime maturity. "Checking your Mac…" is just copy.