Everything you need beyond the README's quickstart. The in-app Help panel is a short version of the first-launch path; this file is the full reference.
- First launch
- The operating model
- One central folder
- Accepted profile formats
- Supported runtime styles
- How detection works
- What is standard on a Mac
- Why JSON is the right next step
- Resource profile
- Controller API contract
- Remote gateways
- Benchmark artifacts
- Build from source
- Release pipeline
- Raycast and power users
- Troubleshooting
- Known limitations
A GitHub Releases DMG (or Scripts/install.sh) already embeds the local controller. Do not clone this repo or run Controller/install-modelmatrix-controller.sh unless you are developing from source.
- Drag the app to Applications (or
~/Applicationsfrominstall.sh) and open the menu bar extra. - First launch registers a LaunchAgent. If the board shows a connection error, enable ModelMatrix in System Settings → General → Login Items & Extensions, then Quit and reopen.
- Profiles live in
~/Library/Application Support/ModelMatrix/Controller/model-profiles. The board is empty until you add.env/.jsonfiles there (not in theexamples/subfolder). - Copy a template from
examples/, fill in paths that exist on this Mac, Refresh, then Activate. - Optional remote GPU host: Settings → Remote Gateways. Paste a
modelmatrix-gateway://code, or enteruser+hostand Install Agent on Host. Remote default profiles folder is~/model-profileson that host, not the Mac Application Support path.
Source checkouts that install only the controller (no app bundle) still use <controller-root>/model-profiles. See Controller/README.md.
ModelMatrix is the control surface; model processes live in the runtimes.
There are three layers:
- A native macOS UI layer
- menu bar app via
MenuBarExtra - desktop / Notification Center widget via
WidgetKit
- menu bar app via
- A controller API
- profile discovery
- lifecycle actions
- optional integrations
- Runtime adapters
llama.cppmlx_lm.serverollamavLLM- custom shell scripts
- anything else that can be launched and health-checked
The app only needs a controller URL. The controller discovers profiles, launches runtimes, and reports health.
This repo includes one generic reference implementation under Controller/. Use it directly, or replace it with your own backend that exposes the same HTTP contract.
Adapter layout:
- Distributed app:
~/Library/Application Support/ModelMatrix/Controller/model-profiles - Source controller checkout:
<adapter-root>/model-profiles - Remote agent:
~/model-profileson the host (override with--profiles-dir/ Settings)
That folder is the source of truth. Add one manifest per model. Keep examples/ as templates; the controller only loads .env / .json files in the folder itself.
The example adapter accepts both:
*.env*.json
Use .env for shell-style configuration. Use .json for structured, tool-agnostic config.
Strict example manifests live under Controller/model-profiles/examples/.
Every profile must resolve to a unique endpoint. If two profiles share the same HOST:PORT or BASE_URL, activation and status attribution become ambiguous. The controller doctor treats that as a profile error.
Best default for quantized GGUF models on Apple Silicon.
Typical fields:
RUNTIME=llama.cppMODEL_PATHorMODEL_FILEwithMODEL_ROOTSERVER_BINorLLAMA_SERVER_BINifllama-serveris not already onPATHPORTREQUEST_MODELSERVER_MODEL_ID
Best for MLX-native converted models on Apple Silicon.
Typical fields:
RUNTIME=mlxMODEL_DIRorMODEL_REPOSERVER_BINorMLX_SERVER_BINifmlx_lm.serveris not already onPATHPORTREQUEST_MODEL
Best when you want to launch an OpenAI-compatible MLX server from a dedicated binary.
Typical fields:
RUNTIME=rvllm-mlxSERVER_BINMODEL_DIRPORTREQUEST_MODELSERVER_MODEL_ID
Best for high-throughput MLX model directories on Apple Silicon.
Typical fields:
RUNTIME=vllm-mlxSERVER_BINorVLLM_MLX_BINifvllm-mlxis not already onPATHMODEL_DIRorMODEL_REPOPORTREQUEST_MODELSERVER_MODEL_ID- optional tuning fields:
MAX_TOKENS,MAX_REQUEST_TOKENS,GPU_MEMORY_UTILIZATION,CACHE_MEMORY_PERCENT,PREFILL_STEP_SIZE,ENABLE_TOOL_CALLS,TOOL_CALL_PARSER
Best when the model is owned by another runtime, desktop app, daemon, or wrapper.
ModelMatrix uses three launch modes:
adapter: known runtimes where the controller builds the command (llama.cpp,mlx,rvllm-mlx,vllm-mlx,ollama,vllm,sglang,tgi,llama-cpp-python).command: profile-ownedSTART_COMMAND, optionalSTOP_COMMAND, and readiness.external: an already-running OpenAI-compatible endpoint such as LM Studio, Jan, LocalAI, or a manually launched server.
Named command and generic-binary profiles can still use runtime ids such as ddtree-mlx, turboquant, mlx-vlm, mlx-omni-server, mistral.rs, mlc-llm, lightllm, fastchat, openllm, nexa, exllamav2, aphrodite, and lmdeploy. They retain their real runtime label instead of displaying as custom. Use LAUNCH_MODE=external when a named runtime is already running outside ModelMatrix. Every profile status includes runtime_label, runtime_tags, and launch_mode. Add custom tags with RUNTIME_TAGS="coding q8 long-context".
Profiles can be JSON or declarative .env files. .env files are parsed as key/value data, not shell scripts. Use quoted START_COMMAND or STOP_COMMAND values for commands that intentionally run through the launcher.
Generic JSON example:
{
"DISPLAY_NAME": "My Custom Server",
"RUNTIME": "command",
"START_COMMAND": "/absolute/path/to/start-my-server.sh",
"STOP_COMMAND": "curl -fsS -X POST http://127.0.0.1:8099/shutdown || true",
"BASE_URL": "http://127.0.0.1:8099/v1",
"HEALTHCHECK_MODE": "http-200",
"HEALTHCHECK_URL": "http://127.0.0.1:8099/health",
"REQUEST_MODEL": "my-local-model",
"SERVER_MODEL_ID": "my-local-model"
}Generic binary example for runtimes without a first-class adapter:
{
"DISPLAY_NAME": "Any Local Server",
"RUNTIME": "tabbyapi",
"SERVER_BIN": "/absolute/path/to/server",
"SERVER_ARGS_JSON": ["--host", "127.0.0.1", "--port", "5000", "--model", "/models/model"],
"BASE_URL": "http://127.0.0.1:5000/v1",
"REQUEST_MODEL": "local-model",
"SERVER_MODEL_ID": "local-model"
}See Controller/RUNTIME_SUPPORT.md for the full runtime matrix and examples.
Keep the schema stable. Path layout can vary by repo.
The reference macOS launcher is intentionally strict:
- it does not guess Homebrew paths, repo-local runtime builds, or personal directories
- binaries come from the profile manifest or
PATH - model locations come from the profile manifest
For each profile it does this:
- Read the managed PID file if present.
- If the PID is stale, clear it.
- If no managed PID exists, fall back to checking whether the configured port has a listener.
- Run the configured health check.
Supported health-check modes:
openai-models- default for
llama.cppandmlx - probes
/v1/models - verifies the expected model ID is present
- default for
http-200- generic HTTP readiness
- considers the profile ready when the configured URL returns success
disabled- only use this when you cannot probe readiness
- process state may still be visible, but endpoint health is not verified
The UI is launcher-agnostic as long as each profile defines startup and readiness checks.
For Apple Silicon local inference, common stacks are:
llama.cppwith MetalMLX/mlx_lm.serverrvllm-mlx
Other tools fit into one of these buckets:
Ollama- convenience packaging and model management around a local runtime
LM Studio- GUI-first local model UX built around local backends
Jan,Open WebUI,SoulForge,vLLM Studio- client UX, orchestration, or alternate serving layers
vLLM- strong on CUDA/Linux; a secondary option on Apple Silicon laptops
A high-performance macOS stack:
llama.cppfor GGUFMLXfor MLX-native models- OpenAI-compatible endpoints for tool interoperability
JSON is easier to operate than raw env files for shared setups.
Reasons:
- explicit structure
- easier schema validation later
- easier editor tooling
- easier runtime-agnostic fields such as
START_COMMAND,BASE_URL, andHEALTHCHECK_MODE - easier future migration to a full profile registry
The current adapter keeps .env support for backward compatibility.
ModelMatrix itself is intentionally light.
- native SwiftUI app
- native WidgetKit extension
- no Electron
- no embedded browser runtime
- no bundled inference engine
- no constant high-frequency polling
Design choices that keep the app light:
- the controller refresh loop uses an adaptive low-frequency cadence instead of aggressive constant polling
- the visible footer clock only updates while the menu is open
- the menu bar hover text is derived from the same lightweight controller snapshot the shell uses
- the widget refreshes on a simple timeline instead of running its own always-on helper
Most memory/thermal load stays in runtimes, not the operator UI.
The app expects a controller base URL, defaulting to http://127.0.0.1:7700.
| Method | Path | Purpose |
|---|---|---|
GET |
/api/status |
Profile + readiness snapshot |
GET |
/api/integrations |
Optional integration manifest |
GET |
/api/benchmark/status |
Current benchmark run state |
POST |
/api/start |
Start a profile |
POST |
/api/stop |
Stop a profile |
POST |
/api/restart |
Restart a profile |
POST |
/api/switch |
Stop others, start target (the Activate path) |
POST |
/api/stop-all |
Stop every managed profile |
POST |
/api/integrations/run |
Trigger an optional integration |
POST |
/api/benchmark/start |
Run benchmark(s) |
Any backend that returns the same profile-status JSON shape and supports these lifecycle actions is compatible. Exact contracts live in Sources/ModelMatrixCore/Models/ and Sources/ModelMatrixControllerCore/ControllerRouter.swift.
Each remote gateway is a Linux/Unix stdlib-only Python agent speaking the controller contract. Full commands and routes: RemoteAgent/README.md.
| Step/mode | Facts |
|---|---|
| Install/update | Install Agent on Host pushes bundled files over SSH and creates a systemd user service under ~/.local/share/modelmatrix-agent/; the host downloads nothing. Alternatives: documented curl | bash or repo checkout. Re-run the same method to update. |
| Profiles | Same .env / .json format; default ~/model-profiles/. link scans $HOME, confirms/pastes and persists a folder, then prints pairing. Overrides: --profiles-dir / MODELMATRIX_PROFILES_DIR; Mac --profiles-dir persists in config.json. Templates: vllm, llama.cpp (llama-server), sglang, tgi; otherwise START_COMMAND. |
| Pair | Paste the installer's or link's modelmatrix-gateway://… code into Settings → Remote Gateways (the empty-state field, or Add Remote Gateway). It locally prefills editable user/host/port fields; no pairing service is involved. Click Update on that gateway anytime to push a newer bundled agent and refresh models. |
| SSH (recommended) | Loopback-only agent; app maintains ssh -N -L via your config/keys/agent (BatchMode=yes; no passwords). Trust the host once in Terminal. ControlMaster forwards running models to the same local ports. Reconnect uses backoff; the panel reports missing keys, changed host keys, and unreachable hosts. SSH over Tailscale also works, including Tailscale SSH. |
| Tailscale direct | --tailscale or serve --tailscale --auth-token-file … binds only the WireGuard tailnet address, not open LAN. Token is required/generated by default; paste it into the Mac form. mode=direct + MagicDNS uses no tunnel. Only on a personal tailnet, --allow-unauthenticated opts out (any tailnet member can control models). Reachable model profiles need HOST=0.0.0.0 or the Tailscale IP because remote loopback is not forwarded. |
| Trusted plain LAN | Direct http://host:7700 requires --unsafe-bind and a ≥16-byte bearer token for non-loopback binds. Per-gateway token is stored in macOS Keychain. |
Benchmarks/integrations and the desktop widget remain local-only. Menu-bar count and Stop Everything span all gateways. Remote state is excluded from the local cache, so tunnel failure cannot appear as local model state.
The reference benchmark harness writes both machine-readable and human-readable outputs to:
Controller/benchmark-results/
Each completed run produces:
benchmark-YYYYMMDD-HHMMSS.jsonbenchmark-YYYYMMDD-HHMMSS.mdlatest.jsonlatest.md
The Benchmarks panel reads latest.json. Export CSV writes a portable report from the current latest run.
# Run tests
swift test
# Iterative dev (launches a debug build)
./Scripts/run-dev.sh
# Release build (produces dist/ModelMatrix.app)
./Scripts/build-app.sh
# DMG (produces dist/ModelMatrix-<version>.dmg)
./Scripts/build-dmg.sh
# Verify an installed copy
./Scripts/verify-installed-app.shSource installs use the hardened installer:
./Scripts/install.sh
./Scripts/install.sh --skip-open
./Scripts/install.sh --verifyUseful flags:
| Flag | Use |
|---|---|
--install-dir PATH |
Install the app outside ~/Applications. |
--bin-dir PATH |
Install modelmatrixctl outside ~/.local/bin. |
--easy-mode |
Add the CLI bin directory to writable shell rc files. |
--quiet / --no-gum |
Run in CI or plain ANSI mode. |
--no-cli / --no-completions |
Skip CLI or shell completion installation. |
Or via make:
make test # swift test
make app # build-app.sh
make dmg # build-dmg.sh
make install # install.sh
make uninstall # uninstall.shThe Xcode build path regenerates ModelMatrix.xcodeproj from project.yml via XcodeGen before building, so you never hand-edit the project file.
For GitHub distribution:
- a Developer ID-signed app
- a notarized
.dmg - a GitHub Release that points users to the DMG
This repo builds DMGs locally; public releases should be signed and notarized.
Included:
Scripts/release-preflight.shScripts/bump-versionScripts/sign-and-notarize-dmg.sh.github/workflows/release.yml
The release workflow signs, notarizes, verifies, and uploads the single unified-app DMG when either:
- a
v*tag is pushed - a commit on
mainchangesVERSION
Normal maintainer flow:
./Scripts/bump-version patch # or minor / major / x.y.z
git push origin mainGitHub Actions will create or update the matching v<version> release for that commit. Required repo secrets:
APPLE_CERTIFICATE_P12_BASE64APPLE_CERTIFICATE_PASSWORDAPPLE_DEVELOPER_IDENTITYAPPLE_NOTARY_API_KEY_P8_BASE64APPLE_NOTARY_API_KEY_IDAPPLE_NOTARY_API_ISSUER_ID
Recommended local preflight before push/tag:
./Scripts/release-preflight.shManual release checklist:
- install the built app
- toggle
Launch At Loginand confirm the app remains in macOS Login Items - run
./Scripts/verify-installed-app.sh
Raycast users have two paths:
- The app appears as a normal macOS application after install.
- Keyboard-first operators can also call scriptable actions without opening the menu.
This repo supports both:
Scripts/install.shexplicitly registers the app with Launch Services and forces a Spotlight import so Raycast can discover it faster.Scripts/modelmatrixctlis a tiny controller CLI. Agents can start withmodelmatrixctl capabilities,modelmatrixctl robot-docs guide,modelmatrixctl triage, ormodelmatrixctl doctor --json. Mutating commands support--dry-run/--planand structured--jsonenvelopes.Integrations/Raycast/Script Commands/has Script Commands for status, opening the profiles folder, stopping all models, and running quick benchmarks.
If Finder shows .app extensions, that is the macOS AppleShowAllExtensions Finder preference, not a bundle naming issue.
The app doesn't appear in Spotlight or Raycast.
Run ./Scripts/install.sh. It registers the bundle with Launch Services and forces a Spotlight import. If the old ModelMatrix.app name is still cached, the installer removes it automatically.
A profile shows "Not Running" even though I can curl the endpoint.
The default health check for llama.cpp and mlx profiles probes /v1/models and verifies the expected model ID is present. If your server returns a different id, set SERVER_MODEL_ID in the profile to match. For a looser check, switch the profile to HEALTHCHECK_MODE=http-200.
Activate doesn't kill the previously running model.
Each runtime is tracked by a managed PID file. If you started the process outside ModelMatrix (e.g. a terminal llama-server invocation), the controller doesn't own that PID. Stop the outside process manually, then use Activate.
The widget doesn't show up in the widget gallery. See Known limitations; the widget extension is bundled correctly but only registers reliably with a Developer-ID-signed build.
Benchmarks panel is empty or cooldown won't clear.
The Benchmarks panel reads Controller/benchmark-results/latest.json. Delete it to reset the panel, or run Benchmark All once to regenerate.
Controller port 7700 is already in use.
The default can be overridden when the controller starts. If you changed it, also update the controller URL in the app's Settings panel.
Installed-app verification fails in headless or restricted Apple Events environments.
Run MMX_VERIFY_UI=0 ./Scripts/verify-installed-app.sh to execute lifecycle/API checks without UI automation.
Desktop / Notification Center widget requires a Developer-ID-signed build.
The widget target (ModelMatrixWidget) is real. It is embedded at Contents/PlugIns/ModelMatrixWidget.appex and wired through project.yml. Local installs from ./Scripts/install.sh ad-hoc sign the bundle (codesign --sign -). Ad-hoc-signed widget extensions are not reliably registered by WidgetKit's gallery. The widget begins to register once the app is installed from a Developer ID-signed, notarized DMG (the GitHub Release build). To verify on a local build:
APP="${MMX_APP_PATH:-}"
if [ -z "$APP" ]; then
for candidate in \
"/Applications/ModelMatrix.app" \
"$HOME/Applications/ModelMatrix.app"
do
[ -d "$candidate" ] && APP="$candidate" && break
done
fi
pluginkit -a "$APP/Contents/PlugIns/ModelMatrixWidget.appex"
killall chronod cfprefsd 2>/dev/null || true
open -a "$APP"Then wait ~60 seconds and check the Widget gallery. Results vary across macOS versions.
Widget note. WidgetKit distribution follows the host app: users install and launch the containing app once before the widget appears in the gallery, once registration succeeds.
To keep this reusable:
- keep the app generic
- document the controller contract
- treat external tools like Droid as optional integrations, not required features
- ship one backend adapter as an example
- let other people plug in their own runtime stack