Run the best local models for your machine
Magnitude is an open source local inference engine. It runs on the hardware you already have, whether that's a Mac, an NVIDIA or AMD GPU, or just a CPU. It profiles your machine, recommends the right models for it, then downloads, tunes, and runs them. Plug it into Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline.
⭐ Help us reach more developers and grow the Magnitude community. Star this repo!
See what your machine can run:
npm i -g @magnitudedev/cli
magnitude setupSetup profiles your hardware, ranks models by speed, accuracy, intelligence, and memory, and connects your harness to the one you pick.
Or let your agent handle it. Send this to Pi, Claude Code, OpenCode, or whatever you use:
Set up local models for me with the Magnitude CLI. Install it with `npm i -g @magnitudedev/cli` (or my package manager), then run `magnitude docs onboarding` and follow the instructions.
Your agent will profile your hardware, walk you through the best local models for it, download the ones you pick, and switch itself over to them.
- Knows your machine: profiles your hardware to assess fit and estimate tok/s
- Recommends the best models: ranked by speed, accuracy, intelligence, and memory
- Tuned end to end: speculative decoding and more, all set for your hardware
- Easy setup: one command and your agent is running local models
- Free to run: no token costs, API keys, or rate limits
- Fully private and offline: models, prompts, and files stay on your machine
- Models on demand: loaded on request, unloaded when idle or memory fills
- Open source: Apache 2.0, yours to modify
An open source inference server for the hardware you already have. It profiles your machine, recommends the right models for it, then downloads, tunes, and runs them. Plug it into the agent you already use.
Magnitude profiles your chip, memory, and bandwidth, then estimates fit and tok/s for every model in the catalog. It ranks them by speed, accuracy, intelligence, and memory so you can pick.
There's no fixed minimum. Magnitude profiles your machine and recommends what runs well on it. More memory lets you run larger models.
Magnitude runs on macOS, Linux, and Windows through WSL. It supports Apple Silicon, NVIDIA GPUs, AMD GPUs, and CPU-only setups. It profiles your hardware and recommends models that fit.
Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. During setup, your agent connects your harness to the model you pick.
No. It runs in the background, loads models when your agent needs them, and unloads them when idle or memory gets tight. Your agent can install or switch models through the CLI anytime.
Yes. Prompts, files, and models stay on your machine. Once a model is downloaded, no internet connection is needed.
Magnitude is licensed under the Apache License 2.0.