Streamlit app for running Cohere North Micro Vision locally on Apple Silicon with MLX.
North Micro Vision is a 2.4B-parameter, Apache-2.0 vision-language model that reads images at their native resolution rather than squashing them to a fixed square. Its strengths are documents, charts, OCR and visual grounding — exactly the workload where running locally beats shipping scans to an API.
This app runs the 8-bit conversion only
(mlx-community/North-Micro-Vision-Instruct-8bit, ~3.2 GB), which leaves room for a second
model on a 16 GB Mac: Command R7B, which answers the questions on Ask documents.
Multi-turn document Q&A, OCR, charts and captioning. Attach images with the + button; each one stays bound to the turn it arrived on, so a later question can refer back to a page uploaded several turns earlier. Responses stream token by token.
The follow-up carries no image of its own. Prompt tokens go 2,166 → 2,213 across the two turns, so the page attached on turn one is still bound to that turn.
Single-shot detection with a bounding-box overlay, a table of parsed coordinates, and a PNG download. Turn on Deterministic for steadier boxes.
Grounding on assets/sample-invoice.png — 2,145 visual tokens at 1248×1760. Box #1
matches row 1 of the table, and the "where" column describes it in words.
Upload a PDF or page images, transcribe each page once with North Micro Vision, then ask questions across them. A PDF's pages are rendered at 200 dpi, or at a scan's own resolution, and read from how they look, not from any text layer. Command R7B answers from the transcripts and cites what it used. Pick a citation to see it boxed on the page.
The selected citation, $4,182.55, is boxed on the page as #1.
assets/sample-invoice.png is a synthetic invoice for exercising all three pages.
- Apple Silicon Mac with 16 GB of unified memory. With both models resident, peak usage is about 11 GB.
- Python 3.11–3.13 (the pinned venv uses 3.12).
- uv.
It runs on your own Mac only. MLX needs Apple's GPU, so hosted platforms and Docker cannot run it.
Measured quality and memory are in docs/performance.md.
git clone --branch <latest release tag> https://github.com/darylalim/cohere-document-intelligence
cd cohere-document-intelligence && uv run --no-project --python 3.12 scripts/doctor.py
uv sync --locked && uv run streamlit run streamlit_app.pyTake the tag from the releases page:
a release tag is a commit that passed CI, which main may not be. uv provides the Python
in .python-version, so no other install is needed. The second line checks this Mac can
run the app, and says what to fix if it cannot.
An earlier version used the bf16 weights; uv run hf cache rm model/mlx-community/North-Micro-Vision-Instruct-bf16 frees that space.
Start the app when you need it and press Ctrl-C to quit. Do not run it as a login service: that would keep about 8 GB of models resident on a 16 GB Mac all day.
The first launch pulls ~3.2 GB of weights into the Hugging Face cache, and Ask documents
offers to download Command R7B (4.5 GB) on the page. The app loads a fixed commit of each
checkpoint rather than whatever main points at, so to fetch them ahead of time, ask for
those same commits:
uv run hf download mlx-community/North-Micro-Vision-Instruct-8bit \
--revision 26e6b3aef507899fbf86057593d99fc12418e26f
uv run hf download mlx-community/c4ai-command-r7b-12-2024-4bit \
--revision 2ae601b3d780410caacd2148c6c96e620144f22dIf the internal disk is short, set HF_HOME to a folder on an external drive. Once the
weights are cached the app can run offline; see
Network traffic.
git fetch --tags && git checkout <new tag> && uv sync --lockedA release that moves a model pin says so in its notes, with the download size, because that
update pulls new weights. uv run hf cache rm <revision hash> removes the old ones, and
uv run hf cache ls --revisions lists what is cached.
Documents stay on this Mac. The only network traffic is downloading the weights, after which
the app can run offline. Transcripts are saved to a local database so a slow transcription
survives a restart. Turn that off, or clear them, on Ask documents. The app does not encrypt
that file, so turn on FileVault. The details are in docs/privacy.md.
Every box is described in words and numbered to match its table row, every image has a
caption, and streaming can be turned off in the sidebar. Contrast floors are tested on both
themes. Known gaps, and what was checked by hand, are in
docs/architecture.md.
- Privacy: what is stored, logged and sent
- Troubleshooting: what each on-screen error means
- Architecture: how the app works, and the model's limits
- Performance: measured quality, memory and speed
- Evaluation: how those measurements are made
- Contributing: development, tests and CI
- Releasing: the steps before a version bump
- Security: how to report a vulnerability, and the trust model
The app's own code is Apache-2.0 — see LICENSE. Apache-2.0 rather than MIT
for the patent grant (§3) and, more to the point here, for the explicit statement that no
trademark rights travel with it (§6). Cohere, North Micro Vision and Command are
Cohere's marks; this is an unaffiliated project that runs their published checkpoints.
No weights are vendored here: uv sync installs no checkpoint, and they arrive in the
Hugging Face cache when first needed.
| Part | Licence | Commercial use |
|---|---|---|
| App code (this repo) | Apache-2.0 | Yes |
| North Micro Vision 8-bit weights | Apache-2.0 | Yes |
| pypdfium2 and its bundled PDFium | Apache-2.0 / BSD-3-Clause | Yes |
| Command R7B 4-bit weights | CC-BY-NC-4.0 + C4AI licence agreement | No |
Ask documents uses R7B, so it is non-commercial; Chat and Grounding use only North Micro
Vision. The R7B download is not gated on the Hub, but its terms still apply. Each
checkpoint's model card governs its weights, not this repository's LICENSE.