Bonsai extraction demo [DO NOT MERGE]

[Shchoholiev]

What

A private, local PDF extraction demo for Ternary Bonsai 27B. Drop a PDF and the app reads page 1, streams schema fields, and highlights matched values on the uploaded page.

Run

brew install python cmake poppler
./scripts/run_bonsai_demo.sh

The first run builds the pinned Metal llama.cpp runtime and downloads about 6.7 GB of model weights. The demo currently supports Apple Silicon macOS.

Verified on M4 Pro · 48 GB

Clean first-run test from this branch:

• created the isolated environment, built llama.cpp, downloaded the model, and started both health-checked services
• processed the included one-page PDF successfully
• streamed 10 schema fields; extracted Rosa Nguyen, DOB, license class/number/state, run date, accidents 0, and violations None
• 584 prompt tokens · 77 completion tokens
• prefill 95.1 tok/s · decode 18.3 tok/s · 10.75 s total