Run large language models on hardware that should not fit them.
Green Engine is a local LLM CLI and research runtime for consumer GPUs and CPUs. The ge command searches, pulls, benchmarks, and serves models offline. Scheduling, expert paging, KV experiments, and native .green inference are under active development.
| Capability | Status |
|---|---|
ge run / ge chat serve on .gguf |
Working — launches llama-cli, llama-server, or llama_cpp with ordinary GGUF models (static ggml offload; compatibility mode) |
| Phase 1 compressed GGUF | Working — greencompress export-gguf output runs with ge chat serve --model file.gguf |
| Model search / pull / bench / MCP stack | Working |
Native .green dense generate |
Working (experimental) — ge run model.green and ge chat serve --model model.green run through engine-core; good for engineering and smoke, not the production default |
Tiny MoE .green generate |
Working (experimental) — tiny-moe.green opens and generates through the native router + PackageExpertStore; larger MoE still needs quantized expert packs |
| Long-context / paged KV | Working (experimental) — bounded-RAM KV is live on native decode with full-window default, optional tighter hot cap, and K=Q8 keys for RAM savings |
Green-compressed inference in ge run |
Experimental — native .green path exists for dense and tiny-MoE engineering, while production chat still prefers exported GGUF via llama.cpp |
- One CLI (
ge) — Search, pull, run, bench, compress (via Green Compress), and serve local GGUF models offline. - MCP-friendly local stack — Embeddings and chat servers for Codehelper.
- Research path for oversized models — Native
.greenruntime, MoE paging, and KV policies now exist in the engineering path; production rollout is still gated by speed and large-model gaps.
Prebuilt binaries (Linux, macOS, Windows): GitHub Releases
From source:
git clone https://github.com/VeyrForge/GreenEngine.git && cd GreenEngine
# Prefer GPU when CUDA Toolkit / kernels DLL is present; otherwise CPU-only (feature stays optional):
python scripts/build_ge_release.py
# Equivalent explicit forms:
# cargo build --release -p ge --features gpu # when CUDA/kernels available
# cargo build --release -p ge # CPU-only machines
./target/release/ge helpRequires Rust stable. Pair with Green Compress via ge install when you need smaller weights.
Native .green --gpu-layers N needs a GPU-featured binary (--features gpu + green_engine_kernels DLL). A CPU-only ge prints a loud stderr warning and falls back to CPU — see docs/gpu-decode.md.
ge models search llama
ge pull bartowski/Llama-3.2-1B-Instruct-GGUF
ge run ~/.green/models/.../*.gguf --prompt "Explain KV cache in one paragraph"
ge chat serve --model ~/.green/models/.../*.ggufFor Green-compressed weights today, export to GGUF first:
ge install
greencompress export-gguf /path/to/compressed-workdir -o model.gguf
ge chat serve --model model.gguf| Phase | Deliverable | Status |
|---|---|---|
| Phase 1 | Compressed weights → GGUF fallback for llama.cpp | Available via greencompress export-gguf |
| Phase 2+ | Native .green packages via Green runtime |
Available (experimental) — dense generate, native serve, and tiny-MoE smoke are live |
| Phase 4 | Paged KV store wired to generation | Available (experimental) — bounded-RAM decode path is live; longer-context quality and larger limits still need validation |
Typical ge session (benchmarks reflect scheduling research, not live chat):
$ ge bench
portable_bench: hit rate 94.2% bytes/token 12.1 MB
$ ge ui serve
dashboard: http://127.0.0.1:8780
$ ge run model.gguf --prompt "Hello"
compatibility mode — static llama.cpp offload
61 tokens in 4.1s = 14.8 tok/s
Measured numbers and reproduction: docs/BENCHMARKS.md.
| Platform | Notes |
|---|---|
| Linux | Full support (x86_64, arm64) |
| macOS | arm64 + x86_64 release binaries |
| Windows | x64 release binaries |
Works with Codehelper for local MCP embed/chat (ge embed serve, ge chat serve).
Today: ge orchestrates llama.cpp (GGUF), Green Compress (weight compression), and optional Python servers (green_chat.py remains a fallback, not the primary inference path).
Experimental native path: engine-core now powers ge run model.green and ge chat serve --model model.green for dense packages, tiny-MoE smoke, and bounded-RAM KV decode. Production chat, demos, and MCP still prefer GGUF + llama.cpp until native speed is closer.
ge install # build Green Compress companion
ge stack setup # deps + local MCP profile
ge embed serve # embeddings (optional, for codehelper)
ge chat serve # OpenAI-compatible local chat (GGUF / llama.cpp)
ge compress <args...> # delegate to greencompressge orchestrates Green Engine and Green Compress without merging their codebases.
| Situation | Typical outcome (benchmarks) |
|---|---|
| MoE trace under memory pressure | Higher expert hit-rate vs plain LRU |
| Long context (KV policies) | More retained attention at same KV budget (simulation) |
| Compression + scheduling | Lower bytes/token when manifest reflects compressed experts |
Full index: docs/BENCHMARKS.md. Benchmark results do not imply the same speedups in live ge run / ge chat serve today.
- docs/BENCHMARKS.md — performance index
- CHANGELOG.md — version history
ge help— CLI reference
- Production chat still prefers GGUF + llama.cpp. Native
.greenworks for engineering and smoke, but warm token speed is still behind the production path. - Dense models that already fit in VRAM may see little benefit vs plain llama.cpp in compatibility mode.
- GGUF model quality and speed depend on your CPU/GPU and chosen quantization.
- Large MoE packs still need quantized expert greenpack support; tiny-MoE raw-f32 packs are the verified path today.
- Native CPU context is bounded in practice: default full-window decode is suitable for
4096,8192is the current hard cap, and longer windows should stay on the research path until re-measured.
Issues, benchmark results, and suggested improvements are welcome on VeyrForge/GreenEngine.
Pull requests improving Green Engine are welcome. You may also keep private/internal forks for your own deployment under the BUSL-1.1 Additional Use Grant. Do not offer Green Engine (or a substantially similar substitute) to third parties as a hosted or competing product. See License and permitted use and LICENSE-FAQ.md.
See CHANGELOG.md and GitHub Releases.
Green Engine is source-available under the Business Source License 1.1 (BUSL-1.1). It is not OSI open source until the Change License applies.
You may:
- Use and run Green Engine (including in production) for personal use or internal business purposes (including employees and contractors acting on your behalf)
- Copy, modify, and create derivative works for those same purposes — without having to contribute changes back
- Use Green Engine to develop, test, maintain, review, or operate software for yourself or your customers (without offering Green Engine itself as a product or service)
- Study the published source
You may not (until the Change License applies):
- Offer Green Engine or a modified version to third parties as a hosted, managed, embedded, or distributed product or service whose primary purpose is to provide functionality substantially similar to Green Engine as a substitute
- Sell a renamed fork or embed Green Engine as the main feature of another paid product without a commercial license
Change Date: 2029-07-24 — on that date, or the fourth anniversary of the first public BSL distribution of this version (whichever is earlier), this version becomes available under Apache License 2.0.
Tutorials and blog posts may include short illustrative snippets from the published source for explanation, provided they do not redistribute the software as a competing product or imply an OSI open-source grant before the Change Date.
For commercial redistribution, OEM licensing, or other usage not covered above, contact licensing@veyrforge.com.
This section is a plain-language summary. The binding terms are in LICENSE. Interpretive Q&A: LICENSE-FAQ.md.