Unofficial FreeToken fork: Flash-Next 125B at 38 tok/s on two RTX 3060s with 2-bit experts, gpt-oss-120b on one; a benchmark tunes your settings. Qwen3.8-Flash-Next, 12 GB, MTP, 250k context, image input, WSL2. Runs on Turing: RTX 2060, RTX 20 series, sm_75.
moe offloading inference-engine vram multimodal mixture-of-experts pipeline-parallelism wsl2 long-context local-llm gguf speculative-decoding qwen3 consumer-gpu gpt-oss gpt-oss-120b rtx-3060 freetoken nvidia-turing rtx-2060
-
Updated
Oct 25, 2026 - Python