can I run Qwen 3.8 27b on RTX 2060 8GB + 48GB RAM?

[krisheeetu]

it would be amazing if I were able to do it with my local setup.

[vcruz305]

Short answer: not that model on that card, but your box is a good fit for a different one.

Qwen3.8-27B is dense (arch qwen35, no expert tensors), so there's nothing to offload. FreeToken's offload cache streams MoE experts from host RAM; dense weights have to sit in VRAM. At Q3_K_M that's ~13.3GB against your 8GB, and even IQ2_XXS lands around 7GB with nothing left for KV and activations. Your 48GB of system RAM can't help here because there's no dense-offload path.

Where that 48GB does pay off is MoE. On an A3B-class model the expert banks live in host RAM and only ~1.2GB of non-expert weights need VRAM. Ornith-1.5-35B-A3B IQ3_XXS needs 11.5GiB pinned in host RAM, which is comfortable at 48GB, and I run the IQ3_S version on an 8GB 4060 laptop at 45 tok/s.

One thing worth knowing before you try: the RTX 2060 is Turing (sm_75), which didn't work at all until a fix I landed today in #131. Two separate problems — the extend-attention kernel picked shared-memory tiles sized for Ampere and blew Turing's 64KB limit at warmup, and an untied lm_head skipped the prefill last-position slice. Both are fixed and verified on a Quadro RTX 6000 (also sm_75), running a 35B MoE at 64-69 tok/s. You'd want that branch rather than main.

Two caveats on the MoE path, so you know what you're getting into. The expert banks are held in pinned host RAM, which can't be swapped, so treat 11.5GiB as a hard floor rather than something that pages gracefully. And a lot of published quants are refused: llama.cpp's _M/_XXS recipes bump some layers' ffn_down_exps to a wider type, and the expert slot pool is one allocation with one stride shared by every layer, so a bank whose ggml type varies by layer can't be served. You get a clear error rather than garbage output. Of the ten published Ornith quants only IQ3_S and IQ3_XXS happen to come out uniform.