Any newer alternatives to qwen-2.5-coder?

[akavel]

I'm currently still using llama.vim with ggml-org/Qwen2.5-Coder-7B-Q8_0-GGUF. Since when it was published, quite many newer models were made available. I tried a bunch, but unfortunately they either didn't work at all with llama.vim, or were much slower and/or giving worse results.

Was anyone of you successful in finding a model and a set of llama-server options that would work better than qwen-2.5-coder-7b, at similar in-memory size?

[ggerganov]

I don't think there is anything better for local FIM than Qwen2.5 Coder and Qwen3 Coder. New models don't seem to train a FIM template. And the new Qwens are all recurrent based, which is not compatible with cache reuse.

[akavel]

Oh; thanks for the answer!

[Green-Sky]

Most new models can handle diff-type tools, so some special prompt can probably be crafted for those.