Try to load antirez/deepseek-v4-gguf (q2 87GB) but I get the error: architecture 'deepseek4' is not supported

[dan64]

I cannot load antirez/deepseek-v4-gguf (I can load it using llama.cpp), here the full log

[18:34:40] cmd/INFO  detecting CPU/PCIe bandwidth to pick the MoE backend (ft bench bw)…
[18:35:21] cmd/INFO  ft serve --model D:\Programs\FreeToken Desktop\models\ds4\DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --port 1919 --moe-backend auto --max-running-requests 4 --memory-ratio 0.85 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420
[18:35:21] stdout/INFO  serve started (pid=21916 model=D:\Programs\FreeToken Desktop\models\ds4\DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf port=1919)
[18:35:31] stdout/INFO  Traceback (most recent call last):
[18:35:31] stdout/INFO    File "<frozen runpy>", line 198, in _run_module_as_main
[18:35:31] stdout/INFO    File "<frozen runpy>", line 88, in _run_code
[18:35:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 138, in <module>
[18:35:31] stdout/INFO      raise SystemExit(main())
[18:35:31] stdout/INFO                       ^^^^^^
[18:35:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 134, in main
[18:35:31] stdout/INFO      return runner(args[1:])
[18:35:31] stdout/INFO             ^^^^^^^^^^^^^^^^
[18:35:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 31, in _run_serve
[18:35:31] stdout/INFO      launch_server(argv=argv, prog="ft serve")
[18:35:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 123, in launch_server
[18:35:31] stdout/INFO      server_args, run_shell = parse_args(
[18:35:31] stdout/INFO                               ^^^^^^^^^^^
[18:35:31] stdout/INFO    File "python/freetoken/server/args.py", line 664, in freetoken.server.args.parse_args
[18:35:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\utils\hf.py", line 149, in cached_load_hf_config
[18:35:31] stdout/INFO      return build_gguf_shim(gguf_src)
[18:35:31] stdout/INFO             ^^^^^^^^^^^^^^^^^^^^^^^^^
[18:35:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\models\gguf\config.py", line 65, in build_gguf_shim
[18:35:31] stdout/INFO      raise ValueError(
[18:35:31] stdout/INFO  ValueError: GGUF architecture 'deepseek4' is not supported (known: ['gemma4'])
[18:35:32] stdout/INFO  serve exited with code 1 (pid=21916)
[18:36:52] cmd/INFO  ft serve --model D:\Programs\FreeToken Desktop\models\UD-IQ3_XXS\DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00002-of-00004.gguf --port 1919 --moe-backend auto --max-running-requests 4 --memory-ratio 0.85 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420
[18:36:52] stdout/INFO  serve started (pid=20808 model=D:\Programs\FreeToken Desktop\models\UD-IQ3_XXS\DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00002-of-00004.gguf port=1919)
[18:36:57] stdout/INFO  Traceback (most recent call last):
[18:36:57] stdout/INFO    File "<frozen runpy>", line 198, in _run_module_as_main
[18:36:57] stdout/INFO    File "<frozen runpy>", line 88, in _run_code
[18:36:57] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 138, in <module>
[18:36:57] stdout/INFO      raise SystemExit(main())
[18:36:57] stdout/INFO                       ^^^^^^
[18:36:57] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 134, in main
[18:36:57] stdout/INFO      return runner(args[1:])
[18:36:57] stdout/INFO             ^^^^^^^^^^^^^^^^
[18:36:57] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 31, in _run_serve
[18:36:57] stdout/INFO      launch_server(argv=argv, prog="ft serve")
[18:36:57] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 123, in launch_server
[18:36:57] stdout/INFO      server_args, run_shell = parse_args(
[18:36:57] stdout/INFO                               ^^^^^^^^^^^
[18:36:57] stdout/INFO    File "python/freetoken/server/args.py", line 664, in freetoken.server.args.parse_args
[18:36:57] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\utils\hf.py", line 149, in cached_load_hf_config
[18:36:57] stdout/INFO      return build_gguf_shim(gguf_src)
[18:36:57] stdout/INFO             ^^^^^^^^^^^^^^^^^^^^^^^^^
[18:36:57] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\models\gguf\config.py", line 62, in build_gguf_shim
[18:36:57] stdout/INFO      arch = gguf_architecture(model_path)
[18:36:57] stdout/INFO             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[18:36:57] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\models\gguf\reader.py", line 139, in gguf_architecture
[18:36:57] stdout/INFO      raise ValueError(f"GGUF file {model_path} has no general.architecture")
[18:36:57] stdout/INFO  ValueError: GGUF file D:\Programs\FreeToken Desktop\models\UD-IQ3_XXS\DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00002-of-00004.gguf has no general.architecture
[18:36:57] stdout/INFO  serve exited with code 1 (pid=20808)
[18:39:22] cmd/INFO  ft serve --model D:\Programs\FreeToken Desktop\models\ds4\DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf --port 1919 --moe-backend auto --max-running-requests 4 --memory-ratio 0.85 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420
[18:39:22] stdout/INFO  serve started (pid=27304 model=D:\Programs\FreeToken Desktop\models\ds4\DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-0731.gguf port=1919)
[18:39:31] stdout/INFO  Traceback (most recent call last):
[18:39:31] stdout/INFO    File "<frozen runpy>", line 198, in _run_module_as_main
[18:39:31] stdout/INFO    File "<frozen runpy>", line 88, in _run_code
[18:39:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 138, in <module>
[18:39:31] stdout/INFO      raise SystemExit(main())
[18:39:31] stdout/INFO                       ^^^^^^
[18:39:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 134, in main
[18:39:31] stdout/INFO      return runner(args[1:])
[18:39:31] stdout/INFO             ^^^^^^^^^^^^^^^^
[18:39:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 31, in _run_serve
[18:39:31] stdout/INFO      launch_server(argv=argv, prog="ft serve")
[18:39:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 123, in launch_server
[18:39:31] stdout/INFO      server_args, run_shell = parse_args(
[18:39:31] stdout/INFO                               ^^^^^^^^^^^
[18:39:31] stdout/INFO    File "python/freetoken/server/args.py", line 664, in freetoken.server.args.parse_args
[18:39:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\utils\hf.py", line 149, in cached_load_hf_config
[18:39:31] stdout/INFO      return build_gguf_shim(gguf_src)
[18:39:31] stdout/INFO             ^^^^^^^^^^^^^^^^^^^^^^^^^
[18:39:31] stdout/INFO    File "C:\Users\Danil\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\models\gguf\config.py", line 65, in build_gguf_shim
[18:39:31] stdout/INFO      raise ValueError(
[18:39:31] stdout/INFO  ValueError: GGUF architecture 'deepseek4' is not supported (known: ['gemma4'])
[18:39:32] stdout/INFO  serve exited with code 1 (pid=27304)

[jason-fxz]

see roadmap #79 , gguf not supported yet.