fix(download): include json files when fetching weights, so NVFP4's index.json is no longer silently skipped

[ascorb12]

Fixes #124.

Root cause

download_hf_weight() fetches only *.safetensors from the Hub:

return snapshot_download(
    model_path,
    allow_patterns=["*.safetensors"],
    tqdm_class=DisabledTqdm,
)

model.safetensors.index.json doesn't match that glob, so it never downloads.
Most weight-loading paths reconstruct the shard map from each shard's own
safetensors header and never miss it. nvfp4_banks.load_nvfp4_expert_source_banks
does require the prebuilt index file, so NVFP4 checkpoints fail at load with:

FileNotFoundError: [Errno 2] No such file or directory:
'...\snapshots\<rev>\model.safetensors.index.json'

despite the repo (e.g. nvidia/Qwen3.6-35B-A3B-NVFP4) shipping the file —
download_hf_weight simply never asked for it.

Fix

Add *.json to the allow-list. Same call, one extra pattern.

Tested

Verified against huggingface_hub.utils.filter_repo_objects — the actual
function snapshot_download uses internally to decide what to fetch — not a
reimplementation of the filtering logic:

CURRENT pattern (bug reproduced):
    FETCHED  model-00001-of-00003.safetensors
    FETCHED  model-00002-of-00003.safetensors
    skipped  model.safetensors.index.json
    skipped  config.json
    skipped  tokenizer.json

FIXED pattern (add *.json):
    FETCHED  model-00001-of-00003.safetensors
    FETCHED  model-00002-of-00003.safetensors
    FETCHED  model.safetensors.index.json
    FETCHED  config.json
    FETCHED  tokenizer.json

Also confirmed operationally: manually placing the missing file from the repo
into an existing snapshot (workaround noted in #124) lets
nvidia/Qwen3.6-35B-A3B-NVFP4 load and serve normally — 262,144-token context,
42.2 tok/s decode, on an RTX 4090. This PR makes that automatic instead of a
manual step.

The added *.json pattern also brings in config.json / tokenizer.json /
etc. up front, which are small (tens of KB, hf_quant_config.json ~35 KB) —
no meaningful download-size impact, and likely redundant with whatever second
mechanism already fetches those today, but harmless either way.

Per the AI policy in CONTRIBUTING: this change was produced with AI assistance;
the submitter has reviewed and tested everything in it and takes responsibility
for it.

[App-311]

Related to #124 and PR #129 — possible follow-up issue with local .ftw mixed-precision checkpoints.
I've been testing RadixArk/Qwen3.8-27B-NVFP4 (converted to .ftw by the Desktop app) and wanted to share findings that might extend beyond #124.
What works
Desktop app successfully converts the model to .ftw (~20 GB, Mixed Precision: NVFP4 + FP8)
#124 workaround (manually adding model.safetensors.index.json) resolves the FileNotFoundError
What doesn't work for us
Even with the index file in place, the CLI still loads with dtype=torch.bfloat16 and OOMs at ~15 GB on a 16 GB VRAM card:
plain

Copy
CUDA out of memory. Tried to allocate 86.00 MiB.
GPU 0 has a total capacity of 16.00 GiB of which 0 bytes is free.
The ServerArgs consistently show dtype=torch.bfloat16, and --dtype only offers auto/float16/bfloat16/float32 — no NVFP4/FP8 option.
Our setup
Windows 11 native, RTX 4090 / 16 GB VRAM
freetoken 0.1.1+g30aa89115 (Desktop v0.2.0-beta.14 wheel)
Local folder: .ftw files + hf_quant_config.json + model.safetensors.index.json (fetched per #124 workaround)
Questions
Has anyone successfully served a local .ftw mixed-precision checkpoint via CLI, or is the .ftw path currently Desktop-only?
Should hf_quant_config.json + quantization_config in config.json trigger NVFP4/FP8 loading automatically, or is an explicit flag needed?
Could the Windows-specific path be a factor here?
Appreciate any pointers

[ascorb12]

The .ftw path works from the CLI, its not Desktop only. Loaded the Desktop app's own conversion straight through python -m freetoken --model-path:

model_path='C:/Users/Jing/.freetoken/models/Qwen3.6-35B-A3B-FP8'   (freetoken-*.ftw)
Allocating 8219 tokens for KV cache, K + V = 0.16 GiB
Free memory after initialization: 1.24 GiB

On the dtype I dont think thats your problem. dtype=torch.bfloat16 shows up in ServerArgs for NVFP4 models too, just checked it against nvidia/Qwen3.6-35B-A3B-NVFP4. Its the compute dtype, the expert weights stay quantized. That same model served 262,144 ctx in about 10 GB VRAM here, which wouldnt happen if it was really loading bf16 weights (35B at bf16 is ~70 GB).

The OOM reads like plain VRAM sizing to me, 20 GB of weights on a 16 GB card needs the experts pushed off. --moe-backend offload with --moe-cache-size small enough to leave headroom is what worked here.

One other thing, you listed RTX 4090 / 16 GB VRAM. 4090s are 24 GB so might be worth double checking which card its actually picking up, that would change the math a fair bit.