heiervang-technologies · marksverdhei · Aug 7, 2026 · Aug 7, 2026
diff --git a/docs/benchmarks/data/pascal-p5200-2026-08-02/README.md b/docs/benchmarks/data/pascal-p5200-2026-08-02/README.md
@@ -0,0 +1,142 @@
+# Pascal CUDA benchmark: Qwen3.6-35B-A3B and Gemma 4
+
+Tested on 2026-08-02 with the `ht` branch of ht-llama.cpp.
+
+## Runtime
+
+- ht-llama.cpp commit: `798cf6cbe56440132df23eb2318f587b16e3c00e`
+- build: `b9862-798cf6cbe`
+- GPU: NVIDIA Quadro P5200, 16 GiB, compute capability 6.1
+- CPU: Intel Core i7-7820HQ, 4 cores / 8 threads
+- CUDA build settings:
+  - `CMAKE_CUDA_ARCHITECTURES=61`
+  - `GGML_CUDA_FORCE_MMQ=ON`
+  - `GGML_CUDA_F16=OFF`
+- benchmark settings: 4 CPU threads, flash attention on, automatic device fitting,
+  1024 MiB VRAM safety target, 4096-token fit context, three measured repetitions
+
+## Verified models
+
+| Model | File size | SHA-256 |
+| --- | ---: | --- |
+| Qwen3.6-35B-A3B Q4_K_M | 20,419,565,568 bytes | `671e47e0ec53c665d048b98c3ecbfd5236b5ca9c3e02ed19fc8f81f7b85140c7` |
+| Gemma 4 26B-A4B IT QAT Q4_0 | 14,439,363,584 bytes | `3eca3b8f6d7baf218a7dd6bba5fb59a56ee25fe2d567b6f5f589b4f697eca51d` |
+
+Both files passed `sha256sum --check` before testing.
+
+## Results
+
+| Model | PP128 | PP512 | TG32 | TG128 | Warm TG128 |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Qwen3.6-35B-A3B Q4_K_M | 225.72 tok/s | 445.85 tok/s | 30.35 tok/s | 27.50 tok/s | 33.73 tok/s |
+| Gemma 4 26B-A4B Q4_0 | 503.47 tok/s | 861.37 tok/s | 52.23 tok/s | 51.52 tok/s | 51.80 tok/s |
+
+`PP` is prompt processing and `TG` is token generation. The normal result columns
+are arithmetic means of all three samples. Qwen's first TG128 sample was cold-page
+limited at 15.06 tok/s; its next two samples were 33.12 and 34.33 tok/s, whose mean
+is shown as Warm TG128. Gemma's TG128 samples were stable at 50.95, 51.84, and
+51.76 tok/s.
+
+## Device placement and telemetry
+
+| Model | Placement | Peak VRAM | Peak GPU | Peak power | Peak temperature |
+| --- | --- | ---: | ---: | ---: | ---: |
+| Qwen3.6 | 41/41 layers use CUDA; 13 layers partially overflow to host; 14,146 MiB CUDA model buffer | 15,036 MiB | 100% | 161.82 W | 64 C |
+| Gemma 4 | 31/31 layers fully offloaded; 13,755 MiB CUDA model buffer; 577.5 MiB CPU-mapped model data | 14,652 MiB | 100% | 155.68 W | 66 C |
+
+Across samples with more than 1 GiB VRAM allocated, average GPU utilization was
+30.9% for Qwen and 83.9% for Gemma. Qwen's host overflow and file paging account
+for the lower utilization and larger cold-run variance. Gemma fits almost entirely
+in VRAM and is correspondingly steadier.
+
+## Smoke tests
+
+Both models loaded, generated tokens, and exited with status 0 using the same
+Pascal CUDA build. Single-turn checks produced `QWEN_SMOKE_OK` and
+`GEMMA_SMOKE_OK`. Gemma's embedded chat template requires `--jinja`; without it,
+the legacy template path reports that the custom template is unsupported.
+
+Representative command:
+
+```sh
+./build-cuda/bin/llama-bench \
+  -m /home/me/Models/Qwen3.6-35B-A3B-Q4_K_M.gguf \
+  -p 128,512 -n 32,128 -r 3 -t 4 \
+  -fa on -fitt 1024 -fitc 4096 --progress -o jsonl
+```
+
+For Gemma chat/completion, add `--jinja`:
+
+```sh
+./build-cuda/bin/llama completion \
+  -m /home/me/Models/gemma-4-26B_q4_0-it.gguf \
+  -c 2048 -ngl auto -fit on -fitt 1024 -fa on --jinja -st \
+  -p 'Your prompt' -n 256
+```
+
+Raw data:
+
+- `qwen-bench.jsonl` and `gemma-bench.jsonl`: llama-bench results and samples
+- `qwen-telemetry.csv` and `gemma-telemetry.csv`: 500 ms `nvidia-smi` samples
+- `qwen-bench.log` and `gemma-bench.log`: benchmark progress and device detection
+
+## Gemma 4 12B official QAT + MTP
+
+The official Google QAT target and the matching QAT-derived MTP assistant were
+also installed and tested with the same `ht` Pascal build:
+
+| Role | File | Size | SHA-256 |
+| --- | --- | ---: | --- |
+| Target | `gemma-4-12b-it-qat-q4_0.gguf` | 6,975,879,296 bytes | `93567e57a8fe10b23569b9d9ec38cd005deedf71e29477c421a4b83f418a538b` |
+| MTP assistant | `mtp-gemma-4-12B-it-Q4_0.gguf` | 253,708,960 bytes | `b894e614824dfc2746b26d3c3ba78c50000a464382682502392b4325257b7602` |
+
+Both files were exact-size checked and SHA-256 verified after download. The target
+is from `google/gemma-4-12B-it-qat-q4_0-gguf`; the assistant is the QAT-derived
+`gemma4-assistant` GGUF from `ggml-org/gemma-4-12B-it-GGUF`.
+
+### Server generation benchmark
+
+The test used one server slot, a 4096-token context, full GPU offload, flash
+attention, greedy sampling, and three independent 128-token requests.
+
+| Mode | Generation | Prompt processing | Peak VRAM | Peak GPU | Peak power |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Target only | 24.02 tok/s | 159.77 tok/s | 7,524 MiB | 98% | 149.10 W |
+| QAT MTP | 56.44 tok/s | 156.16 tok/s | 7,824 MiB | 94% | 172.38 W |
+
+MTP improved generation throughput by **2.35x**. Every MTP run accepted 92 of
+111 proposed draft tokens (82.88%), and all three MTP responses were byte-identical
+to one another. All three target-only responses were also byte-identical to one
+another.
+
+The required MTP launch options are:
+
+```sh
+./build-cuda/bin/llama serve \
+  -m /home/me/Models/gemma-4-12b-it-qat-q4_0.gguf \
+  -md /home/me/Models/mtp-gemma-4-12B-it-Q4_0.gguf \
+  -c 4096 -ngl all -ngld all -fa on --parallel 1 \
+  --spec-type draft-mtp --spec-draft-n-max 16 --spec-draft-p-min 0.9 \
+  --no-spec-draft-backend-sampling \
+  --host 127.0.0.1 --port 8080 --no-webui --jinja
+```
+
+Two Pascal-specific cautions were confirmed. Supplying `-md` without the explicit
+`--spec-type draft-mtp` does not enable MTP. Also, draft backend sampling produced
+non-repeatable greedy output on this P5200; `--no-spec-draft-backend-sampling`
+made repeated MTP runs deterministic without materially reducing speed.
+
+The deterministic MTP response was not byte-identical to the target-only response.
+The target still verifies proposed tokens, but batched target evaluation can choose
+a different greedy token from single-token evaluation because GPU floating-point
+evaluation order differs. Therefore this local result demonstrates verified-token
+speculation and repeatability, but it should not be described as bit-identical to
+non-speculative generation.
+
+Additional raw data:
+
+- `gemma12-baseline-bench.jsonl` and `gemma12-mtp-bench.jsonl`: three API responses
+  per mode, including llama.cpp timing and MTP acceptance counters
+- `gemma12-baseline-telemetry.csv` and `gemma12-mtp-telemetry.csv`: 500 ms GPU samples
+- `gemma12-*-smoke.json` and `gemma12-mtp-cpu-sampler-*.json`: smoke and
+  determinism-isolation runs
diff --git a/docs/benchmarks/data/pascal-p5200-2026-08-02/gemma-bench.jsonl b/docs/benchmarks/data/pascal-p5200-2026-08-02/gemma-bench.jsonl
@@ -0,0 +1,4 @@
+{"build_commit": "798cf6cbe", "build_number": 9862, "cpu_info": "Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz", "gpu_info": "Quadro P5200", "backends": "CUDA", "model_filename": "/home/me/Models/gemma-4-26B_q4_0-it.gguf", "model_type": "gemma4 26B.A4B Q4_0", "model_size": 14423538808, "model_n_params": 25233142046, "n_batch": 2048, "n_ubatch": 512, "n_threads": 4, "cpu_mask": "0x0", "cpu_strict": false, "poll": 50, "type_k": "f16", "type_v": "f16", "n_gpu_layers": -1, "n_cpu_moe": 0, "split_mode": "layer", "main_gpu": 0, "no_kv_offload": false, "flash_attn": 1, "devices": "auto", "tensor_split": "0.00", "tensor_buft_overrides": "none", "use_mmap": true, "use_direct_io": false, "embeddings": false, "no_op_offload": 0, "no_host": false, "fit_target": 1024, "fit_min_ctx": 4096, "n_prompt": 128, "n_gen": 0, "n_depth": 0, "test_time": "2026-08-02T11:00:11Z", "avg_ns": 254244569, "stddev_ns": 1942637, "avg_ts": 503.471781, "stddev_ts": 3.834547, "samples_ns": [ 253634389, 256419044, 252680274 ],"samples_ts": [ 504.663, 499.183, 506.569 ]}
+{"build_commit": "798cf6cbe", "build_number": 9862, "cpu_info": "Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz", "gpu_info": "Quadro P5200", "backends": "CUDA", "model_filename": "/home/me/Models/gemma-4-26B_q4_0-it.gguf", "model_type": "gemma4 26B.A4B Q4_0", "model_size": 14423538808, "model_n_params": 25233142046, "n_batch": 2048, "n_ubatch": 512, "n_threads": 4, "cpu_mask": "0x0", "cpu_strict": false, "poll": 50, "type_k": "f16", "type_v": "f16", "n_gpu_layers": -1, "n_cpu_moe": 0, "split_mode": "layer", "main_gpu": 0, "no_kv_offload": false, "flash_attn": 1, "devices": "auto", "tensor_split": "0.00", "tensor_buft_overrides": "none", "use_mmap": true, "use_direct_io": false, "embeddings": false, "no_op_offload": 0, "no_host": false, "fit_target": 1024, "fit_min_ctx": 4096, "n_prompt": 512, "n_gen": 0, "n_depth": 0, "test_time": "2026-08-02T11:00:17Z", "avg_ns": 594408106, "stddev_ns": 2621377, "avg_ts": 861.372253, "stddev_ts": 3.805618, "samples_ns": [ 596507863, 591470453, 595246004 ],"samples_ts": [ 858.329, 865.639, 860.149 ]}
+{"build_commit": "798cf6cbe", "build_number": 9862, "cpu_info": "Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz", "gpu_info": "Quadro P5200", "backends": "CUDA", "model_filename": "/home/me/Models/gemma-4-26B_q4_0-it.gguf", "model_type": "gemma4 26B.A4B Q4_0", "model_size": 14423538808, "model_n_params": 25233142046, "n_batch": 2048, "n_ubatch": 512, "n_threads": 4, "cpu_mask": "0x0", "cpu_strict": false, "poll": 50, "type_k": "f16", "type_v": "f16", "n_gpu_layers": -1, "n_cpu_moe": 0, "split_mode": "layer", "main_gpu": 0, "no_kv_offload": false, "flash_attn": 1, "devices": "auto", "tensor_split": "0.00", "tensor_buft_overrides": "none", "use_mmap": true, "use_direct_io": false, "embeddings": false, "no_op_offload": 0, "no_host": false, "fit_target": 1024, "fit_min_ctx": 4096, "n_prompt": 0, "n_gen": 32, "n_depth": 0, "test_time": "2026-08-02T11:00:25Z", "avg_ns": 612631473, "stddev_ns": 913033, "avg_ts": 52.233763, "stddev_ts": 0.077763, "samples_ns": [ 611930312, 612300661, 613663447 ],"samples_ts": [ 52.2935, 52.2619, 52.1458 ]}
+{"build_commit": "798cf6cbe", "build_number": 9862, "cpu_info": "Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz", "gpu_info": "Quadro P5200", "backends": "CUDA", "model_filename": "/home/me/Models/gemma-4-26B_q4_0-it.gguf", "model_type": "gemma4 26B.A4B Q4_0", "model_size": 14423538808, "model_n_params": 25233142046, "n_batch": 2048, "n_ubatch": 512, "n_threads": 4, "cpu_mask": "0x0", "cpu_strict": false, "poll": 50, "type_k": "f16", "type_v": "f16", "n_gpu_layers": -1, "n_cpu_moe": 0, "split_mode": "layer", "main_gpu": 0, "no_kv_offload": false, "flash_attn": 1, "devices": "auto", "tensor_split": "0.00", "tensor_buft_overrides": "none", "use_mmap": true, "use_direct_io": false, "embeddings": false, "no_op_offload": 0, "no_host": false, "fit_target": 1024, "fit_min_ctx": 4096, "n_prompt": 0, "n_gen": 128, "n_depth": 0, "test_time": "2026-08-02T11:00:31Z", "avg_ns": 2484774465, "stddev_ns": 23834946, "avg_ts": 51.516873, "stddev_ts": 0.491517, "samples_ns": [ 2512217852, 2469251358, 2472854187 ],"samples_ts": [ 50.951, 51.8376, 51.762 ]}
diff --git a/docs/benchmarks/data/pascal-p5200-2026-08-02/gemma-bench.log b/docs/benchmarks/data/pascal-p5200-2026-08-02/gemma-bench.log
@@ -0,0 +1,22 @@
+ggml_cuda_init: found 1 CUDA devices (Total VRAM: 16266 MiB):
+  Device 0: Quadro P5200, compute capability 6.1, VMM: yes, VRAM: 16266 MiB
+llama-bench: benchmark 1/4: starting
+llama-bench: benchmark 1/4: warmup prompt run
+llama-bench: benchmark 1/4: prompt run 1/3
+llama-bench: benchmark 1/4: prompt run 2/3
+llama-bench: benchmark 1/4: prompt run 3/3
+llama-bench: benchmark 2/4: starting
+llama-bench: benchmark 2/4: warmup prompt run
+llama-bench: benchmark 2/4: prompt run 1/3
+llama-bench: benchmark 2/4: prompt run 2/3
+llama-bench: benchmark 2/4: prompt run 3/3
+llama-bench: benchmark 3/4: starting
+llama-bench: benchmark 3/4: warmup generation run
+llama-bench: benchmark 3/4: generation run 1/3
+llama-bench: benchmark 3/4: generation run 2/3
+llama-bench: benchmark 3/4: generation run 3/3
+llama-bench: benchmark 4/4: starting
+llama-bench: benchmark 4/4: warmup generation run
+llama-bench: benchmark 4/4: generation run 1/3
+llama-bench: benchmark 4/4: generation run 2/3
+llama-bench: benchmark 4/4: generation run 3/3
diff --git a/docs/benchmarks/data/pascal-p5200-2026-08-02/gemma-telemetry.csv b/docs/benchmarks/data/pascal-p5200-2026-08-02/gemma-telemetry.csv
@@ -0,0 +1,77 @@
+timestamp, utilization.gpu [%], memory.used [MiB], power.draw [W], temperature.gpu
+2026/08/02 13:00:01.733, 0 %, 90 MiB, 16.44 W, 54
+2026/08/02 13:00:02.237, 0 %, 92 MiB, 9.72 W, 54
+2026/08/02 13:00:02.739, 0 %, 198 MiB, 40.06 W, 55
+2026/08/02 13:00:03.241, 0 %, 198 MiB, 40.23 W, 55
+2026/08/02 13:00:03.744, 0 %, 198 MiB, 40.20 W, 55
+2026/08/02 13:00:04.246, 0 %, 198 MiB, 40.19 W, 55
+2026/08/02 13:00:04.748, 0 %, 198 MiB, 40.30 W, 55
+2026/08/02 13:00:05.249, 0 %, 198 MiB, 40.33 W, 55
+2026/08/02 13:00:05.751, 0 %, 198 MiB, 40.31 W, 55
+2026/08/02 13:00:06.252, 0 %, 198 MiB, 40.34 W, 55
+2026/08/02 13:00:06.754, 0 %, 198 MiB, 40.17 W, 55
+2026/08/02 13:00:07.256, 0 %, 198 MiB, 40.30 W, 55
+2026/08/02 13:00:07.757, 0 %, 198 MiB, 40.31 W, 55
+2026/08/02 13:00:08.259, 0 %, 198 MiB, 40.31 W, 55
+2026/08/02 13:00:08.760, 0 %, 198 MiB, 40.19 W, 55
+2026/08/02 13:00:09.262, 32 %, 13954 MiB, 41.76 W, 55
+2026/08/02 13:00:09.764, 84 %, 13954 MiB, 54.60 W, 56
+2026/08/02 13:00:10.265, 86 %, 13954 MiB, 54.60 W, 56
+2026/08/02 13:00:10.767, 86 %, 13954 MiB, 54.76 W, 56
+2026/08/02 13:00:11.269, 1 %, 14156 MiB, 62.33 W, 56
+2026/08/02 13:00:11.771, 100 %, 14178 MiB, 145.38 W, 60
+2026/08/02 13:00:12.273, 100 %, 14178 MiB, 148.64 W, 62
+2026/08/02 13:00:12.774, 63 %, 208 MiB, 51.67 W, 59
+2026/08/02 13:00:13.276, 0 %, 208 MiB, 51.31 W, 58
+2026/08/02 13:00:13.777, 0 %, 208 MiB, 51.18 W, 58
+2026/08/02 13:00:14.279, 0 %, 208 MiB, 51.18 W, 58
+2026/08/02 13:00:14.781, 0 %, 208 MiB, 51.14 W, 58
+2026/08/02 13:00:15.282, 0 %, 208 MiB, 51.14 W, 58
+2026/08/02 13:00:15.784, 84 %, 13964 MiB, 53.61 W, 58
+2026/08/02 13:00:16.285, 62 %, 13964 MiB, 53.30 W, 57
+2026/08/02 13:00:16.787, 56 %, 13964 MiB, 52.89 W, 58
+2026/08/02 13:00:17.289, 68 %, 13964 MiB, 53.34 W, 58
+2026/08/02 13:00:17.790, 0 %, 13964 MiB, 51.04 W, 57
+2026/08/02 13:00:18.292, 100 %, 14652 MiB, 142.02 W, 61
+2026/08/02 13:00:18.793, 100 %, 14652 MiB, 142.68 W, 62
+2026/08/02 13:00:19.295, 97 %, 14652 MiB, 125.08 W, 63
+2026/08/02 13:00:19.797, 97 %, 14652 MiB, 155.68 W, 64
+2026/08/02 13:00:20.314, 100 %, 14652 MiB, 148.33 W, 64
+2026/08/02 13:00:20.815, 67 %, 208 MiB, 52.46 W, 61
+2026/08/02 13:00:21.317, 0 %, 208 MiB, 51.96 W, 60
+2026/08/02 13:00:21.818, 0 %, 208 MiB, 51.67 W, 60
+2026/08/02 13:00:22.320, 0 %, 208 MiB, 51.63 W, 59
+2026/08/02 13:00:22.821, 0 %, 208 MiB, 51.59 W, 59
+2026/08/02 13:00:23.323, 63 %, 13964 MiB, 54.31 W, 59
+2026/08/02 13:00:23.825, 86 %, 13964 MiB, 54.31 W, 59
+2026/08/02 13:00:24.326, 85 %, 13964 MiB, 54.15 W, 59
+2026/08/02 13:00:24.828, 48 %, 13964 MiB, 51.37 W, 59
+2026/08/02 13:00:25.330, 99 %, 14056 MiB, 143.01 W, 61
+2026/08/02 13:00:25.831, 99 %, 14056 MiB, 144.13 W, 62
+2026/08/02 13:00:26.333, 99 %, 14056 MiB, 130.68 W, 63
+2026/08/02 13:00:26.834, 99 %, 14056 MiB, 149.54 W, 63
+2026/08/02 13:00:27.336, 94 %, 208 MiB, 53.70 W, 61
+2026/08/02 13:00:27.838, 0 %, 208 MiB, 51.94 W, 60
+2026/08/02 13:00:28.339, 0 %, 208 MiB, 51.83 W, 60
+2026/08/02 13:00:28.841, 0 %, 208 MiB, 51.77 W, 60
+2026/08/02 13:00:29.342, 79 %, 13964 MiB, 54.44 W, 59
+2026/08/02 13:00:29.844, 86 %, 13964 MiB, 54.28 W, 59
+2026/08/02 13:00:30.346, 86 %, 13964 MiB, 54.15 W, 59
+2026/08/02 13:00:30.849, 33 %, 13964 MiB, 51.51 W, 59
+2026/08/02 13:00:31.351, 99 %, 14156 MiB, 134.88 W, 61
+2026/08/02 13:00:31.854, 98 %, 14156 MiB, 135.21 W, 62
+2026/08/02 13:00:32.356, 99 %, 14156 MiB, 136.11 W, 63
+2026/08/02 13:00:32.858, 92 %, 14156 MiB, 138.08 W, 64
+2026/08/02 13:00:33.363, 99 %, 14156 MiB, 130.37 W, 64
+2026/08/02 13:00:33.865, 99 %, 14156 MiB, 135.03 W, 65
+2026/08/02 13:00:34.367, 99 %, 14156 MiB, 144.91 W, 65
+2026/08/02 13:00:34.868, 99 %, 14156 MiB, 139.84 W, 66
+2026/08/02 13:00:35.370, 99 %, 14156 MiB, 137.82 W, 66
+2026/08/02 13:00:35.872, 99 %, 14156 MiB, 139.90 W, 66
+2026/08/02 13:00:36.373, 99 %, 14156 MiB, 139.43 W, 66
+2026/08/02 13:00:36.875, 99 %, 14156 MiB, 144.74 W, 66
+2026/08/02 13:00:37.376, 99 %, 14156 MiB, 131.98 W, 66
+2026/08/02 13:00:37.879, 99 %, 14156 MiB, 136.58 W, 66
+2026/08/02 13:00:38.383, 99 %, 14156 MiB, 132.86 W, 66
+2026/08/02 13:00:38.886, 90 %, 208 MiB, 61.18 W, 64
+2026/08/02 13:00:39.388, 0 %, 208 MiB, 53.24 W, 63
diff --git a/docs/benchmarks/data/pascal-p5200-2026-08-02/gemma12-baseline-bench.jsonl b/docs/benchmarks/data/pascal-p5200-2026-08-02/gemma12-baseline-bench.jsonl
@@ -0,0 +1,3 @@
+{"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"","reasoning_content":"*   Topic: Why RAID is not a backup.\n    *   Constraint 1: Exactly four short paragraphs.\n    *   Constraint 2: Include one practical example.\n\n    *   Paragraph 1: Definition/Purpose of RAID (Redundancy vs. Backup).\n    *   Paragraph 2: Failure modes (Human error, malware, hardware failure).\n    *   Paragraph 3: The practical example (Deleting a file).\n    *   Paragraph 4: Conclusion/Best practice (3-2-1 rule).\n\n    *   *Paragraph 1:* RAID (Red"}}],"created":1785674994,"model":"/home/me/Models/gemma-4-12b-it-qat-q4_0.gguf","system_fingerprint":"b9862-798cf6cbe","object":"chat.completion","usage":{"completion_tokens":128,"prompt_tokens":34,"total_tokens":162,"prompt_tokens_details":{"cached_tokens":0}},"id":"chatcmpl-rFXXVTOCeizmTtKdbrsOL62vWx72JIFT","timings":{"cache_n":0,"prompt_n":34,"prompt_ms":218.519,"prompt_per_token_ms":6.427029411764706,"prompt_per_second":155.59287750721904,"predicted_n":128,"predicted_ms":5323.693,"predicted_per_token_ms":41.5913515625,"predicted_per_second":24.04346005676886}}
+{"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"","reasoning_content":"*   Topic: Why RAID is not a backup.\n    *   Constraint 1: Exactly four short paragraphs.\n    *   Constraint 2: Include one practical example.\n\n    *   Paragraph 1: Definition/Purpose of RAID (Redundancy vs. Backup).\n    *   Paragraph 2: Failure modes (Human error, malware, hardware failure).\n    *   Paragraph 3: The practical example (Deleting a file).\n    *   Paragraph 4: Conclusion/Best practice (3-2-1 rule).\n\n    *   *Paragraph 1:* RAID (Red"}}],"created":1785675000,"model":"/home/me/Models/gemma-4-12b-it-qat-q4_0.gguf","system_fingerprint":"b9862-798cf6cbe","object":"chat.completion","usage":{"completion_tokens":128,"prompt_tokens":34,"total_tokens":162,"prompt_tokens_details":{"cached_tokens":0}},"id":"chatcmpl-hcfj6UnLHjOFc7kcSg1W4KTaxgiLbpCJ","timings":{"cache_n":0,"prompt_n":34,"prompt_ms":210.052,"prompt_per_token_ms":6.178,"prompt_per_second":161.8646811265782,"predicted_n":128,"predicted_ms":5330.966,"predicted_per_token_ms":41.648171875,"predicted_per_second":24.010657730700213}}
+{"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"","reasoning_content":"*   Topic: Why RAID is not a backup.\n    *   Constraint 1: Exactly four short paragraphs.\n    *   Constraint 2: Include one practical example.\n\n    *   Paragraph 1: Definition/Purpose of RAID (Redundancy vs. Backup).\n    *   Paragraph 2: Failure modes (Human error, malware, hardware failure).\n    *   Paragraph 3: The practical example (Deleting a file).\n    *   Paragraph 4: Conclusion/Best practice (3-2-1 rule).\n\n    *   *Paragraph 1:* RAID (Red"}}],"created":1785675005,"model":"/home/me/Models/gemma-4-12b-it-qat-q4_0.gguf","system_fingerprint":"b9862-798cf6cbe","object":"chat.completion","usage":{"completion_tokens":128,"prompt_tokens":34,"total_tokens":162,"prompt_tokens_details":{"cached_tokens":0}},"id":"chatcmpl-DklyrNQ4SCFFuljBFuhrMPzE72g9ZJA1","timings":{"cache_n":0,"prompt_n":34,"prompt_ms":210.056,"prompt_per_token_ms":6.178117647058824,"prompt_per_second":161.8615988117454,"predicted_n":128,"predicted_ms":5331.456,"predicted_per_token_ms":41.652,"predicted_per_second":24.00845097474311}}