diff --git a/docs/community-benchmarks.md b/docs/community-benchmarks.md index 87a242cdee..0e54064674 100644 --- a/docs/community-benchmarks.md +++ b/docs/community-benchmarks.md @@ -13,7 +13,7 @@ The command starts its own single-request `serve` on a loopback port, disables p An engine that decodes greedily only refuses the sampled cells, so it cannot complete community-v1; the run prints the server's reason, and `--temperatures 0` gives a local run. A large checkpoint may need its engine's own load limit raised (`GLM_LOAD_LIMIT_GB` for GLM-5.3-Flash) and a longer `--startup-timeout`: without the flag, the run waits up to 600 s plus 6 s per GB of weights past 20 GB, at most 3600 s. -Results are saved locally in `~/.tensorfold/benchmarks/RUN_ID.json`; `--output FILE` chooses another location. Files contain public hardware and model metadata and individual attempts, never prompt text, generated output, machine names, serial numbers, GPU UUIDs, addresses or local model paths. Local model directories can be benchmarked but cannot be published, and a drafter directory is never named. Managed native runs record `python_version` as `"none"`. An attached manifest may attest the same value; an unattested attached run records `"unknown"`. +Results are saved locally in `~/.tensorfold/benchmarks/RUN_ID.json`; `--output FILE` chooses another location. Files contain public hardware and model metadata and individual attempts, never prompt text, generated output, machine names, serial numbers, GPU UUIDs, addresses or local model paths. Local model directories can be benchmarked but cannot be published, and a drafter directory is never named. Managed native runs record `python_version` as `"none"`. An attached manifest may attest the same value; an unattested attached run records `"unknown"`. Beside each receipt, `RUN_ID.drafts.json` (for `--output FILE.json`, `FILE.drafts.json`) keeps what the server reported for each attempt: its verify rounds, the tokens a drafter proposed and the tokens the target accepted, with their totals (null, and the run says so, when a sum would pass 64 bits) and whether the run passed a drafter. The board's receipt has no field for these counts, so they are never uploaded; they are how a local run shows that its drafter ran, since equal timings do not. When a drafter was passed and the server drafted no tokens, the run says so. ## Commands diff --git a/docs/receipts/drafter-counts/README.md b/docs/receipts/drafter-counts/README.md new file mode 100644 index 0000000000..8ba197acb4 --- /dev/null +++ b/docs/receipts/drafter-counts/README.md @@ -0,0 +1,124 @@ +# Draft counts beside the receipt: gfx1151 runs and host controls + +These are the receipts behind [#1045](https://github.com/ashhart/TensorFold/pull/1045): each benchmark attempt's +verify rounds, drafted tokens and accepted tokens, as the server reported them, kept in `RUN_ID.drafts.json` beside +the board's receipt. They show that the counts in the file are the server's own: for the two attached runs, every +sample's counts are the same numbers the server printed for that request in its log. + +## Where and when + +The GPU runs were measured on 18 October 2026 on a Bosgame M5 (AMD Ryzen AI Max+ 395, Radeon 8060S, `gfx1151`, +one GPU, Linux, HIP backend; the receipts record it as `rocm`). Times below are UTC, as in the receipts. + +| Folder | Measured | Binary SHA-256 (`tensorfold-native`, HIP build) | Source | +| --- | --- | --- | --- | +| `out/` | 17:33 to 17:47 | `e279e29a54b56dc81ff70985f8c4cffaa0d9aff4a7456a234eb7247667492247` | see below | +| `out-r5b/` | ended 18:18 | `7fcaa74ec93561902ef97675f00db1b6d022392c665a7578960e18c92bcd2407` | `out/` plus the identity fix | + +The `out/` binary was built after the merge that `heads.txt` records (`7b84f909`, made on the box and not published) +of `hip-refuses-drafter` and this branch. The bundle prepared for the run carries main at `76aafcf2`, +`hip-refuses-drafter` at `920d1348` and this branch at `1b4b2d7b`; the branch's later `d2dfb603` changes comments +only. The `out-r5b/` binary adds `benchmark-rocm-host-identity` at `624e2ce6` (merge `25a7201f`, +`out-r5b/heads-r5b.txt`). The second line of `heads.txt` is `git rev-parse` echoing a ref the bundle clone did not +have (the matching `fatal:` line is in `run.log`). + +The checkpoints are `TensorFold/Qwen3.8-27B-MLX-4bit` at `70ae7fac63274ff2eac54152031433374cb80f2f` and +`TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP` at `f84b054c5677b6e59bdb969ae61915b0983d68bb` (`pull.log`). Every run is +community-v1: code and chat, temperatures 1 and 0, one warm-up and five repeats per cell, 256 tokens per reply. + +## The runs + +| Run | How | Drafter passed | Rounds | Drafted | Accepted | Server log | +| --- | --- | --- | ---: | ---: | ---: | --- | +| `out/27b-auto` | managed | no | 5030 | 262 | 70 | not kept | +| `out/35b` | managed | no | 2495 | 4490 | 2615 | not kept | +| `out/35b-serial` | managed, `serial: true` | no | 5100 | 0 | 0 | not kept | +| `out/35b-attached` | attached to `out/35b-serve.log`'s server | unknown | 2441 | 4626 | 2668 | `out/35b-serve.log` | +| `out-r5b/35b-attached` | attached to `out-r5b/35b-serve.log`'s server | unknown | 2473 | 4576 | 2636 | `out-r5b/35b-serve.log` | + +Each total is the sum of its 20 samples and the same as the run's own `Draft counts:` line (line 37 of each `.log`). +A managed run unlinks its server's log as it starts, so only the attached runs can be held against a server log. + +## Files + +- `out/.json`: the board's receipt, community-v1 fields only. `out/.drafts.json`: the counts file beside + it. `out/.log`: the run's console output. The same three for `out-r5b/35b-attached`. +- `out/35b-serve.log`, `out-r5b/35b-serve.log`: the attached server's log for each attached run. +- `out/35b-manifest.json`, `out-r5b/35b-manifest.json`: the attached server's benchmark manifest, saved with each run. +- `out/27b-dflash2.log`, `out/27b-serve-drafter.log`: two refusals of `--drafter` on HIP (below). +- `run.log`, `out-r5b/run-r5b.log`: the box's run script: merges, build exit, binary SHA-256, each step's exit, GPU + memory and busy readings between steps, and the state of the box's other LLM service before and after. +- `heads.txt`, `out-r5b/heads-r5b.txt`: the merged source. `binary.sha256`: the first binary's SHA-256. + `pull.log`: the two checkpoint downloads. +- `match.py`, `match.txt`: the check of the counts against the server logs, and its output (written for this folder). +- `host-controls/`: the CPU checks of the overflow fix. `SHA256SUMS`: see the last section. + +## Counts against the server's log + +Each attached server prints one line per finished request, for example `out/35b-serve.log` line 10: +`done cmpl-... tokens=256 sha=5d7effdcf06f finish=length rounds=106 accepted=150/246`, where `accepted=A/D` is +accepted over drafted. The POST line just before it names the route. In both server logs the 24 finished requests are +lines 8 to 54 (every even line), each after its POST line: one warm-up per cell at lines 8, 20, 32 and 44, then the +cell's five repeats. Warm-ups are not samples, so the file leaves them out. + +`match.py` (Python 3, standard library) reads the sanitized copies here and checks, for every sample of both attached +runs, that the file's rounds, drafted, accepted and completion tokens are the server's for that request, that the +receipt's `token_sha` is the server's `sha=`, and that the route is `/v1/completions` for code and +`/v1/chat/completions` for chat. It prints each sample beside its log line, route and `finish=` (all `length`), since +the counts file records neither the route nor the finish reason and the receipt's schema stays as it is. For all five +runs it checks that the totals are the samples' sum and the run's printed line. Its output is `match.txt`. Two +controls run in the same script: a file with one drafted count off by one, and the server log read one request out of +step; it refuses both. + +```sh +python3 docs/receipts/drafter-counts/match.py +``` + +## What else the GPU runs show + +- `out/35b` and `out/35b-serial` load the same checkpoint. With `serial: true` the server drafted no tokens: 255 + rounds for each 256-token reply. The default run drafted 4490 tokens and accepted 2615 in 2495 rounds. +- `out/27b-auto.log` says the run passes no drafter, as the HIP engine runs no DFlash2 drafter yet, and the file says + `drafter_passed: false`. The server still reported 262 drafted and 70 accepted tokens in 14 of the 20 attempts. + These receipts do not show where those drafts came from: the managed run's server log is not kept. +- `out/27b-dflash2.log` and `out/27b-serve-drafter.log` are the refusals from `hip-refuses-drafter`: the benchmark and + `serve` both refuse `--drafter` on HIP. `run.log` records both steps (R2a, R2b) exiting 1, and no `27b-dflash2.json`. +- `out/35b-manifest.json` and `out-r5b/35b-manifest.json` are the attached server's benchmark manifest, saved with + each run; the second carries the identity fix. The counts match their server logs in both runs. + +## Host controls + +`host-controls/` holds the CPU-only checks, on a MacBook (Apple silicon, macOS, Zig 0.17.0) on 19 October 2026. +`commands.txt` says what each log ran. + +- `host-tests.log`: the host test step on `616d3065`, the overflow fix: 292 of 292 steps succeed. Steps whose inputs + did not change since the first run come from the cache, so its summary counts the 958 tests that ran again (957 + passed, 1 skipped), among them the 433 of 433 in the server's test binary, which holds the one new unit test and + the one new case in the existing run test. +- `host-tests-full.log`: the first, uncached run of the same step, before the commit, on a tree that differs from + `616d3065` only in one comment's wording: 292 of 292 steps, 4315 of 4331 tests passed, 16 skipped. +- `negative-control.log`: `zig/src/server/benchmark_drafts.zig` put back as `d2dfb603` has it, the unchecked sums, + with the new run test kept. That test aborts with `panic: integer overflow` at `t.rounds += d.rounds` + (`benchmark_drafts.zig:18`, from `totals`), in both binaries that hold the server's tests: 431 passed and 1 + crashed of 432 in each. The summary: 288 of 292 steps, 2 failed. +- `lean.txt`: the lean check finds 503 problems on `616d3065`, on main (`b7ce8121`) and on the merge base + (`76aafcf2`), none new. `fmt.txt`: `zig fmt --check` passes. + +In a passing log, a `failed command:` line is the build runner printing a test binary's stderr (expected warnings +from the tests) under its command; the `Build Summary` line is the result. + +The GPU runs predate the overflow fix. For sums that fit in 64 bits the fix writes the same file and the same line. + +## Sanitized copies + +These are sanitized copies of the raw files. Changed: `/home/` became `~` in `binary.sha256`, `run.log`, +`pull.log`, `out-r5b/run-r5b.log` and the `Receipt saved:` and `Draft counts:` lines of every run's `.log`; +trailing spaces were removed from line 1 of both `35b-serve.log` files; the host logs' local paths became `~` and +their trailing spaces were removed. Every receipt, counts file and manifest is byte for byte the raw file, so its hash +in `SHA256SUMS` is the original's. Counts, hashes, run IDs, times and the binary SHA-256 values above are unchanged. + +The hashes of the raw files that were sanitized are not published. Such a file differs from its published copy only in +the replaced paths and the removed spaces, so its raw hash would let anyone who guesses a replaced path check the +guess. `SHA256SUMS` lists the published bytes of every file here except itself; run `shasum -a 256 -c SHA256SUMS` in +this folder. Not copied: an empty `build.log` (the build's exit, 0, is in `run.log`) and two scripts that check +receipts against the board's backend list, which is not about draft counts. diff --git a/docs/receipts/drafter-counts/SHA256SUMS b/docs/receipts/drafter-counts/SHA256SUMS new file mode 100644 index 0000000000..04850a8116 --- /dev/null +++ b/docs/receipts/drafter-counts/SHA256SUMS @@ -0,0 +1,36 @@ +5c6540f8fdc7b7131a0576ad27d44f4f8b045dbc9f7c996f5c9ec39e5cc7ca09 README.md +c8d46b5b5e8f9361422854deb9ec085dddcf4cbe1024963ac8d48ec0570e9b34 binary.sha256 +f9ef9a708c394481e9541c28d9852536dee07d9a5e644e17daa4f6542dc6cb9e heads.txt +5518e54903ed0183d01a4818af45a15a8b3fd3da4311216f1daf83f0ccea04c4 host-controls/commands.txt +c1b5e4c5afe80315680e2389c046fe88950f61f9d95926dd7fd809ed2c59e255 host-controls/fmt.txt +8a6e05131f8d36349a7f4411c335d184ff5cab6673ea2de0a9f889b2ddf0f189 host-controls/host-tests-full.log +dc5719f3986c65a850849ba148826c75978bfeb74d364762f8415ccb58461ef7 host-controls/host-tests.log +502144bc91d20a4ec26cfa5ef5aab36b266913b3b9f9b876e75b72daf20f4c65 host-controls/lean.txt +73f79a91aa5c7133b5b6b4e5138f6e1b00b624f340fc0e0446ec2948afbdba0a host-controls/negative-control.log +384d539232f442966c80368c1c194711b8299de92bce67d540976e65a8145d91 match.py +79a9859f54a31680e21d8acccb7a0dc8cd254340e062d5d22736e4d6ebbeb642 match.txt +ed60c8674304318863468c3db31f7ef3148c0d53a24f9b15b325ce01b2e181ce out-r5b/35b-attached.drafts.json +9646641b849fbed3d9f27311b36add30dcddfe68036a755cca10fd67fd1af457 out-r5b/35b-attached.json +e267ac9084bc4927e40e625f579736237bf7930ab279d2a51c01f86e9f2e4f0e out-r5b/35b-attached.log +09a0007d8ce0b59842642ebb8b95f5995e94006d3194b567ac04b6733dab3d11 out-r5b/35b-manifest.json +1f946eab3ea5eeb58a250dbeed91b4ad4aec61cf888bd54e01d91f5324430012 out-r5b/35b-serve.log +3bbe4a9b12f87d63e4e6fed9ab5095923c748d09e209138521b281ee6b1e4a31 out-r5b/heads-r5b.txt +70cfeffeb0d1e6c9cfc28e8cd3636823bbc03cbace8514ddcf0097a613bd3261 out-r5b/run-r5b.log +689ee610811cc72e505ddded6e982bdbdf337f7d6aafcb2c2bd227b78acb8870 out/27b-auto.drafts.json +95ae8bad28ac5702aa66c326f96ac7b6816a4a5834f4f779c35ed117bb5c7a6e out/27b-auto.json +32ed01a245b52ad8823c332a4ef30ae381503d754abdb1a3efa8bf98d2baad66 out/27b-auto.log +d0c05b5d100079881ccb6a5cf05060629cc529c87893eac3c625fda50439e8c2 out/27b-dflash2.log +c53992d45e846da1a8d97e0864ea658c256624bc891621b558d1de5f9aae5a9e out/27b-serve-drafter.log +b96cd7d5b770026988bf13a1d5b7a6301fda14de94350d5744ac26fd2c9e2807 out/35b-attached.drafts.json +097812b10a5665c85289b2d4142dd79385855f23cac533ae9b8098c9b4dd677c out/35b-attached.json +6f0cb0ba3dc3a742be06063aa5bb20ab54b506534f9319858ed0e1b29229bbb1 out/35b-attached.log +376497f0f00b1b5277e641ccf66abc79b1ded09f947f0016e87040f80ae276f0 out/35b-manifest.json +92fe121f484b1682a0ed911e8fc025a580343a496b362464dc24c089d7715a62 out/35b-serial.drafts.json +c61da6ee47d938af90a55fea95153c06cfceccfbd6272d50dbfa08aff0d2b08c out/35b-serial.json +7e6d8dfd93e5b2ee7a516fc7b6e12bd7179a6b867f3113bb7118d94db4d54111 out/35b-serial.log +86c2555de69aa7699a5d6e0d9c660120d5cede3d65a1d6528ba335c348824050 out/35b-serve.log +9bb8baa0274cf8e4a1658dbc77ecf8fa22764e79ab3cec1bac093bbda47e6470 out/35b.drafts.json +c895bc792123d2542555d71a17d5c563879ff4434b585daba38bec920026d70e out/35b.json +a4af1cfe5c5b51d751ce6d4f560f106bb2faa84315bf9055896634839548ffb8 out/35b.log +8c9fe07b7b34c04f534b69eab48a9e2261072e40fcee63d5aedab132935ae4e8 pull.log +e7118e8cc40d5469c81eca4215abfd7e2c7ed03ada41153ce8ddc4728dbc6d9c run.log diff --git a/docs/receipts/drafter-counts/binary.sha256 b/docs/receipts/drafter-counts/binary.sha256 new file mode 100644 index 0000000000..757094470c --- /dev/null +++ b/docs/receipts/drafter-counts/binary.sha256 @@ -0,0 +1 @@ +e279e29a54b56dc81ff70985f8c4cffaa0d9aff4a7456a234eb7247667492247 ~/tf-1001/zig-out/native-hip/bin/tensorfold-native diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/README.md b/docs/receipts/drafter-counts/fresh-2026-10-19/README.md new file mode 100644 index 0000000000..51da325b4d --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/README.md @@ -0,0 +1,66 @@ +# A fresh gfx1151 run with all three #1001 branches + +The run offered on [#1001](https://github.com/ashhart/TensorFold/issues/1001): GPU, CPU and checkpoint identity and +each request's drafted and accepted tokens, measured after the fixes in #1043, #1044 and #1045, with all three merged +into one binary. The 18 October receipts in `../out/` and `../out-r5b/` were made with earlier heads of these +branches, before the visible-device commits `82237515` and `fb8eaf85` (#1043) and the last fixes of #1044 and #1045. + +## Where and when + +A Bosgame M5, 19 October 2026, 17:31 to 17:47 UTC (the `t=` stamps in `run.log`). The receipts record the CPU +(`AMD RYZEN AI MAX+ 395 with Radeon 8060S`), one GPU (`AMD Radeon 8060S Graphics`), Linux and the `rocm` backend; the +binary was built for `gfx1151` (`prep.sh`). The checkpoints are the 18 October ones, offline: +`TensorFold/Qwen3.8-27B-MLX-4bit` at `70ae7fac63274ff2eac54152031433374cb80f2f` and +`TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP` at `f84b054c5677b6e59bdb969ae61915b0983d68bb`. + +Source (`heads.txt`, `prep.log`): main `76aafcf2` with `benchmark-rocm-host-identity` at `c949cc98` (#1043), +`hip-refuses-drafter` at `f2c78eeb` (#1044) and `benchmark-drafter-counts` at `134a5f43` (#1045), merged on the box +(`7f4db1cf`, not published; git merged the three without a conflict) and built with `-Doptimize=ReleaseFast` +(`binary.sha256`). `prep.sh` merged and built; `run.sh` ran R1 to R6 on the box. `match.py` and `check.mjs` were run +afterwards on a Mac. + +## The runs + +Each benchmark ran 20 attempts, all `ok`; the counts are summed over the 20 and every attempt reported them. + +| Run | What it is | Exit | Accepted of drafted, rounds | +| --- | --- | --- | --- | +| R1 `27b-auto` | 27B, default drafting, no drafter passed | 0 | 70 of 262, 5030 | +| R2a `27b-dflash2` | `benchmark --drafter z-lab/Qwen3.8-27B-DFlash2`: refused, no receipt | 1 | none | +| R2b `27b-serve-drafter` | `serve --drafter z-lab/Qwen3.8-27B-DFlash2`: refused | 1 | none | +| R3 `35b` | 35B-A3B-MTP, MTP drafts | 0 | 2466 of 4251, 2642 | +| R4 `35b-serial` | the same, `--serial` (plain decode) | 0 | 0 of 0, 5100 | +| R5 `35b-attached` | the same served, benchmarked attached (`35b-manifest.json`) | 0 | 2626 of 4537, 2483 | +| R6 `35b-hipvis0` | R3 with `HIP_VISIBLE_DEVICES=0` set by `run.sh` | 0 | 2601 of 4500, 2507 | + +R1's counts are the same as 18 October's (`../out/27b-auto`): 70 of 262 over 5030 rounds. This checkpoint has no +MTP tensors (its file list in `../pull.log` has no `mtp-4bit.safetensors`), so these are not MTP drafts; a managed run +keeps no server log, so the receipts do not show where they come from. + +## What the receipts show + +- Identity: every receipt names the CPU, the one GPU (40 compute units, 128,849,018,880 bytes), unified memory with + source `detected`, and the checkpoint's `revision`, `config_sha256` and `tokenizer_sha256` with `local` false. R5's + attached receipt takes them from the server's manifest, which carries no `family` or `quantization`, so those two are + null there, as in `../out-r5b/`. +- Per-request counts: `match.py` is `../match.py` with `RUNS` set to these five runs and its two controls pointed at + R5. It finds every count matching: each file's samples sum to its totals and to the run's printed line, and each of + R5's 20 samples has the counts the server printed for that request (`35b-serve.log`; the four warm-ups are left out). + A drafted count off by one and the server log read one request out of step are both refused (`match.txt`). +- R6 names the same GPU as R3. The receipts do not record the environment; `run.sh` sets the variable for R6 only. + On a box with one GPU this shows that the visible-device changes keep the GPU when + `HIP_VISIBLE_DEVICES` is set; it cannot tell the old probe from the new one. The host tests and negative controls in + #1043 (`docs/receipts/rocm-host-identity/host-controls/`) do that. +- The board: `board-validator.log` runs the board's `protocol.js` (the same file, by sha256, as on 18 October) through + `check.mjs` from #1043. Each receipt is refused as `rocm` ("Unsupported benchmark runtime.") and accepted when only + `runtime.backend` is changed to `cuda`, in memory. + +The box's own `llm-server` was stopped for the run and restarted by `run.sh`'s exit trap (stopped by 17:31:37 UTC, +answering again at 17:50:29, in `run.log`). + +## Sanitizing + +Home paths are written as `~` in the text files (`*.log`, `binary.sha256`), and the trailing spaces of line 1 of +`35b-serve.log` were removed, as in `../out/`. The receipts, draft-count files and the manifest are as the run wrote +them. `SHA256SUMS` covers every file in this folder except itself +(`shasum -a 256 -c SHA256SUMS` from this folder). diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/SHA256SUMS b/docs/receipts/drafter-counts/fresh-2026-10-19/SHA256SUMS new file mode 100644 index 0000000000..e608c49074 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/SHA256SUMS @@ -0,0 +1,30 @@ +438a28199e25d340d665713214f623c019df70666597dff2a0a9e80719d2b59e ./README.md +f7c20db123a364bd2e41e2d61675ed643d83f2ae944131ff4f0e30f6bfd4ae72 ./binary.sha256 +d7eed7ba2cb98b2a58b861e8d60666d3f65353f645fcb987e1ab69e25ed5b5b5 ./board-validator.log +23b78e745c0519e1c012e62e17302f81f3ffb4a8441bb2cb0c3a780e80829466 ./check.mjs +e24a91984c9cdccab535cd0bac10682aecc459bb97720411d76a271e40f6bead ./heads.txt +dca0b4bc9e0a71a7f2fb73a9596a5c10fba2aed8109abfe5e6defebd44e0703f ./match.py +b3dac7037c801b249b590a7d91fd9e6adb86e3e49d41d0be646fd7bf3182da87 ./match.txt +91ccaf8e4698ee00c3fd76967da550bb47112e763593de589929b5061e105c87 ./out/27b-auto.drafts.json +2bb91ef0577e69efae3da926f7eebfd0a8be6856465c396b0e74b1c26fe63897 ./out/27b-auto.json +56c3c7189b0b3cf78e857f9b4bf2818be6da7b7388c7c50dc1706ee53482e9ab ./out/27b-auto.log +d0c05b5d100079881ccb6a5cf05060629cc529c87893eac3c625fda50439e8c2 ./out/27b-dflash2.log +c53992d45e846da1a8d97e0864ea658c256624bc891621b558d1de5f9aae5a9e ./out/27b-serve-drafter.log +6aa16eccbff858a46d5b9aa9e768348bbaf206a251056908877224a4d9f8302d ./out/35b-attached.drafts.json +5487128d17ac4b303716d3768bb219ba02c1e89bd5aae9cf8b3799f4b19086ca ./out/35b-attached.json +128f2bd68ac653c01a9bd2b3799662e240c0faa1fe78425951c1aa5956e398f6 ./out/35b-attached.log +c5059267ac760b9fe9743f27cef65d4f618cd4d0a71bf9fb9d56bcce036cd08d ./out/35b-hipvis0.drafts.json +cc0e2d8a95fb742b9473c5ffabefb07ed00fee5db1b0117f9772ec5cfd72dd4b ./out/35b-hipvis0.json +bacab53b2f114f676a112f2fc488b774c0fd9526cad32d0d9f28ddd8ac68fc34 ./out/35b-hipvis0.log +09a0007d8ce0b59842642ebb8b95f5995e94006d3194b567ac04b6733dab3d11 ./out/35b-manifest.json +f8cd168a1e5cb50eb5f3c3726414d2d1e77e98314b750e1171fdb1b9e7f19c14 ./out/35b-serial.drafts.json +d4df29c3744daa24e796fcd9a68c6d4a1ed650405f99c0bd9f672c09ec6d9c88 ./out/35b-serial.json +6bbcdd0b23fb22f15f9d0db7f53ef6b85331a9d8259064ffb1f911253c7dc892 ./out/35b-serial.log +b6496a2ae0244880cfedab8425b0a9acf98c41f5561bcac74118f3bc2f6e7e01 ./out/35b-serve.log +33f02c5060b8b2e0d70c0c1943cb0c1fc1a595da70f30210c57e19d8337a4748 ./out/35b.drafts.json +be4bb0ae6de1d326b7bdb91866468d8355a4cbcf9f9cd466015aa89335d0f840 ./out/35b.json +46d8cf59fdbbf3bc842b0841da380ff7890ae99fd86b307f8d559820509c20df ./out/35b.log +a32e796950908cca2dcbb4e16ed02ee95d0327033d277398ec9d8f1c01261687 ./prep.log +8c614b0c263fce313a3fe4c64319feb081f04347c520b83c934e689672050af6 ./prep.sh +a47602d883d6f0df599d42eff5d517e2b4c89db48e388cdfa5cd5f6f634b6ed5 ./run.log +75d39b97c2366cd0af91418510e4410e62731a58394b5f3cda7da925c4a30573 ./run.sh diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/binary.sha256 b/docs/receipts/drafter-counts/fresh-2026-10-19/binary.sha256 new file mode 100644 index 0000000000..1c7ead982e --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/binary.sha256 @@ -0,0 +1 @@ +a9839e118380da34166502146e5dc696b6ce2c8c0852c56ca3d1565d3cb978da ~/tf-1001-fresh/zig-out/native-hip/bin/tensorfold-native diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/board-validator.log b/docs/receipts/drafter-counts/fresh-2026-10-19/board-validator.log new file mode 100644 index 0000000000..11c7719df9 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/board-validator.log @@ -0,0 +1,12 @@ +# the board's validator on each fresh receipt, run 2026-10-19T17:52:27Z on the Mac with node v26.9.0 +# protocol.mjs: https://tensorfold.dev/benchmarks/protocol.js fetched 2026-10-19, sha256 0252160a717f4f50e12f9b019beff760ee10e415503ee504d72fc6289fc9cc4b (the same file as on 18 Oct) +out/27b-auto.json rocm REFUSED: Unsupported benchmark runtime. +out/27b-auto.json cuda ACCEPTED +out/35b.json rocm REFUSED: Unsupported benchmark runtime. +out/35b.json cuda ACCEPTED +out/35b-serial.json rocm REFUSED: Unsupported benchmark runtime. +out/35b-serial.json cuda ACCEPTED +out/35b-attached.json rocm REFUSED: Unsupported benchmark runtime. +out/35b-attached.json cuda ACCEPTED +out/35b-hipvis0.json rocm REFUSED: Unsupported benchmark runtime. +out/35b-hipvis0.json cuda ACCEPTED diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/check.mjs b/docs/receipts/drafter-counts/fresh-2026-10-19/check.mjs new file mode 100644 index 0000000000..7dd3a327cc --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/check.mjs @@ -0,0 +1,8 @@ +import {validateReceipt} from './protocol.mjs'; import fs from 'fs'; +for (const f of process.argv.slice(2)) { + const r = JSON.parse(fs.readFileSync(f)); + for (const backend of ['rocm','cuda']) { + const x = structuredClone(r); x.runtime.backend = backend; + try { validateReceipt(x); console.log(f, backend, 'ACCEPTED'); } catch (e) { console.log(f, backend, 'REFUSED:', e.message); } + } +} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/heads.txt b/docs/receipts/drafter-counts/fresh-2026-10-19/heads.txt new file mode 100644 index 0000000000..af3cfd0d8c --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/heads.txt @@ -0,0 +1,5 @@ +merge 7f4db1cf9ef7f4d8fb281d204c62d3fcd8b0cd0c +main 76aafcf2 +benchmark-rocm-host-identity c949cc9887e722740845c2d6b4e285a374b6cffa +hip-refuses-drafter f2c78eebe05bc96999bb543ea688a3310110f7f5 +benchmark-drafter-counts 134a5f4326e3cfff63106438e131a1f4f0ef436c diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/match.py b/docs/receipts/drafter-counts/fresh-2026-10-19/match.py new file mode 100644 index 0000000000..985dc4028e --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/match.py @@ -0,0 +1,117 @@ +"""Matches each run's draft counts file against its receipt, its run log and, when kept, the server's own log lines.""" +import copy +import json +import re +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +RUNS = ("out/27b-auto", "out/35b", "out/35b-serial", "out/35b-attached", "out/35b-hipvis0") +DONE = re.compile(r"\] done (\S+) .* tokens=(\d+) sha=([0-9a-f]+) finish=(\w+) rounds=(\d+) accepted=(\d+)/(\d+)$") +POST = re.compile(r'"POST (\S+) HTTP/1\.1" (\d+)') +SAID = re.compile(r"Draft counts: (\d+) of (\d+) drafted tokens accepted over (\d+) rounds \((\d+) of (\d+) attempts") +ROUTE = {"code": "/v1/completions", "chat": "/v1/chat/completions"} +KEYS = ("rounds", "drafted", "accepted") + + +def served(log): + """Every finished request in a server log: its line, the POST line before it and the counters it printed.""" + out, post = [], None + for n, line in enumerate(log.read_text().splitlines(), 1): + if m := POST.search(line): + post = (n, m[1], m[2]) + elif m := DONE.search(line): + out.append({"line": n, "post_line": post[0], "route": post[1], "http": post[2], "id": m[1], "tokens": int(m[2]), + "sha": m[3], "finish": m[4], "rounds": int(m[5]), "accepted": int(m[6]), "drafted": int(m[7])}) + return out + + +def cells(samples): + """The receipt's samples grouped by cell, in run order: [(fixture, temperature, [index, ...]), ...].""" + out = [] + for i, s in enumerate(samples): + if not out or out[-1][:2] != (s["fixture_id"], s["temperature"]): + out.append((s["fixture_id"], s["temperature"], [])) + out[-1][2].append(i) + return out + + +def check(run, drafts, rec, log_text, serve): + """Problems with one run, plus the lines to print; an empty problem list means every count matches.""" + bad, lines = [], [] + rows, kept = drafts["samples"], rec["samples"] + if drafts["run_id"] != rec["run_id"] or drafts["receipt"] != Path(run).name + ".json" or len(rows) != len(kept): + bad.append(f"{run}: the draft counts file is not this receipt's") + return bad, lines + for d, r in zip(rows, kept): + if any(d[k] != r[k] for k in ("fixture_id", "temperature", "repeat", "status", "completion_tokens")): + bad.append(f"{run}: sample {d['fixture_id']} T={d['temperature']} r{d['repeat']} differs from the receipt's") + sums = {k: sum(d[k] for d in rows) for k in KEYS} + t = drafts["totals"] + if any(t[k] != sums[k] for k in KEYS) or t["attempts"] != len(rows) or t["reported"] != sum(d["rounds"] is not None for d in rows): + bad.append(f"{run}: totals {t} are not the sum of its samples {sums}") + said = SAID.findall(log_text) + if [tuple(map(int, x)) for x in said] != [(t["accepted"], t["drafted"], t["rounds"], t["reported"], t["attempts"])]: + bad.append(f"{run}: the run's printed line {said} is not its totals") + lines.append(f"{run}: drafter_passed={drafts['drafter_passed']} attempts={t['attempts']} reported={t['reported']} " + f"rounds={t['rounds']} drafted={t['drafted']} accepted={t['accepted']} (sum of samples, and the run's line)") + if serve is None: + lines.append(" no server log: a managed run unlinks its server's log as it starts") + return bad, lines + groups, done, at, log = cells(kept), serve, 0, f"{Path(run).parent}/35b-serve.log" + if len(done) != len(kept) + len(groups): + bad.append(f"{run}: {len(done)} finished requests in the server log for {len(kept)} samples and {len(groups)} warm-ups") + return bad, lines + from_log = dict.fromkeys(KEYS, 0) + for fixture, temp, members in groups: + w = done[at] + lines.append(f" warm-up {fixture} T={temp}: {log}:{w['line']} rounds={w['rounds']} accepted={w['accepted']}/{w['drafted']} (not a sample)") + at += 1 + for i in members: + d, r, s = rows[i], kept[i], done[at] + at += 1 + same = all(d[k] == s[k] for k in KEYS) and d["completion_tokens"] == s["tokens"] and r["token_sha"] == s["sha"] + if not same or s["route"] != ROUTE[fixture] or s["http"] != "200": + bad.append(f"{run}: sample {fixture} T={temp} r{d['repeat']} does not match {log}:{s['line']}") + for k in KEYS: + from_log[k] += s[k] + lines.append(f" {fixture} T={temp} r{d['repeat']}: rounds={d['rounds']} drafted={d['drafted']} accepted={d['accepted']} " + f"sha={r['token_sha']} = {log}:{s['line']} ({s['route']} at line {s['post_line']}, finish={s['finish']}, " + f"rounds={s['rounds']} accepted={s['accepted']}/{s['drafted']} sha={s['sha']})") + if from_log != sums: + bad.append(f"{run}: the server log's samples sum to {from_log}, the file to {sums}") + lines.append(f" server log, warm-ups left out: rounds={from_log['rounds']} drafted={from_log['drafted']} accepted={from_log['accepted']}") + return bad, lines + + +def load(run): + """One run's draft counts, receipt, run log and, for an attached run, its server's log.""" + folder = (HERE / run).parent + serve = folder / "35b-serve.log" if run.endswith("attached") else None + return (json.loads((HERE / f"{run}.drafts.json").read_text()), json.loads((HERE / f"{run}.json").read_text()), + (HERE / f"{run}.log").read_text(), served(serve) if serve else None) + + +def main(): + problems = [] + for run in RUNS: + bad, lines = check(run, *load(run)) + problems += bad + print("\n".join(lines)) + # controls: the same check refuses a count off by one and a server log read one request out of step + drafts, rec, text, serve = load("out/35b-attached") + off = copy.deepcopy(drafts) + off["samples"][7]["drafted"] += 1 + off["totals"]["drafted"] += 1 + shifted = serve[1:] + serve[:1] + for name, args in (("one drafted count +1", (off, rec, text, serve)), ("log one request out of step", (drafts, rec, text, shifted))): + bad, _ = check("out/35b-attached", *args) + print(f"control, {name}: {'refused: ' + '; '.join(bad) if bad else 'NOT REFUSED'}") + if not bad: + problems.append(f"control {name} was not refused") + print("\n".join(problems) if problems else "every count matches") + return 1 if problems else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/match.txt b/docs/receipts/drafter-counts/fresh-2026-10-19/match.txt new file mode 100644 index 0000000000..43fc2177e1 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/match.txt @@ -0,0 +1,37 @@ +out/27b-auto: drafter_passed=False attempts=20 reported=20 rounds=5030 drafted=262 accepted=70 (sum of samples, and the run's line) + no server log: a managed run unlinks its server's log as it starts +out/35b: drafter_passed=False attempts=20 reported=20 rounds=2642 drafted=4251 accepted=2466 (sum of samples, and the run's line) + no server log: a managed run unlinks its server's log as it starts +out/35b-serial: drafter_passed=False attempts=20 reported=20 rounds=5100 drafted=0 accepted=0 (sum of samples, and the run's line) + no server log: a managed run unlinks its server's log as it starts +out/35b-attached: drafter_passed=None attempts=20 reported=20 rounds=2483 drafted=4537 accepted=2626 (sum of samples, and the run's line) + warm-up code T=1.0: out/35b-serve.log:8 rounds=106 accepted=150/248 (not a sample) + code T=1.0 r0: rounds=102 drafted=264 accepted=154 sha=5d7effdcf06f = out/35b-serve.log:10 (/v1/completions at line 9, finish=length, rounds=102 accepted=154/264 sha=5d7effdcf06f) + code T=1.0 r1: rounds=89 drafted=249 accepted=166 sha=bd3964ef7930 = out/35b-serve.log:12 (/v1/completions at line 11, finish=length, rounds=89 accepted=166/249 sha=bd3964ef7930) + code T=1.0 r2: rounds=89 drafted=228 accepted=166 sha=bc55cc799f87 = out/35b-serve.log:14 (/v1/completions at line 13, finish=length, rounds=89 accepted=166/228 sha=bc55cc799f87) + code T=1.0 r3: rounds=118 drafted=220 accepted=138 sha=306e0fe05bc2 = out/35b-serve.log:16 (/v1/completions at line 15, finish=length, rounds=118 accepted=138/220 sha=306e0fe05bc2) + code T=1.0 r4: rounds=111 drafted=250 accepted=144 sha=5a89476cf1f5 = out/35b-serve.log:18 (/v1/completions at line 17, finish=length, rounds=111 accepted=144/250 sha=5a89476cf1f5) + warm-up chat T=1.0: out/35b-serve.log:20 rounds=151 accepted=104/162 (not a sample) + chat T=1.0 r0: rounds=143 drafted=183 accepted=112 sha=460edda16244 = out/35b-serve.log:22 (/v1/chat/completions at line 21, finish=length, rounds=143 accepted=112/183 sha=460edda16244) + chat T=1.0 r1: rounds=112 drafted=252 accepted=144 sha=0eba9c9d3070 = out/35b-serve.log:24 (/v1/chat/completions at line 23, finish=length, rounds=112 accepted=144/252 sha=0eba9c9d3070) + chat T=1.0 r2: rounds=169 drafted=150 accepted=86 sha=15d18697d5f1 = out/35b-serve.log:26 (/v1/chat/completions at line 25, finish=length, rounds=169 accepted=86/150 sha=15d18697d5f1) + chat T=1.0 r3: rounds=209 drafted=84 accepted=46 sha=d813ade7fc53 = out/35b-serve.log:28 (/v1/chat/completions at line 27, finish=length, rounds=209 accepted=46/84 sha=d813ade7fc53) + chat T=1.0 r4: rounds=142 drafted=214 accepted=114 sha=7675315607f0 = out/35b-serve.log:30 (/v1/chat/completions at line 29, finish=length, rounds=142 accepted=114/214 sha=7675315607f0) + warm-up code T=0.0: out/35b-serve.log:32 rounds=82 accepted=173/290 (not a sample) + code T=0.0 r0: rounds=82 drafted=290 accepted=173 sha=9f2923ac14d0 = out/35b-serve.log:34 (/v1/completions at line 33, finish=length, rounds=82 accepted=173/290 sha=9f2923ac14d0) + code T=0.0 r1: rounds=82 drafted=290 accepted=173 sha=9f2923ac14d0 = out/35b-serve.log:36 (/v1/completions at line 35, finish=length, rounds=82 accepted=173/290 sha=9f2923ac14d0) + code T=0.0 r2: rounds=82 drafted=290 accepted=173 sha=9f2923ac14d0 = out/35b-serve.log:38 (/v1/completions at line 37, finish=length, rounds=82 accepted=173/290 sha=9f2923ac14d0) + code T=0.0 r3: rounds=83 drafted=283 accepted=172 sha=9f2923ac14d0 = out/35b-serve.log:40 (/v1/completions at line 39, finish=length, rounds=83 accepted=172/283 sha=9f2923ac14d0) + code T=0.0 r4: rounds=82 drafted=290 accepted=173 sha=9f2923ac14d0 = out/35b-serve.log:42 (/v1/completions at line 41, finish=length, rounds=82 accepted=173/290 sha=9f2923ac14d0) + warm-up chat T=0.0: out/35b-serve.log:44 rounds=135 accepted=121/249 (not a sample) + chat T=0.0 r0: rounds=135 drafted=249 accepted=121 sha=a8c28b8297b4 = out/35b-serve.log:46 (/v1/chat/completions at line 45, finish=length, rounds=135 accepted=121/249 sha=a8c28b8297b4) + chat T=0.0 r1: rounds=134 drafted=248 accepted=122 sha=a8c28b8297b4 = out/35b-serve.log:48 (/v1/chat/completions at line 47, finish=length, rounds=134 accepted=122/248 sha=a8c28b8297b4) + chat T=0.0 r2: rounds=191 drafted=133 accepted=65 sha=a8c28b8297b4 = out/35b-serve.log:50 (/v1/chat/completions at line 49, finish=length, rounds=191 accepted=65/133 sha=a8c28b8297b4) + chat T=0.0 r3: rounds=140 drafted=217 accepted=116 sha=a8c28b8297b4 = out/35b-serve.log:52 (/v1/chat/completions at line 51, finish=length, rounds=140 accepted=116/217 sha=a8c28b8297b4) + chat T=0.0 r4: rounds=188 drafted=153 accepted=68 sha=a8c28b8297b4 = out/35b-serve.log:54 (/v1/chat/completions at line 53, finish=length, rounds=188 accepted=68/153 sha=a8c28b8297b4) + server log, warm-ups left out: rounds=2483 drafted=4537 accepted=2626 +out/35b-hipvis0: drafter_passed=False attempts=20 reported=20 rounds=2507 drafted=4500 accepted=2601 (sum of samples, and the run's line) + no server log: a managed run unlinks its server's log as it starts +control, one drafted count +1: refused: out/35b-attached: the run's printed line [('2626', '4537', '2483', '20', '20')] is not its totals; out/35b-attached: sample chat T=1.0 r2 does not match out/35b-serve.log:26; out/35b-attached: the server log's samples sum to {'rounds': 2483, 'drafted': 4537, 'accepted': 2626}, the file to {'rounds': 2483, 'drafted': 4538, 'accepted': 2626} +control, log one request out of step: refused: out/35b-attached: sample code T=1.0 r0 does not match out/35b-serve.log:12; out/35b-attached: sample code T=1.0 r1 does not match out/35b-serve.log:14; out/35b-attached: sample code T=1.0 r2 does not match out/35b-serve.log:16; out/35b-attached: sample code T=1.0 r3 does not match out/35b-serve.log:18; out/35b-attached: sample code T=1.0 r4 does not match out/35b-serve.log:20; out/35b-attached: sample chat T=1.0 r0 does not match out/35b-serve.log:24; out/35b-attached: sample chat T=1.0 r1 does not match out/35b-serve.log:26; out/35b-attached: sample chat T=1.0 r2 does not match out/35b-serve.log:28; out/35b-attached: sample chat T=1.0 r3 does not match out/35b-serve.log:30; out/35b-attached: sample chat T=1.0 r4 does not match out/35b-serve.log:32; out/35b-attached: sample code T=0.0 r2 does not match out/35b-serve.log:40; out/35b-attached: sample code T=0.0 r3 does not match out/35b-serve.log:42; out/35b-attached: sample code T=0.0 r4 does not match out/35b-serve.log:44; out/35b-attached: sample chat T=0.0 r0 does not match out/35b-serve.log:48; out/35b-attached: sample chat T=0.0 r1 does not match out/35b-serve.log:50; out/35b-attached: sample chat T=0.0 r2 does not match out/35b-serve.log:52; out/35b-attached: sample chat T=0.0 r3 does not match out/35b-serve.log:54; out/35b-attached: sample chat T=0.0 r4 does not match out/35b-serve.log:8; out/35b-attached: the server log's samples sum to {'rounds': 2495, 'drafted': 4500, 'accepted': 2614}, the file to {'rounds': 2483, 'drafted': 4537, 'accepted': 2626} +every count matches diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.drafts.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.drafts.json new file mode 100644 index 0000000000..c2fa9de948 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":false,"drafter_repo_id":null,"receipt":"27b-auto.json","run_id":"f19a552e-8f32-4f08-9911-0d193bae553d","samples":[{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":0,"rounds":254,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":1,"rounds":255,"status":"ok","temperature":1.0},{"accepted":21,"completion_tokens":256,"drafted":48,"fixture_id":"code","repeat":2,"rounds":234,"status":"ok","temperature":1.0},{"accepted":6,"completion_tokens":256,"drafted":34,"fixture_id":"code","repeat":3,"rounds":249,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":4,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":0,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":1,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":2,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":3,"rounds":255,"status":"ok","temperature":1.0},{"accepted":2,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":4,"rounds":253,"status":"ok","temperature":1.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":0,"rounds":254,"status":"ok","temperature":0.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":1,"rounds":254,"status":"ok","temperature":0.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":2,"rounds":254,"status":"ok","temperature":0.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":3,"rounds":254,"status":"ok","temperature":0.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":4,"rounds":254,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":0,"rounds":248,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":1,"rounds":248,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":2,"rounds":248,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":3,"rounds":248,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":4,"rounds":248,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":70,"attempts":20,"drafted":262,"reported":20,"rounds":5030}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.json new file mode 100644 index 0000000000..3852843ba6 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.json @@ -0,0 +1 @@ +{"created_at":"2026-10-19T17:40:21.038854Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"14b65a0ee06517060a6bbd979bb1a8ff54e7b304b1a1f01d54344b88b8285e85","drafter_repo_id":null,"family":"qwen3_5","local":false,"quantization":"mlx-affine-4bit-g64","repo_id":"TensorFold/Qwen3.8-27B-MLX-4bit","revision":"70ae7fac63274ff2eac54152031433374cb80f2f","tokenizer_sha256":"49d8f5dc9c3e0a89091f4f189ecfd308c8f0250929b815e125d4700bded4df2c"},"run_id":"f19a552e-8f32-4f08-9911-0d193bae553d","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.89750838,"delivery_tps":12.815674964424867,"end_to_end_seconds":20.140451455,"error_code":null,"fixture_id":"code","prefill_seconds":0.2398604,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":12.815613804393331,"status":"ok","temperature":1.0,"token_sha":"94624c419588","ttft_seconds":0.242137453},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.803075672,"delivery_tps":12.876787637616822,"end_to_end_seconds":20.049404591,"error_code":null,"fixture_id":"code","prefill_seconds":0.243871883,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":12.876715963812506,"status":"ok","temperature":1.0,"token_sha":"4e0a0a5f5689","ttft_seconds":0.245762308},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":18.845417218,"delivery_tps":13.5311411283821,"end_to_end_seconds":19.091583913,"error_code":null,"fixture_id":"code","prefill_seconds":0.244006149,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":13.531072561934861,"status":"ok","temperature":1.0,"token_sha":"ed99f8b35926","ttft_seconds":0.245665832},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.841189008,"delivery_tps":12.852052359220185,"end_to_end_seconds":20.091496741,"error_code":null,"fixture_id":"code","prefill_seconds":0.248246924,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":12.85197129107118,"status":"ok","temperature":1.0,"token_sha":"42b152d2efcc","ttft_seconds":0.249922426},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.796010423,"delivery_tps":12.881383397521766,"end_to_end_seconds":20.049398801,"error_code":null,"fixture_id":"code","prefill_seconds":0.250405139,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":12.880974435257686,"status":"ok","temperature":1.0,"token_sha":"f9ef9076534b","ttft_seconds":0.25244296},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.774424733,"delivery_tps":12.895444668711415,"end_to_end_seconds":20.264386883,"error_code":null,"fixture_id":"chat","prefill_seconds":0.487255634,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":12.895382530397082,"status":"ok","temperature":1.0,"token_sha":"388b0ee93e9f","ttft_seconds":0.488998913},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.786729822,"delivery_tps":12.887425173030696,"end_to_end_seconds":20.268713006,"error_code":null,"fixture_id":"chat","prefill_seconds":0.479335244,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":12.887286996397394,"status":"ok","temperature":1.0,"token_sha":"6a931bb6524d","ttft_seconds":0.481590218},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.800629375,"delivery_tps":12.87837851871312,"end_to_end_seconds":20.304582038,"error_code":null,"fixture_id":"chat","prefill_seconds":0.501563254,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":12.878308147500153,"status":"ok","temperature":1.0,"token_sha":"90107a7ec252","ttft_seconds":0.503379962},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.797460334,"delivery_tps":12.880440000784597,"end_to_end_seconds":20.302045055,"error_code":null,"fixture_id":"chat","prefill_seconds":0.501326202,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":12.880153461930988,"status":"ok","temperature":1.0,"token_sha":"2dab910075e1","ttft_seconds":0.504159636},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.875626939,"delivery_tps":12.829783975248521,"end_to_end_seconds":20.368576777,"error_code":null,"fixture_id":"chat","prefill_seconds":0.490587619,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":12.829734040593934,"status":"ok","temperature":1.0,"token_sha":"b1aba87c5a14","ttft_seconds":0.492577171},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.860841765,"delivery_tps":12.83933495957743,"end_to_end_seconds":20.104993299,"error_code":null,"fixture_id":"code","prefill_seconds":0.241665039,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":12.838982393179226,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.243756768},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.864962213,"delivery_tps":12.836671787531682,"end_to_end_seconds":20.112646487,"error_code":null,"fixture_id":"code","prefill_seconds":0.245775199,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":12.83655974188991,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.247287497},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.85074759,"delivery_tps":12.845863806583216,"end_to_end_seconds":20.104282983,"error_code":null,"fixture_id":"code","prefill_seconds":0.251610088,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":12.845786096710466,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.253098768},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.862830517,"delivery_tps":12.838049430153129,"end_to_end_seconds":20.114730676,"error_code":null,"fixture_id":"code","prefill_seconds":0.24982848,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":12.837987940041856,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.251597489},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.866587238,"delivery_tps":12.835621787734452,"end_to_end_seconds":20.12189766,"error_code":null,"fixture_id":"code","prefill_seconds":0.253455555,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":12.835568738719774,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.254961044},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.421726534,"delivery_tps":13.129625708280503,"end_to_end_seconds":19.917292013,"error_code":null,"fixture_id":"chat","prefill_seconds":0.493597986,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":13.12957588725391,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.495199042},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.420598292,"delivery_tps":13.130388475469527,"end_to_end_seconds":19.925875299,"error_code":null,"fixture_id":"chat","prefill_seconds":0.502515041,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":13.130262582982734,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.504933648},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.424132022,"delivery_tps":13.12799973307348,"end_to_end_seconds":19.938675991,"error_code":null,"fixture_id":"chat","prefill_seconds":0.512046905,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":13.127648557344521,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.514257129},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.426561958,"delivery_tps":13.126357641218606,"end_to_end_seconds":19.934581951,"error_code":null,"fixture_id":"chat","prefill_seconds":0.50544899,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":13.125982560866927,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.507719174},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.421004148,"delivery_tps":13.130114079413355,"end_to_end_seconds":19.933822157,"error_code":null,"fixture_id":"chat","prefill_seconds":0.510370562,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":13.12974072458992,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.512311906}],"schema_version":1,"settings":{"context_tokens":null,"managed":true,"rank_count":1,"repetitions":5,"serial":false,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.log b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.log new file mode 100644 index 0000000000..3e80b54974 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-auto.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +This run passes no drafter: the HIP engine runs no DFlash2 drafter yet. +code T=1 warm-up: ok 12.83 delivery tok/s +code T=1 repeat 1/5: ok 12.82 delivery tok/s +code T=1 repeat 2/5: ok 12.88 delivery tok/s +code T=1 repeat 3/5: ok 13.53 delivery tok/s +code T=1 repeat 4/5: ok 12.85 delivery tok/s +code T=1 repeat 5/5: ok 12.88 delivery tok/s +chat T=1 warm-up: ok 12.90 delivery tok/s +chat T=1 repeat 1/5: ok 12.90 delivery tok/s +chat T=1 repeat 2/5: ok 12.89 delivery tok/s +chat T=1 repeat 3/5: ok 12.88 delivery tok/s +chat T=1 repeat 4/5: ok 12.88 delivery tok/s +chat T=1 repeat 5/5: ok 12.83 delivery tok/s +code T=0 warm-up: ok 12.84 delivery tok/s +code T=0 repeat 1/5: ok 12.84 delivery tok/s +code T=0 repeat 2/5: ok 12.84 delivery tok/s +code T=0 repeat 3/5: ok 12.85 delivery tok/s +code T=0 repeat 4/5: ok 12.84 delivery tok/s +code T=0 repeat 5/5: ok 12.84 delivery tok/s +chat T=0 warm-up: ok 13.13 delivery tok/s +chat T=0 repeat 1/5: ok 13.13 delivery tok/s +chat T=0 repeat 2/5: ok 13.13 delivery tok/s +chat T=0 repeat 3/5: ok 13.13 delivery tok/s +chat T=0 repeat 4/5: ok 13.13 delivery tok/s +chat T=0 repeat 5/5: ok 13.13 delivery tok/s +Run f19a552e-8f32-4f08-9911-0d193bae553d | rocm | TensorFold/Qwen3.8-27B-MLX-4bit +Workload Temp Stream delivery tok/s First text, s Status +code 1 12.88 0.246 complete +chat 1 12.88 0.493 complete +code 0 12.84 0.252 complete +chat 0 13.13 0.508 complete +Stream delivery includes transport buffering; it is not an engine token clock. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-fresh-runs/out/27b-auto.json +Draft counts: 70 of 262 drafted tokens accepted over 5030 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-fresh-runs/out/27b-auto.drafts.json diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-dflash2.log b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-dflash2.log new file mode 100644 index 0000000000..83a51751ba --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-dflash2.log @@ -0,0 +1 @@ +tensorfold benchmark: --drafter: the HIP engine runs no DFlash2 drafter yet, so a rocm run takes none; --drafter none runs without it diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-serve-drafter.log b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-serve-drafter.log new file mode 100644 index 0000000000..40d7c5404e --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/27b-serve-drafter.log @@ -0,0 +1 @@ +tensorfold: --drafter: the HIP engine runs no DFlash2 drafter yet; serve without --drafter diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.drafts.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.drafts.json new file mode 100644 index 0000000000..3fa549cdad --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":null,"drafter_repo_id":null,"receipt":"35b-attached.json","run_id":"42e6a803-dac9-4cea-af11-00fc3816cfa6","samples":[{"accepted":154,"completion_tokens":256,"drafted":264,"fixture_id":"code","repeat":0,"rounds":102,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":249,"fixture_id":"code","repeat":1,"rounds":89,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":228,"fixture_id":"code","repeat":2,"rounds":89,"status":"ok","temperature":1.0},{"accepted":138,"completion_tokens":256,"drafted":220,"fixture_id":"code","repeat":3,"rounds":118,"status":"ok","temperature":1.0},{"accepted":144,"completion_tokens":256,"drafted":250,"fixture_id":"code","repeat":4,"rounds":111,"status":"ok","temperature":1.0},{"accepted":112,"completion_tokens":256,"drafted":183,"fixture_id":"chat","repeat":0,"rounds":143,"status":"ok","temperature":1.0},{"accepted":144,"completion_tokens":256,"drafted":252,"fixture_id":"chat","repeat":1,"rounds":112,"status":"ok","temperature":1.0},{"accepted":86,"completion_tokens":256,"drafted":150,"fixture_id":"chat","repeat":2,"rounds":169,"status":"ok","temperature":1.0},{"accepted":46,"completion_tokens":256,"drafted":84,"fixture_id":"chat","repeat":3,"rounds":209,"status":"ok","temperature":1.0},{"accepted":114,"completion_tokens":256,"drafted":214,"fixture_id":"chat","repeat":4,"rounds":142,"status":"ok","temperature":1.0},{"accepted":173,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":0,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":1,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":2,"rounds":82,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":283,"fixture_id":"code","repeat":3,"rounds":83,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":4,"rounds":82,"status":"ok","temperature":0.0},{"accepted":121,"completion_tokens":256,"drafted":249,"fixture_id":"chat","repeat":0,"rounds":135,"status":"ok","temperature":0.0},{"accepted":122,"completion_tokens":256,"drafted":248,"fixture_id":"chat","repeat":1,"rounds":134,"status":"ok","temperature":0.0},{"accepted":65,"completion_tokens":256,"drafted":133,"fixture_id":"chat","repeat":2,"rounds":191,"status":"ok","temperature":0.0},{"accepted":116,"completion_tokens":256,"drafted":217,"fixture_id":"chat","repeat":3,"rounds":140,"status":"ok","temperature":0.0},{"accepted":68,"completion_tokens":256,"drafted":153,"fixture_id":"chat","repeat":4,"rounds":188,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":2626,"attempts":20,"drafted":4537,"reported":20,"rounds":2483}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.json new file mode 100644 index 0000000000..c47cdf47fe --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.json @@ -0,0 +1 @@ +{"created_at":"2026-10-19T17:45:34.967656Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767","drafter_repo_id":null,"family":null,"local":false,"quantization":null,"repo_id":"TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP","revision":"f84b054c5677b6e59bdb969ae61915b0983d68bb","tokenizer_sha256":"4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"},"run_id":"42e6a803-dac9-4cea-af11-00fc3816cfa6","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.284300724,"delivery_tps":77.64209840365397,"end_to_end_seconds":3.344023479,"error_code":null,"fixture_id":"code","prefill_seconds":0.058373383,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":77.64125142556341,"status":"ok","temperature":1.0,"token_sha":"5d7effdcf06f","ttft_seconds":0.059330159},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.95133518,"delivery_tps":86.40157232158226,"end_to_end_seconds":3.010594111,"error_code":null,"fixture_id":"code","prefill_seconds":0.057809141,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":86.40036332226946,"status":"ok","temperature":1.0,"token_sha":"bd3964ef7930","ttft_seconds":0.058926263},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.876120514,"delivery_tps":88.661097043307,"end_to_end_seconds":2.93610881,"error_code":null,"fixture_id":"code","prefill_seconds":0.057713624,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":88.64525338530609,"status":"ok","temperature":1.0,"token_sha":"bc55cc799f87","ttft_seconds":0.059441865},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.244931601,"delivery_tps":78.58409093165966,"end_to_end_seconds":3.307593006,"error_code":null,"fixture_id":"code","prefill_seconds":0.060020306,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":78.5728470534191,"status":"ok","temperature":1.0,"token_sha":"306e0fe05bc2","ttft_seconds":0.061817784},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.218675729,"delivery_tps":79.22512904996029,"end_to_end_seconds":3.283851428,"error_code":null,"fixture_id":"code","prefill_seconds":0.062579488,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":79.20763502335677,"status":"ok","temperature":1.0,"token_sha":"5a89476cf1f5","ttft_seconds":0.064619008},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.450480266,"delivery_tps":73.90275565772501,"end_to_end_seconds":3.545273832,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092204855,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":73.88669655309172,"status":"ok","temperature":1.0,"token_sha":"460edda16244","ttft_seconds":0.094441528},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.450736058,"delivery_tps":73.89727748340003,"end_to_end_seconds":3.547180166,"error_code":null,"fixture_id":"chat","prefill_seconds":0.094710639,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":73.89650509771157,"status":"ok","temperature":1.0,"token_sha":"0eba9c9d3070","ttft_seconds":0.096057132},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.621930388,"delivery_tps":70.40444533248163,"end_to_end_seconds":3.716103375,"error_code":null,"fixture_id":"chat","prefill_seconds":0.09261055,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":70.40360737349702,"status":"ok","temperature":1.0,"token_sha":"15d18697d5f1","ttft_seconds":0.093878727},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.734825354,"delivery_tps":68.27628492103248,"end_to_end_seconds":3.829400417,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092453875,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":68.27507532627075,"status":"ok","temperature":1.0,"token_sha":"d813ade7fc53","ttft_seconds":0.094311442},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.645377596,"delivery_tps":69.95160124970495,"end_to_end_seconds":3.740589566,"error_code":null,"fixture_id":"chat","prefill_seconds":0.093207969,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":69.95097033620063,"status":"ok","temperature":1.0,"token_sha":"7675315607f0","ttft_seconds":0.094466586},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.481081134,"delivery_tps":102.77777558563307,"end_to_end_seconds":2.536695431,"error_code":null,"fixture_id":"code","prefill_seconds":0.054029481,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":102.77644425789181,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055109234},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.493128282,"delivery_tps":102.28113885717815,"end_to_end_seconds":2.548462068,"error_code":null,"fixture_id":"code","prefill_seconds":0.052336828,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":102.25239170529593,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.054824013},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.489102379,"delivery_tps":102.44656955510466,"end_to_end_seconds":2.544001411,"error_code":null,"fixture_id":"code","prefill_seconds":0.052921809,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":102.42739786072137,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.054676009},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.467143671,"delivery_tps":103.35839091877516,"end_to_end_seconds":2.522855605,"error_code":null,"fixture_id":"code","prefill_seconds":0.054619141,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":103.35586150782943,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055438753},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.492440395,"delivery_tps":102.30936736202271,"end_to_end_seconds":2.547608468,"error_code":null,"fixture_id":"code","prefill_seconds":0.053426791,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":102.30787615571732,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.054897022},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.251372651,"delivery_tps":78.42841389515027,"end_to_end_seconds":3.341033342,"error_code":null,"fixture_id":"chat","prefill_seconds":0.087553343,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":78.41871564168736,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.089336572},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.222785999,"delivery_tps":79.12408707221766,"end_to_end_seconds":3.3121829,"error_code":null,"fixture_id":"chat","prefill_seconds":0.08766037,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":79.12306382988778,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.089084731},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.461596375,"delivery_tps":73.6654342030272,"end_to_end_seconds":3.551191899,"error_code":null,"fixture_id":"chat","prefill_seconds":0.087883462,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":73.66301429727004,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.089270515},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.193999344,"delivery_tps":79.83721113751112,"end_to_end_seconds":3.283853479,"error_code":null,"fixture_id":"chat","prefill_seconds":0.088077276,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":79.83616759310115,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.089366162},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.486220533,"delivery_tps":73.14511448320931,"end_to_end_seconds":3.576741624,"error_code":null,"fixture_id":"chat","prefill_seconds":0.088329547,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":73.14472257734447,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.090223522}],"schema_version":1,"settings":{"context_tokens":null,"managed":false,"rank_count":null,"repetitions":5,"serial":false,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.log b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.log new file mode 100644 index 0000000000..d7d705bd9d --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-attached.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +code T=1 warm-up: ok 79.67 delivery tok/s +code T=1 repeat 1/5: ok 77.64 delivery tok/s +code T=1 repeat 2/5: ok 86.40 delivery tok/s +code T=1 repeat 3/5: ok 88.66 delivery tok/s +code T=1 repeat 4/5: ok 78.58 delivery tok/s +code T=1 repeat 5/5: ok 79.23 delivery tok/s +chat T=1 warm-up: ok 74.71 delivery tok/s +chat T=1 repeat 1/5: ok 73.90 delivery tok/s +chat T=1 repeat 2/5: ok 73.90 delivery tok/s +chat T=1 repeat 3/5: ok 70.40 delivery tok/s +chat T=1 repeat 4/5: ok 68.28 delivery tok/s +chat T=1 repeat 5/5: ok 69.95 delivery tok/s +code T=0 warm-up: ok 102.38 delivery tok/s +code T=0 repeat 1/5: ok 102.78 delivery tok/s +code T=0 repeat 2/5: ok 102.28 delivery tok/s +code T=0 repeat 3/5: ok 102.45 delivery tok/s +code T=0 repeat 4/5: ok 103.36 delivery tok/s +code T=0 repeat 5/5: ok 102.31 delivery tok/s +chat T=0 warm-up: ok 78.34 delivery tok/s +chat T=0 repeat 1/5: ok 78.43 delivery tok/s +chat T=0 repeat 2/5: ok 79.12 delivery tok/s +chat T=0 repeat 3/5: ok 73.67 delivery tok/s +chat T=0 repeat 4/5: ok 79.84 delivery tok/s +chat T=0 repeat 5/5: ok 73.15 delivery tok/s +Run 42e6a803-dac9-4cea-af11-00fc3816cfa6 | rocm | TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +Workload Temp Stream delivery tok/s First text, s Status +code 1 79.23 0.059 complete +chat 1 70.40 0.094 complete +code 0 102.45 0.055 complete +chat 0 78.43 0.089 complete +Stream delivery includes transport buffering; it is not an engine token clock. +The attached server's manifest does not attest a rank count and checkpoint revision; this result is unranked. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-fresh-runs/out/35b-attached.json +Draft counts: 2626 of 4537 drafted tokens accepted over 2483 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-fresh-runs/out/35b-attached.drafts.json diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.drafts.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.drafts.json new file mode 100644 index 0000000000..c0248c30e1 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":false,"drafter_repo_id":null,"receipt":"35b-hipvis0.json","run_id":"82de0a73-9eb4-4e80-bf4e-20edebba83fb","samples":[{"accepted":150,"completion_tokens":256,"drafted":246,"fixture_id":"code","repeat":0,"rounds":106,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":249,"fixture_id":"code","repeat":1,"rounds":89,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":228,"fixture_id":"code","repeat":2,"rounds":89,"status":"ok","temperature":1.0},{"accepted":152,"completion_tokens":256,"drafted":248,"fixture_id":"code","repeat":3,"rounds":103,"status":"ok","temperature":1.0},{"accepted":136,"completion_tokens":256,"drafted":219,"fixture_id":"code","repeat":4,"rounds":119,"status":"ok","temperature":1.0},{"accepted":79,"completion_tokens":256,"drafted":142,"fixture_id":"chat","repeat":0,"rounds":176,"status":"ok","temperature":1.0},{"accepted":125,"completion_tokens":256,"drafted":217,"fixture_id":"chat","repeat":1,"rounds":131,"status":"ok","temperature":1.0},{"accepted":97,"completion_tokens":256,"drafted":168,"fixture_id":"chat","repeat":2,"rounds":158,"status":"ok","temperature":1.0},{"accepted":55,"completion_tokens":256,"drafted":102,"fixture_id":"chat","repeat":3,"rounds":201,"status":"ok","temperature":1.0},{"accepted":63,"completion_tokens":256,"drafted":119,"fixture_id":"chat","repeat":4,"rounds":193,"status":"ok","temperature":1.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":0,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":1,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":2,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":3,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":4,"rounds":82,"status":"ok","temperature":0.0},{"accepted":124,"completion_tokens":256,"drafted":249,"fixture_id":"chat","repeat":0,"rounds":132,"status":"ok","temperature":0.0},{"accepted":121,"completion_tokens":256,"drafted":246,"fixture_id":"chat","repeat":1,"rounds":135,"status":"ok","temperature":0.0},{"accepted":62,"completion_tokens":256,"drafted":125,"fixture_id":"chat","repeat":2,"rounds":193,"status":"ok","temperature":0.0},{"accepted":124,"completion_tokens":256,"drafted":259,"fixture_id":"chat","repeat":3,"rounds":132,"status":"ok","temperature":0.0},{"accepted":116,"completion_tokens":256,"drafted":221,"fixture_id":"chat","repeat":4,"rounds":140,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":2601,"attempts":20,"drafted":4500,"reported":20,"rounds":2507}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.json new file mode 100644 index 0000000000..19f0f27dff --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.json @@ -0,0 +1 @@ +{"created_at":"2026-10-19T17:47:28.550736Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767","drafter_repo_id":null,"family":"qwen3_5_moe","local":false,"quantization":"mlx-affine-4bit-g64","repo_id":"TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP","revision":"f84b054c5677b6e59bdb969ae61915b0983d68bb","tokenizer_sha256":"4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"},"run_id":"82de0a73-9eb4-4e80-bf4e-20edebba83fb","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.232439129,"delivery_tps":78.88779643590313,"end_to_end_seconds":3.29403629,"error_code":null,"fixture_id":"code","prefill_seconds":0.059049949,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":78.87618344692764,"status":"ok","temperature":1.0,"token_sha":"5d7effdcf06f","ttft_seconds":0.061226164},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.949364607,"delivery_tps":86.4593002149632,"end_to_end_seconds":3.009208369,"error_code":null,"fixture_id":"code","prefill_seconds":0.058052283,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":86.45744502476046,"status":"ok","temperature":1.0,"token_sha":"bd3964ef7930","ttft_seconds":0.05930653},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.87241447,"delivery_tps":88.77548928375926,"end_to_end_seconds":2.933181311,"error_code":null,"fixture_id":"code","prefill_seconds":0.058370552,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":88.7634184281852,"status":"ok","temperature":1.0,"token_sha":"bc55cc799f87","ttft_seconds":0.060465361},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.138903059,"delivery_tps":81.23857131199158,"end_to_end_seconds":3.198283558,"error_code":null,"fixture_id":"code","prefill_seconds":0.057626238,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":81.23721182129704,"status":"ok","temperature":1.0,"token_sha":"306e0fe05bc2","ttft_seconds":0.058890105},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.180728629,"delivery_tps":80.17030993309552,"end_to_end_seconds":3.241191809,"error_code":null,"fixture_id":"code","prefill_seconds":0.058028414,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":80.1580663755954,"status":"ok","temperature":1.0,"token_sha":"5a89476cf1f5","ttft_seconds":0.060157601},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.604996868,"delivery_tps":70.73515160679467,"end_to_end_seconds":3.700973513,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092721467,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":70.71847909783814,"status":"ok","temperature":1.0,"token_sha":"460edda16244","ttft_seconds":0.095357406},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.506907825,"delivery_tps":72.71363056141917,"end_to_end_seconds":3.601496848,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092177595,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":72.70431197462008,"status":"ok","temperature":1.0,"token_sha":"0eba9c9d3070","ttft_seconds":0.094180607},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.599445227,"delivery_tps":70.84425068819084,"end_to_end_seconds":3.696474657,"error_code":null,"fixture_id":"chat","prefill_seconds":0.094829534,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":70.84206257099673,"status":"ok","temperature":1.0,"token_sha":"15d18697d5f1","ttft_seconds":0.096371072},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.722626112,"delivery_tps":68.50002990576992,"end_to_end_seconds":3.816290195,"error_code":null,"fixture_id":"chat","prefill_seconds":0.090857501,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":68.49213597844425,"status":"ok","temperature":1.0,"token_sha":"d813ade7fc53","ttft_seconds":0.093369214},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.689554194,"delivery_tps":69.11404104449373,"end_to_end_seconds":3.784605611,"error_code":null,"fixture_id":"chat","prefill_seconds":0.093055165,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":69.11304022669324,"status":"ok","temperature":1.0,"token_sha":"7675315607f0","ttft_seconds":0.094459317},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.48704853,"delivery_tps":102.53117175803561,"end_to_end_seconds":2.543184178,"error_code":null,"fixture_id":"code","prefill_seconds":0.054192106,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":102.52897824962238,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055576658},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.493243178,"delivery_tps":102.27642544059935,"end_to_end_seconds":2.550184179,"error_code":null,"fixture_id":"code","prefill_seconds":0.054003572,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":102.2505299129703,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.056604493},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.488365155,"delivery_tps":102.47692123787195,"end_to_end_seconds":2.544108607,"error_code":null,"fixture_id":"code","prefill_seconds":0.053959083,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":102.47356592697062,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055369835},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.4683499,"delivery_tps":103.30788191738942,"end_to_end_seconds":2.524507908,"error_code":null,"fixture_id":"code","prefill_seconds":0.053601146,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":103.27973184508747,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055905197},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.489240055,"delivery_tps":102.4409033945101,"end_to_end_seconds":2.545964794,"error_code":null,"fixture_id":"code","prefill_seconds":0.055169281,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":102.43710601514013,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.056232046},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.190850052,"delivery_tps":79.916008538279,"end_to_end_seconds":3.283481692,"error_code":null,"fixture_id":"chat","prefill_seconds":0.09057847,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":79.91423792019906,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.092081559},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.226451374,"delivery_tps":79.03419901347009,"end_to_end_seconds":3.316923688,"error_code":null,"fixture_id":"chat","prefill_seconds":0.087466026,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":79.02258084130723,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.089475888},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.460237633,"delivery_tps":73.6943606323699,"end_to_end_seconds":3.550712626,"error_code":null,"fixture_id":"chat","prefill_seconds":0.087828044,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":73.68313726792776,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.089787197},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.232184078,"delivery_tps":78.89402145616287,"end_to_end_seconds":3.322856774,"error_code":null,"fixture_id":"chat","prefill_seconds":0.088327197,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":78.88045708427254,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.090330628},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.195963957,"delivery_tps":79.7881338559789,"end_to_end_seconds":3.285087646,"error_code":null,"fixture_id":"chat","prefill_seconds":0.086858987,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":79.7769089236042,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.088775691}],"schema_version":1,"settings":{"context_tokens":null,"managed":true,"rank_count":1,"repetitions":5,"serial":false,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.log b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.log new file mode 100644 index 0000000000..6a46de0c53 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-hipvis0.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +The server gets up to 606 s to load 21 GB of weights; --startup-timeout sets another limit. +code T=1 warm-up: ok 78.63 delivery tok/s +code T=1 repeat 1/5: ok 78.89 delivery tok/s +code T=1 repeat 2/5: ok 86.46 delivery tok/s +code T=1 repeat 3/5: ok 88.78 delivery tok/s +code T=1 repeat 4/5: ok 81.24 delivery tok/s +code T=1 repeat 5/5: ok 80.17 delivery tok/s +chat T=1 warm-up: ok 74.43 delivery tok/s +chat T=1 repeat 1/5: ok 70.74 delivery tok/s +chat T=1 repeat 2/5: ok 72.71 delivery tok/s +chat T=1 repeat 3/5: ok 70.84 delivery tok/s +chat T=1 repeat 4/5: ok 68.50 delivery tok/s +chat T=1 repeat 5/5: ok 69.11 delivery tok/s +code T=0 warm-up: ok 102.19 delivery tok/s +code T=0 repeat 1/5: ok 102.53 delivery tok/s +code T=0 repeat 2/5: ok 102.28 delivery tok/s +code T=0 repeat 3/5: ok 102.48 delivery tok/s +code T=0 repeat 4/5: ok 103.31 delivery tok/s +code T=0 repeat 5/5: ok 102.44 delivery tok/s +chat T=0 warm-up: ok 77.08 delivery tok/s +chat T=0 repeat 1/5: ok 79.92 delivery tok/s +chat T=0 repeat 2/5: ok 79.03 delivery tok/s +chat T=0 repeat 3/5: ok 73.69 delivery tok/s +chat T=0 repeat 4/5: ok 78.89 delivery tok/s +chat T=0 repeat 5/5: ok 79.79 delivery tok/s +Run 82de0a73-9eb4-4e80-bf4e-20edebba83fb | rocm | TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +Workload Temp Stream delivery tok/s First text, s Status +code 1 81.24 0.060 complete +chat 1 70.74 0.094 complete +code 0 102.48 0.056 complete +chat 0 79.03 0.090 complete +Stream delivery includes transport buffering; it is not an engine token clock. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-fresh-runs/out/35b-hipvis0.json +Draft counts: 2601 of 4500 drafted tokens accepted over 2507 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-fresh-runs/out/35b-hipvis0.drafts.json diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-manifest.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-manifest.json new file mode 100644 index 0000000000..f945169c43 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-manifest.json @@ -0,0 +1 @@ +{"schema_version": 1, "runtime": {"tensorfold_version": "1.0.12", "backend": "rocm", "python_version": "none", "platform": "linux"}, "model": {"revision": "f84b054c5677b6e59bdb969ae61915b0983d68bb", "config_sha256": "a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767", "tokenizer_sha256": "4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"}, "hardware": {"cpu_model": "AMD RYZEN AI MAX+ 395 with Radeon 8060S", "system_memory_bytes": 131209781248, "memory_type": "unified", "gpus": [{"name": "AMD Radeon 8060S Graphics", "memory_bytes": 128849018880, "cores": 40}], "gpu_count": 1, "source": "detected"}} \ No newline at end of file diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.drafts.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.drafts.json new file mode 100644 index 0000000000..26d8f9f8a1 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":false,"drafter_repo_id":null,"receipt":"35b-serial.json","run_id":"ee440afd-6373-455f-ba6c-725a24fd261d","samples":[{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":0,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":1,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":2,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":3,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":4,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":0,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":1,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":2,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":3,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":4,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":0,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":1,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":2,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":3,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":4,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":0,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":1,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":2,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":3,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":4,"rounds":255,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":0,"attempts":20,"drafted":0,"reported":20,"rounds":5100}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.json new file mode 100644 index 0000000000..5d7b0b7f77 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.json @@ -0,0 +1 @@ +{"created_at":"2026-10-19T17:43:59.490762Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767","drafter_repo_id":null,"family":"qwen3_5_moe","local":false,"quantization":"mlx-affine-4bit-g64","repo_id":"TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP","revision":"f84b054c5677b6e59bdb969ae61915b0983d68bb","tokenizer_sha256":"4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"},"run_id":"ee440afd-6373-455f-ba6c-725a24fd261d","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.761255099,"delivery_tps":67.79651826003413,"end_to_end_seconds":3.81529262,"error_code":null,"fixture_id":"code","prefill_seconds":0.052487014,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":67.79561704143363,"status":"ok","temperature":1.0,"token_sha":"5d7effdcf06f","ttft_seconds":0.053725981},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.751063878,"delivery_tps":67.98071381710551,"end_to_end_seconds":3.805867833,"error_code":null,"fixture_id":"code","prefill_seconds":0.05172664,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":67.96678196935372,"status":"ok","temperature":1.0,"token_sha":"bd3964ef7930","ttft_seconds":0.053836268},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.749646707,"delivery_tps":68.00640698334463,"end_to_end_seconds":3.801737674,"error_code":null,"fixture_id":"code","prefill_seconds":0.050163983,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":68.00278902105372,"status":"ok","temperature":1.0,"token_sha":"bc55cc799f87","ttft_seconds":0.051759418},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.751563821,"delivery_tps":67.9716545331297,"end_to_end_seconds":3.804884156,"error_code":null,"fixture_id":"code","prefill_seconds":0.051841446,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":67.97064811720776,"status":"ok","temperature":1.0,"token_sha":"306e0fe05bc2","ttft_seconds":0.053004126},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.770719356,"delivery_tps":67.62635346866689,"end_to_end_seconds":3.824614041,"error_code":null,"fixture_id":"code","prefill_seconds":0.05170718,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":67.61792123958934,"status":"ok","temperature":1.0,"token_sha":"5a89476cf1f5","ttft_seconds":0.053404622},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.77964075,"delivery_tps":67.46672947686893,"end_to_end_seconds":3.865819641,"error_code":null,"fixture_id":"chat","prefill_seconds":0.083699595,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":67.45723063172042,"status":"ok","temperature":1.0,"token_sha":"460edda16244","ttft_seconds":0.085648349},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.763464104,"delivery_tps":67.75672437767457,"end_to_end_seconds":3.849848608,"error_code":null,"fixture_id":"chat","prefill_seconds":0.083731734,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":67.74709080608707,"status":"ok","temperature":1.0,"token_sha":"0eba9c9d3070","ttft_seconds":0.086067725},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.764869646,"delivery_tps":67.73142870190092,"end_to_end_seconds":3.852359022,"error_code":null,"fixture_id":"chat","prefill_seconds":0.085692647,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":67.73113170005004,"status":"ok","temperature":1.0,"token_sha":"15d18697d5f1","ttft_seconds":0.087232564},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.766450482,"delivery_tps":67.70300080106031,"end_to_end_seconds":3.854368393,"error_code":null,"fixture_id":"chat","prefill_seconds":0.085696427,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":67.69439743367194,"status":"ok","temperature":1.0,"token_sha":"d813ade7fc53","ttft_seconds":0.08765796},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.76911238,"delivery_tps":67.65518623246781,"end_to_end_seconds":3.856320356,"error_code":null,"fixture_id":"chat","prefill_seconds":0.085366649,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":67.65341360074623,"status":"ok","temperature":1.0,"token_sha":"7675315607f0","ttft_seconds":0.086877027},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.684567834,"delivery_tps":69.2075737205711,"end_to_end_seconds":3.737302959,"error_code":null,"fixture_id":"code","prefill_seconds":0.049754077,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":69.19853416761354,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.052239212},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.682578912,"delivery_tps":69.24495200063754,"end_to_end_seconds":3.734453887,"error_code":null,"fixture_id":"code","prefill_seconds":0.049911652,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":69.23560397492463,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.051621604},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.681552247,"delivery_tps":69.26426216218792,"end_to_end_seconds":3.732986827,"error_code":null,"fixture_id":"code","prefill_seconds":0.050201282,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":69.26344284645636,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.051171289},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.681836518,"delivery_tps":69.2589143361851,"end_to_end_seconds":3.734208175,"error_code":null,"fixture_id":"code","prefill_seconds":0.051099701,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":69.25830921139529,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.052114636},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.684126129,"delivery_tps":69.21587130058869,"end_to_end_seconds":3.735860619,"error_code":null,"fixture_id":"code","prefill_seconds":0.049531575,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":69.20675386713492,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.051477929},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.692146365,"delivery_tps":69.06551766671362,"end_to_end_seconds":3.780397064,"error_code":null,"fixture_id":"chat","prefill_seconds":0.086876286,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":69.06466170664525,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.087970539},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.692492403,"delivery_tps":69.05904526515013,"end_to_end_seconds":3.779813244,"error_code":null,"fixture_id":"chat","prefill_seconds":0.084374073,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":69.04889985248946,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.08647842},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.694593611,"delivery_tps":69.0197696549852,"end_to_end_seconds":3.781390701,"error_code":null,"fixture_id":"chat","prefill_seconds":0.084018504,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":69.00949290849817,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.085983097},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.695981223,"delivery_tps":68.99385700694076,"end_to_end_seconds":3.783112171,"error_code":null,"fixture_id":"chat","prefill_seconds":0.084822747,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":68.98419969252876,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.086659684},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.692950577,"delivery_tps":69.05047730347678,"end_to_end_seconds":3.780905348,"error_code":null,"fixture_id":"chat","prefill_seconds":0.08503577,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":69.03996990758351,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.086981184}],"schema_version":1,"settings":{"context_tokens":null,"managed":true,"rank_count":1,"repetitions":5,"serial":true,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.log b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.log new file mode 100644 index 0000000000..41b685a2b4 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serial.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +The server gets up to 606 s to load 21 GB of weights; --startup-timeout sets another limit. +code T=1 warm-up: ok 67.76 delivery tok/s +code T=1 repeat 1/5: ok 67.80 delivery tok/s +code T=1 repeat 2/5: ok 67.98 delivery tok/s +code T=1 repeat 3/5: ok 68.01 delivery tok/s +code T=1 repeat 4/5: ok 67.97 delivery tok/s +code T=1 repeat 5/5: ok 67.63 delivery tok/s +chat T=1 warm-up: ok 67.67 delivery tok/s +chat T=1 repeat 1/5: ok 67.47 delivery tok/s +chat T=1 repeat 2/5: ok 67.76 delivery tok/s +chat T=1 repeat 3/5: ok 67.73 delivery tok/s +chat T=1 repeat 4/5: ok 67.70 delivery tok/s +chat T=1 repeat 5/5: ok 67.66 delivery tok/s +code T=0 warm-up: ok 69.23 delivery tok/s +code T=0 repeat 1/5: ok 69.21 delivery tok/s +code T=0 repeat 2/5: ok 69.24 delivery tok/s +code T=0 repeat 3/5: ok 69.26 delivery tok/s +code T=0 repeat 4/5: ok 69.26 delivery tok/s +code T=0 repeat 5/5: ok 69.22 delivery tok/s +chat T=0 warm-up: ok 69.04 delivery tok/s +chat T=0 repeat 1/5: ok 69.07 delivery tok/s +chat T=0 repeat 2/5: ok 69.06 delivery tok/s +chat T=0 repeat 3/5: ok 69.02 delivery tok/s +chat T=0 repeat 4/5: ok 68.99 delivery tok/s +chat T=0 repeat 5/5: ok 69.05 delivery tok/s +Run ee440afd-6373-455f-ba6c-725a24fd261d | rocm | TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +Workload Temp Stream delivery tok/s First text, s Status +code 1 67.97 0.053 complete +chat 1 67.70 0.087 complete +code 0 69.24 0.052 complete +chat 0 69.05 0.087 complete +Stream delivery includes transport buffering; it is not an engine token clock. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-fresh-runs/out/35b-serial.json +Draft counts: 0 of 0 drafted tokens accepted over 5100 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-fresh-runs/out/35b-serial.drafts.json diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serve.log b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serve.log new file mode 100644 index 0000000000..98ae6d8ad1 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b-serve.log @@ -0,0 +1,59 @@ +[tensorfold] HIP rank 0 of 1: policy matrix=auto,attention=auto,kernels=auto,graphs=auto,prefill_step=1024,fill_rows=512; mtp drafts 3 under 0.3, 8 snapshots +[tensorfold] open-file limit raised: soft 1024 -> 524288 +[tensorfold] HIP gfx1151 device 0, rank 0 of 1: model 18.71 GiB; 8 streams at once, 0.69 GiB each at a 32768-token window, of 83.9 GiB left after a 12.2 GiB reserve; prompt cache 76.70 GiB (62434 pages, 8 snapshots); prompts in 1024-row chunks +[tensorfold] serving amd-35b at http://127.0.0.1:8092/v1 (sampling: temperature 1.0, top_k 20, top_p 0.95; drafts: on; context: 32768; loaded in 18.0s) +[tensorfold] 127.0.0.1 "GET /v1/benchmark/manifest HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/benchmark/manifest HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-16634e4518da3a4abc6252a1522d549c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5d7effdcf06f finish=length rounds=106 accepted=150/248 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-1ceccdf4c9075b3c19c86536a19f7a14 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5d7effdcf06f finish=length rounds=102 accepted=154/264 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-e4f93e3b7da6ac45dbea01731b0fa3fd prompt=14 cached=0 thinking=False effort=none tokens=256 sha=bd3964ef7930 finish=length rounds=89 accepted=166/249 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-420ec0866a33ff19510446d53a50fc80 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=bc55cc799f87 finish=length rounds=89 accepted=166/228 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-5eb9bdcaee56cec905d5a56a81af71c9 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=306e0fe05bc2 finish=length rounds=118 accepted=138/220 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-714716bb0a4d6f82f0ffbce18f9a838c prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5a89476cf1f5 finish=length rounds=111 accepted=144/250 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-f123297701353ed897b038a9babfbcfe prompt=31 cached=0 thinking=False effort=none tokens=256 sha=460edda16244 finish=length rounds=151 accepted=104/162 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b9c2e2ed2a37b1166a99e347bda95ec9 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=460edda16244 finish=length rounds=143 accepted=112/183 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-ac6f4727b299b289a3c16e78fd1f5741 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=0eba9c9d3070 finish=length rounds=112 accepted=144/252 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-2304227cc0f771862dfec3d7393c2df3 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=15d18697d5f1 finish=length rounds=169 accepted=86/150 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-dde506579ad229551ef9f20405a6d569 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=d813ade7fc53 finish=length rounds=209 accepted=46/84 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a3db1531e8b245a9bf39ccfd09ab6bd6 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=7675315607f0 finish=length rounds=142 accepted=114/214 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-8dda737d69b71f046b4b2a8ab2d3b047 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=82 accepted=173/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-1606a3f441422dd9499684932046a43f prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=82 accepted=173/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-7b40e4d74d6381618d3eb145de9a44fd prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=82 accepted=173/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-b9bfc681e93a8674690fff3582fd29a6 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=82 accepted=173/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-bd77266f9e3b20c22b61820468f7190b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/283 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-e053fca02b9e53ae19625b44cdce153a prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=82 accepted=173/290 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-61ebca87a04291690d6e27f2cf02cdf9 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=135 accepted=121/249 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d8ddbc7251946efb50edd9537920916c prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=135 accepted=121/249 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-7fa6ae652589bcfcb0255dd24919bcd3 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=134 accepted=122/248 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9059a37394cd02ca394543382f458656 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=191 accepted=65/133 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-01a2ca6db29b7b2750d637fcdbd5da61 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=140 accepted=116/217 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-315f9bd60100f00f5216e45b6e1c5377 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=188 accepted=68/153 +info: backend verify: 2957 calls, 21658 us each +info: backend keep: 1313 calls, 11 us each +info: backend draft: 2981 calls, 3631 us each +info: head graphs: 4 captured, 1171 of 1175 batches replayed +info: graphs: 17 captured (3 ms each), 3108 of 3125 rounds replayed diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.drafts.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.drafts.json new file mode 100644 index 0000000000..5300e74d20 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":false,"drafter_repo_id":null,"receipt":"35b.json","run_id":"b8652dff-88fe-42bd-b73a-cbb5f94813ab","samples":[{"accepted":154,"completion_tokens":256,"drafted":264,"fixture_id":"code","repeat":0,"rounds":102,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":249,"fixture_id":"code","repeat":1,"rounds":89,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":228,"fixture_id":"code","repeat":2,"rounds":89,"status":"ok","temperature":1.0},{"accepted":138,"completion_tokens":256,"drafted":220,"fixture_id":"code","repeat":3,"rounds":118,"status":"ok","temperature":1.0},{"accepted":137,"completion_tokens":256,"drafted":221,"fixture_id":"code","repeat":4,"rounds":118,"status":"ok","temperature":1.0},{"accepted":114,"completion_tokens":256,"drafted":211,"fixture_id":"chat","repeat":0,"rounds":141,"status":"ok","temperature":1.0},{"accepted":105,"completion_tokens":256,"drafted":166,"fixture_id":"chat","repeat":1,"rounds":151,"status":"ok","temperature":1.0},{"accepted":34,"completion_tokens":256,"drafted":74,"fixture_id":"chat","repeat":2,"rounds":221,"status":"ok","temperature":1.0},{"accepted":118,"completion_tokens":256,"drafted":209,"fixture_id":"chat","repeat":3,"rounds":137,"status":"ok","temperature":1.0},{"accepted":28,"completion_tokens":256,"drafted":67,"fixture_id":"chat","repeat":4,"rounds":227,"status":"ok","temperature":1.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":0,"rounds":83,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":1,"rounds":83,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":2,"rounds":82,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":3,"rounds":83,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":4,"rounds":83,"status":"ok","temperature":0.0},{"accepted":92,"completion_tokens":256,"drafted":173,"fixture_id":"chat","repeat":0,"rounds":164,"status":"ok","temperature":0.0},{"accepted":108,"completion_tokens":256,"drafted":229,"fixture_id":"chat","repeat":1,"rounds":148,"status":"ok","temperature":0.0},{"accepted":65,"completion_tokens":256,"drafted":133,"fixture_id":"chat","repeat":2,"rounds":191,"status":"ok","temperature":0.0},{"accepted":88,"completion_tokens":256,"drafted":184,"fixture_id":"chat","repeat":3,"rounds":168,"status":"ok","temperature":0.0},{"accepted":92,"completion_tokens":256,"drafted":173,"fixture_id":"chat","repeat":4,"rounds":164,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":2466,"attempts":20,"drafted":4251,"reported":20,"rounds":2642}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.json b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.json new file mode 100644 index 0000000000..9249a7c0b2 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.json @@ -0,0 +1 @@ +{"created_at":"2026-10-19T17:42:15.909275Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767","drafter_repo_id":null,"family":"qwen3_5_moe","local":false,"quantization":"mlx-affine-4bit-g64","repo_id":"TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP","revision":"f84b054c5677b6e59bdb969ae61915b0983d68bb","tokenizer_sha256":"4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"},"run_id":"b8652dff-88fe-42bd-b73a-cbb5f94813ab","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.269828599,"delivery_tps":77.9857390928643,"end_to_end_seconds":3.33056039,"error_code":null,"fixture_id":"code","prefill_seconds":0.0587277,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":77.9749077865912,"status":"ok","temperature":1.0,"token_sha":"5d7effdcf06f","ttft_seconds":0.060243008},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.944776014,"delivery_tps":86.59402235948802,"end_to_end_seconds":3.005612722,"error_code":null,"fixture_id":"code","prefill_seconds":0.057962766,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":86.57932824482447,"status":"ok","temperature":1.0,"token_sha":"bd3964ef7930","ttft_seconds":0.059864651},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.873688966,"delivery_tps":88.7361168926171,"end_to_end_seconds":2.93467434,"error_code":null,"fixture_id":"code","prefill_seconds":0.058623804,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":88.7216625571413,"status":"ok","temperature":1.0,"token_sha":"bc55cc799f87","ttft_seconds":0.060378274},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.238959975,"delivery_tps":78.72897534030194,"end_to_end_seconds":3.299033799,"error_code":null,"fixture_id":"code","prefill_seconds":0.057624457,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":78.71854261060689,"status":"ok","temperature":1.0,"token_sha":"306e0fe05bc2","ttft_seconds":0.059697137},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.170860966,"delivery_tps":80.41979851348677,"end_to_end_seconds":3.229906225,"error_code":null,"fixture_id":"code","prefill_seconds":0.057407095,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":80.41888018648855,"status":"ok","temperature":1.0,"token_sha":"5a89476cf1f5","ttft_seconds":0.0587252},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.592170876,"delivery_tps":70.98771433834207,"end_to_end_seconds":3.686965153,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092375999,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":70.97511035505887,"status":"ok","temperature":1.0,"token_sha":"460edda16244","ttft_seconds":0.094480827},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.443445686,"delivery_tps":74.05373084197385,"end_to_end_seconds":3.536983295,"error_code":null,"fixture_id":"chat","prefill_seconds":0.091274956,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":74.04333160542039,"status":"ok","temperature":1.0,"token_sha":"0eba9c9d3070","ttft_seconds":0.09320464},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.798190106,"delivery_tps":67.13723981250348,"end_to_end_seconds":3.893436876,"error_code":null,"fixture_id":"chat","prefill_seconds":0.093459171,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":67.13601952882865,"status":"ok","temperature":1.0,"token_sha":"15d18697d5f1","ttft_seconds":0.094882792},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.57923338,"delivery_tps":71.24430651124516,"end_to_end_seconds":3.67420012,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092779285,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":71.24310353871125,"status":"ok","temperature":1.0,"token_sha":"d813ade7fc53","ttft_seconds":0.094438228},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.799162292,"delivery_tps":67.12005973973801,"end_to_end_seconds":3.894189769,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092504613,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":67.11158973213689,"status":"ok","temperature":1.0,"token_sha":"7675315607f0","ttft_seconds":0.094449667},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.482445838,"delivery_tps":102.7212743563592,"end_to_end_seconds":2.538304436,"error_code":null,"fixture_id":"code","prefill_seconds":0.054364169,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":102.71927876584095,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055566328},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.48701947,"delivery_tps":102.53236980086851,"end_to_end_seconds":2.541835385,"error_code":null,"fixture_id":"code","prefill_seconds":0.05259266,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":102.51361267709866,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.054519015},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.469374705,"delivery_tps":103.2650085398846,"end_to_end_seconds":2.524788739,"error_code":null,"fixture_id":"code","prefill_seconds":0.053958963,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":103.26281237404177,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055115604},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.477137599,"delivery_tps":102.94139498061851,"end_to_end_seconds":2.533387824,"error_code":null,"fixture_id":"code","prefill_seconds":0.054100248,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":102.9236365081771,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055745712},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.491217267,"delivery_tps":102.35959881053601,"end_to_end_seconds":2.547101395,"error_code":null,"fixture_id":"code","prefill_seconds":0.05377999,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":102.33636579850301,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055594208},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.288562848,"delivery_tps":77.54147078413992,"end_to_end_seconds":3.378787739,"error_code":null,"fixture_id":"chat","prefill_seconds":0.08796408,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":77.53102512513817,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.089878433},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.294400858,"delivery_tps":77.40405949105056,"end_to_end_seconds":3.384737636,"error_code":null,"fixture_id":"chat","prefill_seconds":0.08822377,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":77.40258051570775,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.0896999},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.451451313,"delivery_tps":73.88196352054409,"end_to_end_seconds":3.543147444,"error_code":null,"fixture_id":"chat","prefill_seconds":0.089431789,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":73.88050992803105,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.091124821},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.357607084,"delivery_tps":75.946944839124,"end_to_end_seconds":3.448013191,"error_code":null,"fixture_id":"chat","prefill_seconds":0.088079905,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":75.93596779977631,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.090068818},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.294830023,"delivery_tps":77.39397729774785,"end_to_end_seconds":3.385796789,"error_code":null,"fixture_id":"chat","prefill_seconds":0.089235586,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":77.39273970168342,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.090660877}],"schema_version":1,"settings":{"context_tokens":null,"managed":true,"rank_count":1,"repetitions":5,"serial":false,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.log b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.log new file mode 100644 index 0000000000..85d38ddcbb --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/out/35b.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +The server gets up to 606 s to load 21 GB of weights; --startup-timeout sets another limit. +code T=1 warm-up: ok 79.44 delivery tok/s +code T=1 repeat 1/5: ok 77.99 delivery tok/s +code T=1 repeat 2/5: ok 86.59 delivery tok/s +code T=1 repeat 3/5: ok 88.74 delivery tok/s +code T=1 repeat 4/5: ok 78.73 delivery tok/s +code T=1 repeat 5/5: ok 80.42 delivery tok/s +chat T=1 warm-up: ok 70.67 delivery tok/s +chat T=1 repeat 1/5: ok 70.99 delivery tok/s +chat T=1 repeat 2/5: ok 74.05 delivery tok/s +chat T=1 repeat 3/5: ok 67.14 delivery tok/s +chat T=1 repeat 4/5: ok 71.24 delivery tok/s +chat T=1 repeat 5/5: ok 67.12 delivery tok/s +code T=0 warm-up: ok 102.32 delivery tok/s +code T=0 repeat 1/5: ok 102.72 delivery tok/s +code T=0 repeat 2/5: ok 102.53 delivery tok/s +code T=0 repeat 3/5: ok 103.27 delivery tok/s +code T=0 repeat 4/5: ok 102.94 delivery tok/s +code T=0 repeat 5/5: ok 102.36 delivery tok/s +chat T=0 warm-up: ok 73.52 delivery tok/s +chat T=0 repeat 1/5: ok 77.54 delivery tok/s +chat T=0 repeat 2/5: ok 77.40 delivery tok/s +chat T=0 repeat 3/5: ok 73.88 delivery tok/s +chat T=0 repeat 4/5: ok 75.95 delivery tok/s +chat T=0 repeat 5/5: ok 77.39 delivery tok/s +Run b8652dff-88fe-42bd-b73a-cbb5f94813ab | rocm | TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +Workload Temp Stream delivery tok/s First text, s Status +code 1 80.42 0.060 complete +chat 1 70.99 0.094 complete +code 0 102.72 0.056 complete +chat 0 77.39 0.090 complete +Stream delivery includes transport buffering; it is not an engine token clock. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-fresh-runs/out/35b.json +Draft counts: 2466 of 4251 drafted tokens accepted over 2642 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-fresh-runs/out/35b.drafts.json diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/prep.log b/docs/receipts/drafter-counts/fresh-2026-10-19/prep.log new file mode 100644 index 0000000000..bd1641a3c2 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/prep.log @@ -0,0 +1,21 @@ +Fast-forwarding to: fork/benchmark-rocm-host-identity +Trying simple merge with fork/hip-refuses-drafter +Simple merge did not work, trying automatic merge. +Auto-merging docs/community-benchmarks.md +Auto-merging zig/src/server/benchmark.zig +Trying simple merge with fork/benchmark-drafter-counts +Simple merge did not work, trying automatic merge. +Auto-merging docs/community-benchmarks.md +Auto-merging zig/src/server/benchmark.zig +Auto-merging zig/src/server/benchmark_run_test.zig +Auto-merging zig/src/server/root.zig +merge 7f4db1cf9ef7f4d8fb281d204c62d3fcd8b0cd0c +main 76aafcf2 +benchmark-rocm-host-identity c949cc9887e722740845c2d6b4e285a374b6cffa +hip-refuses-drafter f2c78eebe05bc96999bb543ea688a3310110f7f5 +benchmark-drafter-counts 134a5f4326e3cfff63106438e131a1f4f0ef436c +/dev/mapper/ubuntu--vg-ubuntu--lv 1.9T 1.7T 75G 96% / +Mem: 122 96 1 0 25 25 +build_exit=0 +a9839e118380da34166502146e5dc696b6ce2c8c0852c56ca3d1565d3cb978da ~/tf-1001-fresh/zig-out/native-hip/bin/tensorfold-native +PREPDONE diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/prep.sh b/docs/receipts/drafter-counts/fresh-2026-10-19/prep.sh new file mode 100644 index 0000000000..a513e758f7 --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/prep.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# #1001 fresh gfx1151 run, step 1 (no service touched): merge the three PR heads onto main 76aafcf2 and build. +# bash ~/tf-1001-fresh-runs/prep.sh > ~/tf-1001-fresh-runs/prep.log 2>&1 +set -u +R=$HOME/tf-1001-fresh-runs; mkdir -p $R +Z=$HOME/zig/zig-x86_64-linux-0.17.0/zig +F=https://github.com/ThinkOffApp/TensorFold.git +cd $HOME/tf-1001 || exit 1 +git fetch -q $F benchmark-rocm-host-identity:refs/remotes/fork/benchmark-rocm-host-identity \ + hip-refuses-drafter:refs/remotes/fork/hip-refuses-drafter \ + benchmark-drafter-counts:refs/remotes/fork/benchmark-drafter-counts || exit 1 +rm -rf $HOME/tf-1001-fresh; git worktree prune +git worktree add -q -B tf1001-fresh $HOME/tf-1001-fresh 76aafcf2 || exit 1 +cd $HOME/tf-1001-fresh || exit 1 +git -c user.name=ThinkOffApp -c user.email=233579950+ThinkOffApp@users.noreply.github.com \ + merge -q --no-edit fork/benchmark-rocm-host-identity fork/hip-refuses-drafter fork/benchmark-drafter-counts || exit 1 +{ echo "merge $(git rev-parse HEAD)"; echo "main 76aafcf2"; for b in benchmark-rocm-host-identity hip-refuses-drafter benchmark-drafter-counts; do echo "$b $(git rev-parse fork/$b)"; done; } | tee $R/heads.txt +df -h / | tail -1; free -g | sed -n 2p +$Z build native-hip -Dhip-arch=gfx1151 -Dhipcc=/usr/bin/hipcc -Dhip-include=/usr/include \ + -Dhip-device-lib=/usr/lib/llvm-21/lib/clang/21/amdgcn/bitcode -Doptimize=ReleaseFast -j8 > $R/build.log 2>&1 +echo "build_exit=$?" +sha256sum $HOME/tf-1001-fresh/zig-out/native-hip/bin/tensorfold-native | tee $R/binary.sha256 +echo PREPDONE diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/run.log b/docs/receipts/drafter-counts/fresh-2026-10-19/run.log new file mode 100644 index 0000000000..63846d479f --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/run.log @@ -0,0 +1,32 @@ +merge 7f4db1cf9ef7f4d8fb281d204c62d3fcd8b0cd0c +main 76aafcf2 +benchmark-rocm-host-identity c949cc9887e722740845c2d6b4e285a374b6cffa +hip-refuses-drafter f2c78eebe05bc96999bb543ea688a3310110f7f5 +benchmark-drafter-counts 134a5f4326e3cfff63106438e131a1f4f0ef436c +/dev/mapper/ubuntu--vg-ubuntu--lv 1.9T 1.7T 75G 96% / +Mem: 122 96 3 0 24 26 +no *_VISIBLE_DEVICES set +llm-server: inactive +gtt_used=1682292736 busy=1 t=17:31:37 +gtt_used=1682292736 busy=1 t=17:31:37 +R1 exit=0 +gtt_used=1682337792 busy=81 t=17:40:21 +R2a exit=1 +ls: cannot access '~/tf-1001-fresh-runs/out/27b-dflash2.json': No such file or directory +gtt_used=1682292736 busy=72 t=17:40:21 +R2b exit=1 +gtt_used=1682292736 busy=53 t=17:40:21 +gtt_used=1682292736 busy=52 t=17:40:21 +R3 exit=0 +gtt_used=1682292736 busy=82 t=17:42:16 +R4 exit=0 +gtt_used=1682292736 busy=81 t=17:43:59 +R5 exit=0 +gtt_used=1682337792 busy=41 t=17:45:35 +gtt_used=1682292736 busy=41 t=17:45:35 +R6 exit=0 +gtt_used=1682292736 busy=74 t=17:47:28 +RUNDONE +llm-server: active t=17:47:28 +llm-server chat OK after 190s t=17:50:29 +LLMBACK diff --git a/docs/receipts/drafter-counts/fresh-2026-10-19/run.sh b/docs/receipts/drafter-counts/fresh-2026-10-19/run.sh new file mode 100644 index 0000000000..6fb9c94b2e --- /dev/null +++ b/docs/receipts/drafter-counts/fresh-2026-10-19/run.sh @@ -0,0 +1,48 @@ +#!/bin/bash +# #1001 fresh gfx1151 run, step 2. Stops llm-server, runs R1-R6 with the binary prep.sh built, +# and ALWAYS restarts llm-server (EXIT trap), then checks it answers a chat request. +# bash ~/tf-1001-fresh-runs/run.sh > ~/tf-1001-fresh-runs/run.log 2>&1 +set -u +R=$HOME/tf-1001-fresh-runs; D=$R/out; mkdir -p $D +B=$HOME/tf-1001-fresh/zig-out/native-hip/bin/tensorfold-native +M27=TensorFold/Qwen3.8-27B-MLX-4bit +M35=TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +gtt() { echo "gtt_used=$(cat /sys/class/drm/card*/device/mem_info_gtt_used | head -1) busy=$(cat /sys/class/drm/card*/device/gpu_busy_percent | head -1) t=$(date -u +%T)"; } +restart_llm() { + for f in $R/serve-*.pid; do [ -f "$f" ] && kill "$(cat "$f")" 2>/dev/null; rm -f "$f"; done + sudo -n systemctl start llm-server.service + echo "llm-server: $(systemctl is-active llm-server) t=$(date -u +%T)" + for i in $(seq 1 60); do + out=$(curl -sS -m 60 http://127.0.0.1:8080/v1/chat/completions -H "Authorization: Bearer $(cat $HOME/.llm-api-key 2>/dev/null)" \ + -H 'Content-Type: application/json' -d '{"messages":[{"role":"user","content":"Say ok."}],"max_tokens":8}' 2>&1) + if echo "$out" | grep -q '"choices"'; then echo "llm-server chat OK after ${i}0s t=$(date -u +%T)"; echo LLMBACK; return; fi + sleep 10 + done + echo "llm-server chat check FAILED: $out" | cut -c1-300; echo LLMFAILED +} +[ "$(sha256sum $B | cut -d' ' -f1)" = "$(cut -d' ' -f1 $R/binary.sha256)" ] || { echo "binary changed since prep"; exit 1; } +cat $R/heads.txt; df -h / | tail -1; free -g | sed -n 2p +env | grep -E '^(HIP|CUDA|ROCR)_VISIBLE_DEVICES=' || echo "no *_VISIBLE_DEVICES set" +trap restart_llm EXIT +sudo -n systemctl stop llm-server.service || exit 1 +echo "llm-server: $(systemctl is-active llm-server)"; gtt +export HF_HUB_OFFLINE=1 +# R1: 27B, default drafting (copy drafts; no MTP head at 70ae7fac) +gtt; timeout 1200 $B benchmark $M27 --output $D/27b-auto.json > $D/27b-auto.log 2>&1; echo "R1 exit=$?"; gtt +# R2a/R2b: --drafter refused before any weights load +timeout 300 $B benchmark $M27 --drafter z-lab/Qwen3.8-27B-DFlash2 --output $D/27b-dflash2.json > $D/27b-dflash2.log 2>&1; echo "R2a exit=$?"; ls $D/27b-dflash2.json 2>&1 +gtt; timeout 180 $B serve $M27 --backend rocm --host 127.0.0.1 --port 8093 --drafter z-lab/Qwen3.8-27B-DFlash2 > $D/27b-serve-drafter.log 2>&1; echo "R2b exit=$?"; gtt +# R3: 35B-MTP, MTP drafting +gtt; timeout 1200 $B benchmark $M35 --output $D/35b.json > $D/35b.log 2>&1; echo "R3 exit=$?"; gtt +# R4: 35B-MTP --serial (plain decode) +timeout 1200 $B benchmark $M35 --serial --output $D/35b-serial.json > $D/35b-serial.log 2>&1; echo "R4 exit=$?"; gtt +# R5: attached to a 35B serve: manifest, receipt and per-request counts against the server log +$B serve $M35 --backend rocm --host 127.0.0.1 --port 8092 --name amd-35b --no-thinking > $D/35b-serve.log 2>&1 & +echo $! > $R/serve-35b.pid +for i in $(seq 1 120); do grep -q "serving" $D/35b-serve.log && break; sleep 2; done +curl -sS http://127.0.0.1:8092/v1/benchmark/manifest > $D/35b-manifest.json +timeout 1200 $B benchmark $M35 --server http://127.0.0.1:8092 --model-id amd-35b --backend rocm --output $D/35b-attached.json > $D/35b-attached.log 2>&1; echo "R5 exit=$?" +kill "$(cat $R/serve-35b.pid)"; wait "$(cat $R/serve-35b.pid)" 2>/dev/null; rm -f $R/serve-35b.pid; gtt +# R6: the visible-device fixes on hardware: 35B-MTP with HIP_VISIBLE_DEVICES=0 set +gtt; HIP_VISIBLE_DEVICES=0 timeout 1200 $B benchmark $M35 --output $D/35b-hipvis0.json > $D/35b-hipvis0.log 2>&1; echo "R6 exit=$?"; gtt +echo RUNDONE diff --git a/docs/receipts/drafter-counts/heads.txt b/docs/receipts/drafter-counts/heads.txt new file mode 100644 index 0000000000..b255214dc4 --- /dev/null +++ b/docs/receipts/drafter-counts/heads.txt @@ -0,0 +1,2 @@ +7b84f9098f2bf3a56adee3d87cbbae16ef7a6e48 +origin/main diff --git a/docs/receipts/drafter-counts/host-controls/commands.txt b/docs/receipts/drafter-counts/host-controls/commands.txt new file mode 100644 index 0000000000..96f00f6d3b --- /dev/null +++ b/docs/receipts/drafter-counts/host-controls/commands.txt @@ -0,0 +1,9 @@ +# Host controls for the draft counts' overflow fix: CPU only, a MacBook (Apple silicon, macOS), 19 October 2026. +# Zig 0.17.0. Every test run is the repository's host test step, from the repository root, with a private cache: +# zig build test -Dcpu=apple_m1 --summary all -j8 --cache-dir /.zc + +host-tests.log the tree of 616d3065: the fix, its two regression tests and the doc line (first line: the SHA) +negative-control.log the same tree with zig/src/server/benchmark_drafts.zig as d2dfb603 has it (unchecked sums) + and the new run test kept; the file was restored with git checkout afterwards +lean.txt tools/zig/lean_check.py and tools/zig/lean_new.py, as the no-gpu workflow runs them +fmt.txt zig fmt --check build.zig zig, as the zig-fmt workflow runs it diff --git a/docs/receipts/drafter-counts/host-controls/fmt.txt b/docs/receipts/drafter-counts/host-controls/fmt.txt new file mode 100644 index 0000000000..031dcf85ce --- /dev/null +++ b/docs/receipts/drafter-counts/host-controls/fmt.txt @@ -0,0 +1,2 @@ +$ zig fmt --check build.zig zig (Zig 0.17.0, on 616d3065) +exit=0 diff --git a/docs/receipts/drafter-counts/host-controls/host-tests-full.log b/docs/receipts/drafter-counts/host-controls/host-tests-full.log new file mode 100644 index 0000000000..6b951618d1 --- /dev/null +++ b/docs/receipts/drafter-counts/host-controls/host-tests-full.log @@ -0,0 +1,892 @@ +test ++- run test w +[default] (warn): a round could not be recorded: OutOfDeviceMemory +failed command: ./.zc/o/04abf0f0f1389fb3893820bf8ff1a3a8/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +failed command: ./.zc/o/61f4f42007c910ae0dba3d28a8174637/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): glm: restoring 1500 tokens failed (PoolMoveNotDrained); pool relocation aborted +failed command: ./.zc/o/f0019315a0e01380b7828cae19930837/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zc/o/7dc46cd228ec88e14e59e92768e8c88f/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): language_model.model.layers.0: gate and up of one expert differ in width; one kernel reads both +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.self_attn.v_proj.weight: another spelling of module layers.0.self_attn.v_proj declares a different width +[default] (warn): language_model.mtp.fc_embedding: not read at its declared width: UnsupportedQuantization +[default] (warn): language_model.model.embed_tokens: not stored at its declared width: InvalidAffineShape +[default] (warn): pack ~/tmp/rv1001-3/.zig-cache/tmp/NecpVWjxKmCjRMQP/pack.safetensors has no recorded source; skipping the identity check +[default] (warn): pack pack.safetensors has no recorded source; skipping the identity check +[default] (warn): lane L0.att.o: LaneRowsNotFours +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +failed command: ./.zc/o/996cdba356502970a6b9b0791b5650e3/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (err): rank 0: another rank's process ended (its line closed); this rank stops instead of waiting on it +[default] (err): rank 0: rank 1 stopped (peer OOM); this rank stops too +failed command: ./.zc/o/8f77ddc20dd6e733b8c011896e0c22e0/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zc/o/8cf2b35f0eb989fd4422afbb63b4c501/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 0 cannot exchange with rank 2; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 1 cannot exchange with rank 3; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 2 cannot exchange with rank 0; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +REFUSE TF_GLM_PREFILL_REDUCE=split: rank 3 cannot exchange with rank 1; split sends every rank's rows to every other, and a fabric where some ranks reach only their neighbours (a ring of four without a switch) needs TF_GLM_PREFILL_REDUCE=ring or gather +failed command: ./.zc/o/2d07c8f61ec9bafcd37e5ab2f870292f/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): /opt/homebrew/opt/jpeg-turbo/lib/libturbojpeg.0.dylib is not libjpeg-turbo 3.1.4.1; JPEG images are refused until it is +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +libjpeg-turbo 3.1.4.1 not found: LFM2's JPEG cases skipped +failed command: ./.zc/o/0196e793c52d44d64df1fbbbcbb1d03f/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lane host: stopping at a round boundary: 1 running and 2 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (the planner refused): every admitted stream fails +[default] (warn): lane host: a round failed (PlannerFull): every admitted stream fails +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (drain deadline) +failed command: ./.zc/o/df0d0ad69192958173e491eb5fa760b7/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): /opt/homebrew/opt/jpeg-turbo/lib/libturbojpeg.0.dylib is not libjpeg-turbo 3.1.4.1; JPEG images are refused until it is +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zc/o/3ae86fb0bdd206be6e3a0818e3ff55f0/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zc/o/4e1b9a76e32557579e3b1e47dccd71c9/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +[default] (warn): qwen3_5: a kept state's host copy goes to the copy-source quarantine: a restore's copies did not drain +[default] (warn): qwen3_5: a failed restore's stream did not drain: its kept state's host copy stays +failed command: ./.zc/o/b224b01689fb39f9ece5438996601159/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zc/o/a529abb901551c852fa65e556ac33565/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 1.000 of its tolerance (element 1) +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +RESULT CPU reference comparison: largest difference 2.000 of its tolerance (element 0) +FAIL CPU reference comparison within its float64 tolerance +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 2 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 1 (got -nan, reference 0e0, tolerance 1e0) +FAIL CPU reference comparison: non-finite error at element 0 (got inf, reference 0e0, tolerance 1e0) +failed command: ./.zc/o/85ebacbf8efe752485249fa934a3542c/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +RESULT valid stale prefix: largest difference 0.000 of its tolerance (element 0) +FAIL no-op projection: non-finite error at element 0 (got -nan, reference 0e0, tolerance 1e0) +failed command: ./.zc/o/c1a07be78dc63d83093aada1c312f26f/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zc/o/eaba94c481a066499f7aa14a4611fabd/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +fake target: proposed=68 verified=68 accepted=68 +failed command: ./.zc/o/70410f3c13e52ac191ae12b4564b69df/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a delete failed (UploadFailed), and its put-back did not reach the device (PutBackNotOnDevice) +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): slide pocket: a delete failed (UploadFailed), and its put-back did not reach the device (PutBackNotOnDevice) +[default] (warn): slide pocket: a deleted fact's 2 layers are put back in memory, but not all on the device (UploadFailed): replies are refused until they are +[default] (warn): checkpoint tensor text.weight stays in its file: read it through a Stage +[default] (warn): language_model.mtp.fc_embedding: not read at its declared width: UnsupportedQuantization +[default] (warn): language_model.model.embed_tokens: not stored at its declared width: InvalidAffineShape +[default] (warn): lane L0.att.o: LaneRowsNotFours +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): pack ~/tmp/rv1001-3/.zig-cache/tmp/A9fb9805W8LG9VU-/pack.safetensors has no recorded source; skipping the identity check +[default] (warn): pack pack.safetensors has no recorded source; skipping the identity check +[default] (warn): model.layers.0.self_attn.v_proj.weight: another spelling of module layers.0.self_attn.v_proj declares a different width +SSM pipeline capacity: forward=1024, backward=1024 +SSM seeded=false heads=2 groups=1 rows=1: max abs dx/dB/dC/ddt { 0, 0, 0, 0 } +SSM seeded=false heads=2 groups=1 rows=7: max abs dx/dB/dC/ddt { 0.000000011360835217022469, 0.000000002917455535836333, 0.000000004647133651849988, 0.00000008929183081818337 } +SSM seeded=false heads=2 groups=1 rows=8: max abs dx/dB/dC/ddt { 0.00000000863441644745333, 0.0000000030472456129596037, 0.000000004647133651849988, 0.00000010044378595530645 } +SSM seeded=false heads=2 groups=1 rows=9: max abs dx/dB/dC/ddt { 0.000000006604320090808358, 0.0000000027633473599630864, 0.0000000051661870337227356, 0.00000006044416009376441 } +SSM seeded=false heads=2 groups=1 rows=15: max abs dx/dB/dC/ddt { 0.000000010416435020088599, 0.0000000050579541700790376, 0.000000008099387559939997, 0.00000010302938735495815 } +SSM seeded=false heads=2 groups=1 rows=16: max abs dx/dB/dC/ddt { 0.000000010399334079633227, 0.000000004927133326337874, 0.000000008099387559939997, 0.00000010076810685877291 } +SSM seeded=false heads=2 groups=1 rows=17: max abs dx/dB/dC/ddt { 0.00000001543697608283323, 0.000000004476264019170895, 0.000000008099387559939997, 0.000000170725301229524 } +SSM seeded=false heads=2 groups=1 rows=25: max abs dx/dB/dC/ddt { 0.00000001223729547902841, 0.000000005462381803822414, 0.000000014270247941650105, 0.00000013346487892640368 } +SSM seeded=false heads=4 groups=2 rows=1: max abs dx/dB/dC/ddt { 0, 0, 0, 0 } +SSM seeded=false heads=4 groups=2 rows=7: max abs dx/dB/dC/ddt { 0.000000015510263740026176, 0.000000006877330351040545, 0.000000003777481768357793, 0.00000016323549628660672 } +SSM seeded=false heads=4 groups=2 rows=8: max abs dx/dB/dC/ddt { 0.0000000129148233773968, 0.000000005653858301674086, 0.000000005027532001822266, 0.00000017245745953520242 } +SSM seeded=false heads=4 groups=2 rows=9: max abs dx/dB/dC/ddt { 0.00000002633087964332148, 0.000000006475598009203498, 0.000000005457366610700509, 0.00000029433054438143813 } +SSM seeded=false heads=4 groups=2 rows=15: max abs dx/dB/dC/ddt { 0.00000002973828677754753, 0.000000009558005810428316, 0.000000008944659693477242, 0.0000003865198301422623 } +SSM seeded=false heads=4 groups=2 rows=16: max abs dx/dB/dC/ddt { 0.000000025754882060358142, 0.000000008921690271973537, 0.000000008944659693477242, 0.0000003074925702950537 } +SSM seeded=false heads=4 groups=2 rows=17: max abs dx/dB/dC/ddt { 0.00000002553487910179897, 0.000000008921690271973537, 0.00000001164840428280467, 0.00000030358411029496324 } +SSM seeded=false heads=4 groups=2 rows=25: max abs dx/dB/dC/ddt { 0.00000003875727619395697, 0.000000014591112693085329, 0.000000022576584234046404, 0.0000004419375038144935 } +SSM seeded=false heads=64 groups=8 rows=1: max abs dx/dB/dC/ddt { 0, 0, 0, 0 } +SSM seeded=false heads=64 groups=8 rows=9: max abs dx/dB/dC/ddt { 0.00000002087263310801557, 0.00000001779203438051491, 0.000000010395453171885283, 0.0000002193045710030539 } +SSM seeded=false heads=64 groups=8 rows=17: max abs dx/dB/dC/ddt { 0.00000009930843825056712, 0.00000002114473045267129, 0.00000001602812140649723, 0.0000007270377315737875 } +SSM pipeline capacity: forward=1024, backward=1024 +SSM seeded=true heads=2 groups=1 rows=1: max abs dx/dB/dC/ddt { 0, 0, 0.000000015089179933314156, 0.00000003066629741810267 } +SSM seeded=true heads=2 groups=1 rows=7: max abs dx/dB/dC/ddt { 0.000000011360835217022469, 0.000000002917455535836333, 0.000000015656003028718946, 0.00000011578910319443025 } +SSM seeded=true heads=2 groups=1 rows=8: max abs dx/dB/dC/ddt { 0.00000000863441644745333, 0.0000000030472456129596037, 0.000000015656003028718946, 0.000000137397136557027 } +SSM seeded=true heads=2 groups=1 rows=9: max abs dx/dB/dC/ddt { 0.000000006604320090808358, 0.0000000027633473599630864, 0.000000015656003028718946, 0.000000058752570086806344 } +SSM seeded=true heads=2 groups=1 rows=15: max abs dx/dB/dC/ddt { 0.000000010416435020088599, 0.0000000050579541700790376, 0.00000003896021144589312, 0.00000011965777954592305 } +SSM seeded=true heads=2 groups=1 rows=16: max abs dx/dB/dC/ddt { 0.000000010399334079633227, 0.000000004927133326337874, 0.00000004227448897886621, 0.0000001134548608394681 } +SSM seeded=true heads=2 groups=1 rows=17: max abs dx/dB/dC/ddt { 0.00000001543697608283323, 0.000000004476264019170895, 0.00000004227448897886621, 0.00000018029616666304094 } +SSM seeded=true heads=2 groups=1 rows=25: max abs dx/dB/dC/ddt { 0.00000001223729547902841, 0.000000005462381803822414, 0.0000000506942888084394, 0.0000001136397048684401 } +SSM seeded=true heads=4 groups=2 rows=1: max abs dx/dB/dC/ddt { 0, 0, 0.0000000073519402093991104, 0.000000027996068407976082 } +SSM seeded=true heads=4 groups=2 rows=7: max abs dx/dB/dC/ddt { 0.000000015510263740026176, 0.000000006877330351040545, 0.00000001807481535276434, 0.00000021659550952790596 } +SSM seeded=true heads=4 groups=2 rows=8: max abs dx/dB/dC/ddt { 0.0000000129148233773968, 0.000000005653858301674086, 0.000000018969054375872485, 0.00000017414204611654327 } +SSM seeded=true heads=4 groups=2 rows=9: max abs dx/dB/dC/ddt { 0.00000002633087964332148, 0.000000006475598009203498, 0.000000018969054375872485, 0.00000017655989847931153 } +SSM seeded=true heads=4 groups=2 rows=15: max abs dx/dB/dC/ddt { 0.00000002973828677754753, 0.000000009558005810428316, 0.000000023961633413560612, 0.0000003531072296780735 } +SSM seeded=true heads=4 groups=2 rows=16: max abs dx/dB/dC/ddt { 0.000000025754882060358142, 0.000000008921690271973537, 0.000000023961633413560612, 0.00000024600469961111315 } +SSM seeded=true heads=4 groups=2 rows=17: max abs dx/dB/dC/ddt { 0.00000002553487910179897, 0.000000008921690271973537, 0.000000023961633413560612, 0.0000003112589632703333 } +SSM seeded=true heads=4 groups=2 rows=25: max abs dx/dB/dC/ddt { 0.00000003875727619395697, 0.000000014591112693085329, 0.00000002654540304236619, 0.0000004793090875310213 } +SSM seeded=true heads=64 groups=8 rows=1: max abs dx/dB/dC/ddt { 0, 0, 0.000000013799211689979884, 0.00000008033819287600608 } +SSM seeded=true heads=64 groups=8 rows=9: max abs dx/dB/dC/ddt { 0.00000002087263310801557, 0.00000001779203438051491, 0.000000027007305641868484, 0.0000002670173361307704 } +SSM seeded=true heads=64 groups=8 rows=17: max abs dx/dB/dC/ddt { 0.00000009930843825056712, 0.00000002114473045267129, 0.00000003555922037379311, 0.0000006937729284395289 } +[default] (warn): lm_head is 8-bit in groups of 64: CUDA reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): backbone.embeddings is 8-bit in groups of 64: the Intel GPU engine reads Nemotron MLX checkpoints at 4-bit in groups of 64 only; mixed and 6- or 8-bit (oQ) checkpoints serve on Macs +[default] (warn): qwen image: num_layers is not 32; this family runs the released geometry only +[default] (warn): /opt/homebrew/opt/jpeg-turbo/lib/libturbojpeg.0.dylib is not libjpeg-turbo 3.1.4.1; JPEG images are refused until it is +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +libjpeg-turbo 3.1.4.1 not found: JPEG pixel cases skipped +[default] (warn): gemma4: layers.0.self_attn.q_proj is 4-bit in groups of 32; the kernels read bits 2-6 and 8 in groups of 64 +[default] (warn): gemma4: layers.0.self_attn.q_proj is 4-bit in groups of 128; the kernels read bits 2-6 and 8 in groups of 64 +[default] (warn): gemma4: layers.3.mlp.up_proj's width is not MLX affine bits 2-6 or 8 +[default] (warn): gemma4: layers.3.mlp.up_proj's width is not MLX affine bits 2-6 or 8 +[default] (warn): gemma4: layers.3.mlp.up_proj is not a width table (false or {} marks a module unquantized); every linear is affine +[default] (warn): gemma4: layers.3.mlp.up_proj is not a width table (false or {} marks a module unquantized); every linear is affine +[default] (warn): model.layers.3.mlp.up_proj: another spelling of module layers.3.mlp.up_proj declares a different width +[default] (warn): expert parallel: this Mac runs .{ .magic = 1196182853, .version = 2, .layers = 45, .run = 45, .mtp = 1, .experts = 288, .own_lo = 0, .own_hi = 144, .cap = 8256, .pad = 0, .model = 4660, .rest = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, the peer .{ .magic = 1196182853, .version = 2, .layers = 45, .run = 8, .mtp = 1, .experts = 288, .own_lo = 144, .own_hi = 288, .cap = 8256, .pad = 0, .model = 4660, .rest = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } } +[default] (warn): expert parallel: this Mac runs .{ .magic = 1196182853, .version = 2, .layers = 45, .run = 8, .mtp = 1, .experts = 288, .own_lo = 144, .own_hi = 288, .cap = 8256, .pad = 0, .model = 4660, .rest = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } }, the peer .{ .magic = 1196182853, .version = 2, .layers = 45, .run = 45, .mtp = 1, .experts = 288, .own_lo = 0, .own_hi = 144, .cap = 8256, .pad = 0, .model = 4660, .rest = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } } +sparse attention (64 heads): 99.97% of outputs equal the row kernel's, at most 0.96 bf16 steps apart +sparse attention (32 heads): 99.96% of outputs equal the row kernel's, at most 0.94 bf16 steps apart +absorb on the tensor units: 99.99% of outputs equal the row kernel's, at most 0.98 bf16 steps apart +KDA heads=64 rows=1: worst 0.4999 bf16 steps, state 0.000000002 +KDA heads=64 rows=2: worst 0.4999 bf16 steps, state 0.000000002 +KDA heads=64 rows=8: worst 0.4999 bf16 steps, state 0.000000003 +KDA heads=64 rows=16: worst 0.4999 bf16 steps, state 0.000000005 +KDA heads=32 rows=1: worst 0.4999 bf16 steps, state 0.000000002 +KDA heads=32 rows=2: worst 0.4999 bf16 steps, state 0.000000002 +KDA heads=32 rows=8: worst 0.4999 bf16 steps, state 0.000000003 +KDA heads=32 rows=16: worst 0.4999 bf16 steps, state 0.000000005 +[default] (warn): language_model.model.layers.1.mlp.down_proj: another spelling of module layers.1.mlp.down_proj declares a different width +glm plan: 4 of 45 layers and the MTP layer, 1.51 GB in buffers, every tensor's name, dtype and shape checked; 4-bit/64 x39 +glm plan: 4 of 45 layers and the MTP layer, 1.51 GB in buffers, every tensor's name, dtype and shape checked; 2-bit/64 x2, 4-bit/64 x35, 5-bit/32 x1, 8-bit/64 x1 +glm plan: 4 of 45 layers and the MTP layer, 1.57 GB in buffers, every tensor's name, dtype and shape checked; 4-bit/64 x37, 5-bit/64 x4, 6-bit/64 x2, 8-bit/64 x4 +glm plan: 4 of 45 layers and the MTP layer, 1.86 GB in buffers, every tensor's name, dtype and shape checked; 2-bit/64 x3, 4-bit/64 x34, 5-bit/64 x1, 6-bit/64 x1, 8-bit/64 x1 +glm plan: 4 of 45 layers and the MTP layer, 1.56 GB in buffers, every tensor's name, dtype and shape checked; 4-bit/64 x40 +glm plan: 4 of 45 layers and the MTP layer, 1.51 GB in buffers, every tensor's name, dtype and shape checked; 4-bit/64 x39 +glm plan: 4 of 45 layers and the MTP layer, 1.56 GB in buffers, every tensor's name, dtype and shape checked; 4-bit/64 x40 +glm plan: 4 of 45 layers and the MTP layer, 1.56 GB in buffers, every tensor's name, dtype and shape checked; 4-bit/64 x40 +failed command: ./.zc/o/0c09ec6a65c322a6ff35be956b2a97ff/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): language_model.lm_head: stored affine but declared unquantized +[default] (warn): language_model.model.embed_tokens: stored affine but declared unquantized +[default] (warn): model.layers.0.self_attn.v_proj.weight: another spelling of module layers.0.self_attn.v_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +failed command: ./.zc/o/dd6282fa16ed7809f1911c61d168a4dc/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): XPU: a command list did not drain at close; its memory and context stay until the process ends +failed command: ./.zc/o/b0c93768170656274af2e454ce06741c/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): pack ~/tmp/rv1001-3/.zig-cache/tmp/pjp-VDUM_WukKQoI/pack.safetensors has no recorded source; skipping the identity check +[default] (warn): pack pack.safetensors has no recorded source; skipping the identity check +[default] (warn): language_model.mtp.fc_embedding: not read at its declared width: UnsupportedQuantization +[default] (warn): language_model.model.embed_tokens: not stored at its declared width: InvalidAffineShape +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): lane L0.att.o: LaneRowsNotFours +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): model.layers.0.self_attn.v_proj.weight: another spelling of module layers.0.self_attn.v_proj declares a different width +failed command: ./.zc/o/044f738c76260555744ab9a852e4428e/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): lane host: stopping at a round boundary: 2 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (server restarting: retry shortly) +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (GpuFailed): every admitted stream fails +[default] (warn): lane host: a round failed (the planner refused): every admitted stream fails +[default] (warn): lane host: a round failed (PlannerFull): every admitted stream fails +[default] (warn): lane host: stopping at a round boundary: 1 running and 1 queued request(s) end (drain deadline) +failed command: ./.zc/o/310137a647dfc9287f4b6169e602b622/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +test ++- run test w +[default] (warn): language_model.mtp.fc_embedding: not read at its declared width: UnsupportedQuantization +[default] (warn): language_model.model.embed_tokens: not stored at its declared width: InvalidAffineShape +[default] (warn): model.layers.0.mlp.up_proj.weight: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): model.layers.0.mlp.up_proj: another spelling of module layers.0.mlp.up_proj declares a different width +[default] (warn): lane L0.att.o: LaneRowsNotFours +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): lane L21.gdn.in: OutOfMemory +[default] (warn): pack ~/tmp/rv1001-3/.zig-cache/tmp/cJXCDsiO3kRrEuMp/pack.safetensors has no recorded source; skipping the identity check +[default] (warn): pack pack.safetensors has no recorded source; skipping the identity check +[default] (warn): model.layers.0.self_attn.v_proj.weight: another spelling of module layers.0.self_attn.v_proj declares a different width +failed command: ./.zc/o/50466f0ace1851a396cf6aa2f82ee209/test --cache-dir=./.zc --seed=0x9c8f20b1 --listen=- + +Build Summary: 292/292 steps succeeded; 4315/4331 tests passed (16 skipped) +test success ++- run test 37 pass (37 total) 49ms MaxRSS:8M +| +- compile test debug native-native success 4s MaxRSS:316M ++- run test 7 pass (7 total) 306ms MaxRSS:70M +| +- compile test debug native success 5s MaxRSS:312M ++- run test 9 pass (9 total) 75ms MaxRSS:4M +| +- compile test debug native success 4s MaxRSS:285M +| +- options success +| +- compile exe digest-writer debug native success 3s MaxRSS:263M ++- run test 6 pass (6 total) 5ms MaxRSS:3M +| +- compile test debug native success 4s MaxRSS:275M ++- test-flashnext-batch-host success +| +- run test 6 pass (6 total) 3s MaxRSS:30M +| +- compile test debug native success 4s MaxRSS:384M +| +- WriteFile flashnext-test-empty.zig success ++- run test 2 pass (2 total) 24ms MaxRSS:3M +| +- compile test debug native-native success 4s MaxRSS:272M ++- run test 4 pass (4 total) 22ms MaxRSS:3M +| +- compile test debug native-native success 3s MaxRSS:266M ++- run test 4 pass (4 total) 10ms MaxRSS:3M +| +- compile test debug native-native success 3s MaxRSS:286M ++- run test 3 pass (3 total) 5ms MaxRSS:3M +| +- compile test debug native-native success 4s MaxRSS:270M ++- run test 23 pass (23 total) 7ms MaxRSS:3M +| +- compile test debug native-native success 5s MaxRSS:284M ++- run test 5 pass (5 total) 10ms MaxRSS:7M +| +- compile test debug native-native success 3s MaxRSS:335M ++- run test 19 pass (19 total) 8ms MaxRSS:6M +| +- compile test fast native-native success 25s MaxRSS:492M +| +- WriteFile drama3.metallib success +| +- run xcrun (drama3.metallib) success 56ms +| +- run xcrun (d3_dense.air) success 160ms +| +- run xcrun (d3_gdn.air) success 143ms +| +- run xcrun (d3_attn.air) success 143ms +| +- run xcrun (d3_moe.air) success 185ms +| +- run xcrun (d3_sample.air) success 91ms +| +- run xcrun (d3_codec.air) success 164ms ++- run test 131 pass (131 total) 1s MaxRSS:21M +| +- compile test fast native-native success 41s MaxRSS:872M +| +- WriteFile draft_ids.txt success +| +- WriteFile nemotron_packed.metallib success +| +- run xcrun (nemotron_packed.metallib) success 447ms +| +- run xcrun (coop_down_1_0.air) success 1s +| +- run xcrun (coop_down_2_0.air) success 485ms +| +- run xcrun (coop_down_2_1.air) success 452ms +| +- run xcrun (coop_draft_1_0.air) success 416ms +| +- run xcrun (coop_draft_2_0.air) success 406ms +| +- run xcrun (coop_draft_2_1.air) success 485ms +| +- run xcrun (coop_eh_1_0.air) success 362ms +| +- run xcrun (coop_eh_1_0_sk.air) success 392ms +| +- run xcrun (coop_eh_2_0.air) success 1s +| +- run xcrun (coop_eh_2_0_sk.air) success 261ms +| +- run xcrun (coop_eh_2_1.air) success 257ms +| +- run xcrun (coop_eh_2_1_sk.air) success 465ms +| +- run xcrun (coop_head_1_0.air) success 1s +| +- run xcrun (coop_head_2_0.air) success 475ms +| +- run xcrun (coop_head_2_1.air) success 318ms +| +- run xcrun (coop_in_1_0.air) success 305ms +| +- run xcrun (coop_in_1_0_sk.air) success 450ms +| +- run xcrun (coop_in_2_0.air) success 402ms +| +- run xcrun (coop_in_2_0_sk.air) success 417ms +| +- run xcrun (coop_in_2_1.air) success 442ms +| +- run xcrun (coop_in_2_1_sk.air) success 442ms +| +- run xcrun (coop_out_1_0.air) success 430ms +| +- run xcrun (coop_out_1_0_sk.air) success 257ms +| +- run xcrun (coop_out_2_0.air) success 322ms +| +- run xcrun (coop_out_2_0_sk.air) success 390ms +| +- run xcrun (coop_out_2_1.air) success 413ms +| +- run xcrun (coop_out_2_1_sk.air) success 408ms +| +- run xcrun (coop_qkv_1_0.air) success 388ms +| +- run xcrun (coop_qkv_2_0.air) success 356ms +| +- run xcrun (coop_qkv_2_1.air) success 294ms +| +- run xcrun (coop_up_1_0.air) success 410ms +| +- run xcrun (coop_up_2_0.air) success 428ms +| +- run xcrun (coop_up_2_1.air) success 1s +| +- run xcrun (up_relu2_1_0.air) success 257ms +| +- run xcrun (up_relu2_2_0.air) success 450ms +| +- run xcrun (up_relu2_2_1.air) success 432ms ++- run test 77 pass (77 total) 66ms MaxRSS:9M +| +- compile test fast native-native success 28s MaxRSS:753M ++- run test 38 pass (38 total) 132ms MaxRSS:5M +| +- compile test fast native-native success 15s MaxRSS:471M ++- run test 86 pass (86 total) 93ms MaxRSS:10M +| +- compile test fast native-native success 33s MaxRSS:762M ++- run test 9 pass (9 total) 7ms MaxRSS:6M +| +- compile test fast native-native success 19s MaxRSS:408M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 2 pass (2 total) 6ms MaxRSS:6M +| +- compile test fast native-native success 16s MaxRSS:398M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- test-exl3 success +| +- compile obj exl3_metal_loader debug native-native success 363ms MaxRSS:102M +| | +- WriteFile exl3_projection.metal success +| +- run test 11 pass (11 total) 21ms MaxRSS:7M +| | +- compile test fast native-native success 19s MaxRSS:425M +| | +- WriteFile exl3_projection.metal (reused) +| +- run test 2 pass (2 total) 5ms MaxRSS:6M +| +- compile test fast native-native success 19s MaxRSS:401M +| +- WriteFile exl3_projection.metal (reused) ++- run test 4 pass (4 total) 5ms MaxRSS:6M +| +- compile test fast native-native success 19s MaxRSS:396M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 19 pass (19 total) 2s MaxRSS:24M +| +- compile test safe native-native success 43s MaxRSS:1G +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile drama3_core.zig success +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal success +| +- WriteFile drama3.metallib cached +| | +- run xcrun (drama3.metallib) cached +| | +- run xcrun (d3_dense.air) cached +| | +- run xcrun (d3_gdn.air) cached +| | +- run xcrun (d3_attn.air) cached +| | +- run xcrun (d3_moe.air) cached +| | +- run xcrun (d3_sample.air) cached +| | +- run xcrun (d3_codec.air) cached +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 9 pass (9 total) 5ms MaxRSS:6M +| +- compile test safe native-native success 20s MaxRSS:413M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 3 pass (3 total) 11ms MaxRSS:7M +| +- compile test fast native-native success 17s MaxRSS:419M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 2 pass (2 total) 6ms MaxRSS:6M +| +- compile test fast native-native success 16s MaxRSS:401M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 1 pass (1 total) 7ms MaxRSS:6M +| +- compile test safe native-native success 19s MaxRSS:412M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 5 pass (5 total) 4ms MaxRSS:2M +| +- compile test safe native-native success 20s MaxRSS:370M ++- run test 10 pass (10 total) 7ms MaxRSS:6M +| +- compile test safe native-native success 22s MaxRSS:435M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 3 pass (3 total) 4ms MaxRSS:2M +| +- compile test safe native-native success 19s MaxRSS:368M ++- run test 4 pass (4 total) 3ms MaxRSS:2M +| +- compile test safe native-native success 13s MaxRSS:368M ++- run test 1 pass (1 total) 19ms MaxRSS:6M +| +- compile test safe native-native success 15s MaxRSS:419M ++- run test 112 pass (112 total) 207ms MaxRSS:17M +| +- compile test safe native-native success 29s MaxRSS:653M +| +- WriteFile xsum.metal (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 7 pass (7 total) 10ms MaxRSS:2M +| +- compile test safe native-native success 12s MaxRSS:395M ++- run test 1 pass (1 total) 5ms MaxRSS:6M +| +- compile test safe native-native success 18s MaxRSS:412M ++- run sh success 75ms +| +- compile exe tensorfold-native safe native-native success 2m MaxRSS:4G +| +- options success +| +- WriteFile drama3_core.zig (reused) +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile drama3.metallib (+1 more reused dependencies) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 433 pass (433 total) 1m MaxRSS:97M +| +- compile test debug native-native success 27s MaxRSS:1G ++- run test 5 pass (5 total) 6ms MaxRSS:3M +| +- compile test debug native-native success 3s MaxRSS:287M ++- run test 25 pass (25 total) 50s MaxRSS:236M +| +- compile test debug native-native success 5s MaxRSS:335M ++- run test 15 pass (15 total) 353ms MaxRSS:33M +| +- compile test debug native-native success 8s MaxRSS:391M ++- run test 204 pass (204 total) 1s MaxRSS:12M +| +- compile test fast native-native success 45s MaxRSS:984M ++- run test 8 pass (8 total) 96ms MaxRSS:14M +| +- compile test fast native-native success 15s MaxRSS:431M ++- run test 1 pass (1 total) 6ms MaxRSS:6M +| +- compile test fast native-native success 9s MaxRSS:393M +| +- WriteFile tensorfold.metallib success +| +- run xcrun (tensorfold.metallib) success 466ms +| +- run xcrun (runtime_tests.air) success 63ms +| +- run xcrun (nemotron_expert_up.air) success 1s +| +- run xcrun (nemotron_expert_down.air) success 1s ++- run test 648 pass, 3 skip (651 total) 10s MaxRSS:156M +| +- compile test fast native-native success 2m MaxRSS:2G +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 163 pass (163 total) 698ms MaxRSS:29M +| +- compile test fast native-native success 36s MaxRSS:793M ++- run test 30 pass, 1 skip (31 total) 42ms MaxRSS:2M +| +- compile test fast native-native success 32s MaxRSS:486M ++- run test 1 pass (1 total) 3ms MaxRSS:2M +| +- compile test fast native-native success 12s MaxRSS:346M ++- run test 6 pass (6 total) 33ms MaxRSS:6M +| +- compile test fast native-native success 15s MaxRSS:403M ++- run test 4 pass (4 total) 4ms MaxRSS:2M +| +- compile test fast native-native success 12s MaxRSS:366M ++- test-cuda-host success +| +- run test 1 pass (1 total) 6ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:266M +| +- run test 30 pass (30 total) 76ms MaxRSS:4M +| | +- compile test debug native success 3s MaxRSS:368M +| | +- WriteFile h3-compile-only.fatbin success +| | +- options cached +| +- run test 3 pass (3 total) 885ms MaxRSS:70M +| | +- compile test debug native success 3s MaxRSS:288M +| +- run test 10 pass (10 total) 2s MaxRSS:15M +| | +- compile test debug native success 5s MaxRSS:414M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- compile obj exl3_host_compile debug native success 497ms MaxRSS:112M +| | +- options (reused) +| +- run test 6 pass (6 total) 10ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:306M +| | +- options (reused) +| +- run test 6 pass (6 total) 103ms MaxRSS:5M +| | +- compile test debug native success 3s MaxRSS:291M +| | +- options (reused) +| +- run test 8 pass (8 total) 16ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:277M +| +- run test 3 pass (3 total) 82ms MaxRSS:5M +| | +- compile test debug native success 4s MaxRSS:274M +| +- run test 8 pass (8 total) 17ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:282M +| +- run test 4 pass (4 total) 20ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:282M +| | +- options (reused) +| +- run test 1 pass (1 total) 57ms MaxRSS:4M +| | +- compile test debug native success 3s MaxRSS:286M +| | +- options (reused) +| +- run test 4 pass (4 total) 4ms MaxRSS:2M +| | +- compile test fast native success 17s MaxRSS:372M +| | +- options (reused) +| +- run test 1 pass (1 total) 26ms MaxRSS:3M +| | +- compile test fast native success 15s MaxRSS:384M +| | +- options (reused) +| +- compile obj bolt_host_compile debug aarch64-linux-gnu success 987ms MaxRSS:149M +| | +- options cached +| | +- WriteFile draft_ids.txt (reused) +| +- run test 26 pass (26 total) 859ms MaxRSS:6M +| | +- compile test debug native success 4s MaxRSS:373M +| | +- options (reused) +| +- compile obj drama_cuda_native_host debug native success 540ms MaxRSS:126M +| | +- options (reused) +| +- run test 5 pass (5 total) 11ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:279M +| +- run test 433 pass (433 total) 1m MaxRSS:96M +| | +- compile test debug native success 27s MaxRSS:1G +| +- run test 5 pass (5 total) 6ms MaxRSS:3M +| | +- compile test debug native success 4s MaxRSS:286M +| +- run test 130 pass (130 total) 75ms MaxRSS:4M +| | +- compile test debug native success 5s MaxRSS:379M +| | +- options (reused) +| +- run test 280 pass, 2 skip (282 total) 12s MaxRSS:132M +| | +- compile test debug native success 11s MaxRSS:587M +| +- run test 163 pass (163 total) 8s MaxRSS:65M +| | +- compile test debug native success 6s MaxRSS:471M +| +- run test 74 pass, 1 skip (75 total) 205ms MaxRSS:10M +| | +- compile test debug native success 6s MaxRSS:494M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test 33 pass (33 total) 14s MaxRSS:24M +| | +- compile test debug native success 5s MaxRSS:341M +| | +- options (reused) +| +- run test 12 pass (12 total) 22ms MaxRSS:3M +| | +- compile test debug native success 5s MaxRSS:333M +| | +- options (reused) +| +- run test 83 pass (83 total) 328ms MaxRSS:74M +| | +- compile test debug native success 5s MaxRSS:440M +| | +- options (reused) +| +- run test 13 pass (13 total) 8s MaxRSS:15M +| | +- compile test debug native success 6s MaxRSS:389M +| | +- options (reused) +| +- run test 29 pass (29 total) 6s MaxRSS:1G +| | +- compile test debug native success 3s MaxRSS:340M +| | +- options (reused) +| +- run test 5 pass (5 total) 7ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:285M +| +- run test 20 pass (20 total) 156ms MaxRSS:5M +| | +- compile test debug native success 5s MaxRSS:346M +| +- run test 204 pass (204 total) 6s MaxRSS:42M +| | +- compile test debug native success 8s MaxRSS:529M +| +- run test 52 pass, 1 skip (53 total) 84ms MaxRSS:24M +| | +- compile test debug native success 12s MaxRSS:631M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test 3 pass (3 total) 4ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:268M +| +- run test 52 pass (52 total) 130ms MaxRSS:61M +| | +- compile test debug native success 5s MaxRSS:391M +| | +- options (reused) +| +- run test 5 pass (5 total) 5ms MaxRSS:3M +| | +- compile test debug native success 3s MaxRSS:279M +| +- run test 25 pass (25 total) 42s MaxRSS:236M +| | +- compile test debug native success 4s MaxRSS:336M +| +- run test 1 pass (1 total) 14ms MaxRSS:3M +| +- compile test debug native success 3s MaxRSS:314M +| +- options (reused) +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile exlr8_alias.zig cached ++- run test 4 pass (4 total) 4ms MaxRSS:3M +| +- compile test debug native success 4s MaxRSS:269M +| +- options success ++- run test 13 pass, 1 skip (14 total) 115ms MaxRSS:4M +| +- compile test debug native success 4s MaxRSS:314M +| +- options (reused) ++- run test 3 pass (3 total) 7ms MaxRSS:3M +| +- compile test debug native success 3s MaxRSS:283M +| +- options (reused) ++- run test 1 pass (1 total) 9ms MaxRSS:3M +| +- compile test debug native success 2s MaxRSS:274M ++- test-kv-pool success +| +- run test 9 pass (9 total) 15ms MaxRSS:3M +| | +- compile test debug native success 4s MaxRSS:276M +| +- run test 6 pass (6 total) 6s MaxRSS:34M +| +- compile test debug native success 8s MaxRSS:388M ++- run test 51 pass (51 total) 41ms MaxRSS:3M +| +- compile test debug native success 3s MaxRSS:324M +| +- options success +| | +- compile lib hip-mock-success debug native success 140ms MaxRSS:54M +| | +- compile lib hip-mock-failed debug native success 146ms MaxRSS:55M +| | +- compile lib hip-mock-missing debug native success 140ms MaxRSS:56M +| +- WriteFile kernels.zig cached ++- run test 23 pass, 1 skip (24 total) 111ms MaxRSS:5M +| +- compile test debug native success 4s MaxRSS:349M ++- run test 49 pass (49 total) 41ms MaxRSS:9M +| +- compile test debug native success 6s MaxRSS:353M +| +- WriteFile kernels.zig success ++- run test 30 pass, 1 skip (31 total) 111ms MaxRSS:5M +| +- compile test debug native success 11s MaxRSS:481M +| +- WriteFile kernels.zig (reused) ++- run test 18 pass (18 total) 394ms MaxRSS:7M +| +- compile test debug native success 10s MaxRSS:495M +| +- WriteFile kernels.zig (reused) ++- run test 4 pass (4 total) 23ms MaxRSS:3M +| +- compile test debug native success 3s MaxRSS:286M +| +- WriteFile kernels.zig (reused) ++- run test 3 pass (3 total) 6ms MaxRSS:3M +| +- compile test debug native success 3s MaxRSS:287M +| +- WriteFile kernels.zig (reused) ++- compile exe tf-qwen35-test debug native success 3s MaxRSS:170M +| +- WriteFile kernels.zig (reused) ++- compile exe tensorfold-native safe native success 7s MaxRSS:277M +| +- options (reused) +| +- WriteFile exlr8_alias.zig success +| +- WriteFile kernels.zig (reused) ++- run test 26 pass, 5 skip (31 total) 21ms MaxRSS:6M +| +- compile test safe native-native success 29s MaxRSS:540M ++- run test 112 pass (112 total) 5s MaxRSS:41M +| +- compile test debug native success 5s MaxRSS:390M ++- run test 20 pass (20 total) 214ms MaxRSS:11M +| +- compile test debug native success 5s MaxRSS:329M ++- run exe learned-warning-test success 20ms + +- compile exe learned-warning-test debug native success 20s MaxRSS:297M + +env ZIG_GLOBAL_CACHE_DIR=$HOME/tmp/zig-global-1001 build test -Dcpu=apple_m1 1047.06s user 62.57s system 277% cpu 6:40.18 total +exit=0 diff --git a/docs/receipts/drafter-counts/host-controls/host-tests.log b/docs/receipts/drafter-counts/host-controls/host-tests.log new file mode 100644 index 0000000000..3bb4e21fc8 --- /dev/null +++ b/docs/receipts/drafter-counts/host-controls/host-tests.log @@ -0,0 +1,412 @@ +616d3065d43a300693c92345bec83061de438148 +test ++- test-cuda-host + +- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zc/o/eaba94c481a066499f7aa14a4611fabd/test --cache-dir=./.zc --seed=0x9ca6eaef --listen=- + +test ++- test-cuda-host + +- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zc/o/a529abb901551c852fa65e556ac33565/test --cache-dir=./.zc --seed=0x9ca6eaef --listen=- + +Build Summary: 292/292 steps succeeded; 957/958 tests passed (1 skipped) +test success ++- run test cached +| +- compile test debug native-native cached 77ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 64ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 70ms MaxRSS:32M +| +- options cached +| +- compile exe digest-writer debug native cached 63ms MaxRSS:32M ++- run test 6 pass (6 total) 15ms MaxRSS:3M +| +- compile test debug native cached 66ms MaxRSS:32M ++- test-flashnext-batch-host success +| +- run test 6 pass (6 total) 2s MaxRSS:30M +| +- compile test debug native cached 91ms MaxRSS:32M +| +- WriteFile flashnext-test-empty.zig cached ++- run test cached +| +- compile test debug native-native cached 66ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 61ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 66ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 72ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 80ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 152ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 90ms MaxRSS:32M +| +- WriteFile drama3.metallib cached +| +- run xcrun (drama3.metallib) cached +| +- run xcrun (d3_dense.air) cached +| +- run xcrun (d3_gdn.air) cached +| +- run xcrun (d3_attn.air) cached +| +- run xcrun (d3_moe.air) cached +| +- run xcrun (d3_sample.air) cached +| +- run xcrun (d3_codec.air) cached ++- run test cached +| +- compile test fast native-native cached 85ms MaxRSS:32M +| +- WriteFile draft_ids.txt cached +| +- WriteFile nemotron_packed.metallib cached +| +- run xcrun (nemotron_packed.metallib) cached +| +- run xcrun (coop_down_1_0.air) cached +| +- run xcrun (coop_down_2_0.air) cached +| +- run xcrun (coop_down_2_1.air) cached +| +- run xcrun (coop_draft_1_0.air) cached +| +- run xcrun (coop_draft_2_0.air) cached +| +- run xcrun (coop_draft_2_1.air) cached +| +- run xcrun (coop_eh_1_0.air) cached +| +- run xcrun (coop_eh_1_0_sk.air) cached +| +- run xcrun (coop_eh_2_0.air) cached +| +- run xcrun (coop_eh_2_0_sk.air) cached +| +- run xcrun (coop_eh_2_1.air) cached +| +- run xcrun (coop_eh_2_1_sk.air) cached +| +- run xcrun (coop_head_1_0.air) cached +| +- run xcrun (coop_head_2_0.air) cached +| +- run xcrun (coop_head_2_1.air) cached +| +- run xcrun (coop_in_1_0.air) cached +| +- run xcrun (coop_in_1_0_sk.air) cached +| +- run xcrun (coop_in_2_0.air) cached +| +- run xcrun (coop_in_2_0_sk.air) cached +| +- run xcrun (coop_in_2_1.air) cached +| +- run xcrun (coop_in_2_1_sk.air) cached +| +- run xcrun (coop_out_1_0.air) cached +| +- run xcrun (coop_out_1_0_sk.air) cached +| +- run xcrun (coop_out_2_0.air) cached +| +- run xcrun (coop_out_2_0_sk.air) cached +| +- run xcrun (coop_out_2_1.air) cached +| +- run xcrun (coop_out_2_1_sk.air) cached +| +- run xcrun (coop_qkv_1_0.air) cached +| +- run xcrun (coop_qkv_2_0.air) cached +| +- run xcrun (coop_qkv_2_1.air) cached +| +- run xcrun (coop_up_1_0.air) cached +| +- run xcrun (coop_up_2_0.air) cached +| +- run xcrun (coop_up_2_1.air) cached +| +- run xcrun (up_relu2_1_0.air) cached +| +- run xcrun (up_relu2_2_0.air) cached +| +- run xcrun (up_relu2_2_1.air) cached ++- run test cached +| +- compile test fast native-native cached 70ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 69ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 87ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 114ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test fast native-native cached 123ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- test-exl3 success +| +- compile obj exl3_metal_loader debug native-native cached 78ms MaxRSS:31M +| | +- WriteFile exl3_projection.metal cached +| +- run test cached +| | +- compile test fast native-native cached 76ms MaxRSS:32M +| | +- WriteFile exl3_projection.metal (reused) +| +- run test cached +| +- compile test fast native-native cached 83ms MaxRSS:32M +| +- WriteFile exl3_projection.metal (reused) ++- run test cached +| +- compile test fast native-native cached 115ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 86ms MaxRSS:32M +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile drama3_core.zig cached +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal cached +| +- WriteFile drama3.metallib cached +| | +- run xcrun (drama3.metallib) cached +| | +- run xcrun (d3_dense.air) cached +| | +- run xcrun (d3_gdn.air) cached +| | +- run xcrun (d3_attn.air) cached +| | +- run xcrun (d3_moe.air) cached +| | +- run xcrun (d3_sample.air) cached +| | +- run xcrun (d3_codec.air) cached +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 95ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test fast native-native cached 80ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test fast native-native cached 108ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 109ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 76ms MaxRSS:32M ++- run test cached +| +- compile test safe native-native cached 115ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 79ms MaxRSS:32M ++- run test cached +| +- compile test safe native-native cached 63ms MaxRSS:32M ++- run test cached +| +- compile test safe native-native cached 100ms MaxRSS:32M ++- run test cached +| +- compile test safe native-native cached 105ms MaxRSS:32M +| +- WriteFile xsum.metal (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 74ms MaxRSS:32M ++- run test cached +| +- compile test safe native-native cached 122ms MaxRSS:32M ++- run sh success 60ms +| +- compile exe tensorfold-native safe native-native success 2m MaxRSS:4G +| +- options cached +| +- WriteFile drama3_core.zig (reused) +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile drama3.metallib (+1 more reused dependencies) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 433 pass (433 total) 58s MaxRSS:97M +| +- compile test debug native-native success 16s MaxRSS:1G ++- run test cached +| +- compile test debug native-native cached 56ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 73ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 79ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 72ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 85ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 68ms MaxRSS:32M +| +- WriteFile tensorfold.metallib cached +| +- run xcrun (tensorfold.metallib) cached +| +- run xcrun (runtime_tests.air) cached +| +- run xcrun (nemotron_expert_up.air) cached +| +- run xcrun (nemotron_expert_down.air) cached ++- run test cached +| +- compile test fast native-native cached 123ms MaxRSS:32M +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test fast native-native cached 73ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 72ms MaxRSS:31M ++- run test cached +| +- compile test fast native-native cached 61ms MaxRSS:31M ++- run test cached +| +- compile test fast native-native cached 113ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 80ms MaxRSS:32M ++- test-cuda-host success +| +- run test cached +| | +- compile test debug native cached 74ms MaxRSS:32M +| +- run test 30 pass (30 total) 88ms MaxRSS:4M +| | +- compile test debug native cached 83ms MaxRSS:32M +| | +- WriteFile h3-compile-only.fatbin cached +| | +- options cached +| +- run test cached +| | +- compile test debug native cached 74ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 94ms MaxRSS:32M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- compile obj exl3_host_compile debug native cached 98ms MaxRSS:31M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 77ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 93ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 73ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 103ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 67ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 93ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 85ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test fast native cached 94ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test fast native cached 86ms MaxRSS:32M +| | +- options (reused) +| +- compile obj bolt_host_compile debug aarch64-linux-gnu cached 52ms MaxRSS:31M +| | +- options cached +| | +- WriteFile draft_ids.txt (reused) +| +- run test 26 pass (26 total) 525ms MaxRSS:6M +| | +- compile test debug native cached 80ms MaxRSS:32M +| | +- options (reused) +| +- compile obj drama_cuda_native_host debug native cached 86ms MaxRSS:31M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 84ms MaxRSS:32M +| +- run test 433 pass (433 total) 58s MaxRSS:97M +| | +- compile test debug native success 15s MaxRSS:1G +| +- run test cached +| | +- compile test debug native cached 73ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 86ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 75ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 101ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 100ms MaxRSS:32M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 81ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 76ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 81ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 98ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 59ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 76ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 75ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 102ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 83ms MaxRSS:32M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test cached +| | +- compile test debug native cached 74ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 86ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 87ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 76ms MaxRSS:32M +| +- run test cached +| +- compile test debug native cached 99ms MaxRSS:32M +| +- options (reused) +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile exlr8_alias.zig cached ++- run test cached +| +- compile test debug native cached 83ms MaxRSS:32M +| +- options cached ++- run test cached +| +- compile test debug native cached 80ms MaxRSS:32M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 85ms MaxRSS:32M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 87ms MaxRSS:32M ++- test-kv-pool success +| +- run test cached +| | +- compile test debug native cached 80ms MaxRSS:32M +| +- run test cached +| +- compile test debug native cached 82ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 78ms MaxRSS:32M +| +- options cached +| | +- compile lib hip-mock-success debug native cached 66ms MaxRSS:31M +| | +- compile lib hip-mock-failed debug native cached 73ms MaxRSS:31M +| | +- compile lib hip-mock-missing debug native cached 67ms MaxRSS:31M +| +- WriteFile kernels.zig cached ++- run test 23 pass, 1 skip (24 total) 138ms MaxRSS:5M +| +- compile test debug native cached 91ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 78ms MaxRSS:32M +| +- WriteFile kernels.zig cached ++- run test cached +| +- compile test debug native cached 79ms MaxRSS:32M +| +- WriteFile kernels.zig (reused) ++- run test cached +| +- compile test debug native cached 67ms MaxRSS:32M +| +- WriteFile kernels.zig (reused) ++- run test cached +| +- compile test debug native cached 84ms MaxRSS:32M +| +- WriteFile kernels.zig (reused) ++- run test cached +| +- compile test debug native cached 80ms MaxRSS:32M +| +- WriteFile kernels.zig (reused) ++- compile exe tf-qwen35-test debug native cached 77ms MaxRSS:31M +| +- WriteFile kernels.zig (reused) ++- compile exe tensorfold-native safe native success 2s MaxRSS:256M +| +- options (reused) +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile kernels.zig (reused) ++- run test cached +| +- compile test safe native-native cached 65ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 68ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 74ms MaxRSS:32M ++- run exe learned-warning-test success 11ms + +- compile exe learned-warning-test debug native cached 59ms MaxRSS:32M + +env ZIG_GLOBAL_CACHE_DIR=$HOME/tmp/zig-global-1001 build test -Dcpu=apple_m1 219.28s user 21.06s system 158% cpu 2:31.32 total +exit=0 diff --git a/docs/receipts/drafter-counts/host-controls/lean.txt b/docs/receipts/drafter-counts/host-controls/lean.txt new file mode 100644 index 0000000000..e2f60d985a --- /dev/null +++ b/docs/receipts/drafter-counts/host-controls/lean.txt @@ -0,0 +1,8 @@ +# lean_check.py on 616d3065, then lean_new.py against main (b7ce8121) and the merge base (76aafcf2), as CI runs it +head: 2626 files, 503 problems +main b7ce8121: 2625 files, 503 problems +merge base 76aafcf2: 2625 files, 503 problems +against main: lean: 503 problems, 503 on the base, 0 new +exit=0 +against the merge base: lean: 503 problems, 503 on the base, 0 new +exit=0 diff --git a/docs/receipts/drafter-counts/host-controls/negative-control.log b/docs/receipts/drafter-counts/host-controls/negative-control.log new file mode 100644 index 0000000000..7759b518a4 --- /dev/null +++ b/docs/receipts/drafter-counts/host-controls/negative-control.log @@ -0,0 +1,480 @@ +test ++- test-cuda-host + +- run test w +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +[default] (warn): Drama readback drain failed: retaining its buffers and engine owner +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +codec requested 4, actual sizes { 4, 4, 4, 4, 1 } +codec requested 16, actual sizes { 16, 1 } +codec requested 1, actual sizes { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 } +Drama replay passed: eager/graphs, poison/reset, codec chunks and interleaved state +[default] (warn): drama3: config hidden_size = 4096, the kernels need 2048 +[default] (warn): codec.pth pickle names os.system, outside the tensor state-dict allow-list +failed command: ./.zc/o/a529abb901551c852fa65e556ac33565/test --cache-dir=./.zc --seed=0x45ae9845 --listen=- + +test ++- test-cuda-host + +- run test w +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Dimensions +tf_h3_forward_host: BadH3Dimensions +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Rotary +tf_h3_forward_host: BadH3Rotary +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Keep +tf_h3_forward_host: BadH3Keep +tf_h3_forward: BadH3Blocks +tf_h3_forward_host: BadH3Blocks +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Scalar +tf_h3_forward_host: BadH3Scalar +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Address +tf_h3_forward_host: BadH3Scalar +H3 model retained at close: NotReady +H3 geometry retained: NotReady +tensor blocks.0.norm1.weight: MissingTensor +tensor blocks.1.attn.qkv_proj.weight: MissingTensor +tensor blocks.1.norm1.weight: MissingTensor +failed command: ./.zc/o/eaba94c481a066499f7aa14a4611fabd/test --cache-dir=./.zc --seed=0x45ae9845 --listen=- + +test ++- test-cuda-host + +- run test 431 pass, 1 crash (432 total) +error: 'benchmark_run_test.test.the draft counts beside the receipt show whether a passed drafter drafted, and the receipt keeps the board's fields' terminated with signal ABRT with stderr: + thread 29985029 panic: integer overflow + ~/tmp/rv1001-3/zig/src/server/benchmark_drafts.zig:18:18: 0x102a5c423 in totals (test) + t.rounds += d.rounds; + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark_drafts.zig:50:21: 0x102a4c5d7 in value (test) + const t = totals(samples); + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark.zig:417:34: 0x102a54763 in run (test) + const kept = try drafts.value(a, &run_id, std.fs.path.basename(dest), facts.model.drafter_repo_id, passed, samples); + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark.zig:127:27: 0x102a55693 in command (test) + .run => return run(ctx, args), + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark_cli_test.zig:50:33: 0x102ead30f in command (test) + return benchmark.command(&ctx, argv); + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark_run_test.zig:400:62: 0x102ebd61b in test.the draft counts beside the receipt show whether a passed drafter drafted, and the receipt keeps the board's fields (test) + try std.testing.expectEqual(@as(u8, 0), try r.command(try argv(&r, &.{ big, "--drafter", c.drafter, "--output", out }))); + ^ + ~/zig/zig-aarch64-macos-0.17.0/lib/compiler/test_runner.zig:143:68: 0x102a120d7 in mainServer (test) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + ~/zig/zig-aarch64-macos-0.17.0/lib/compiler/test_runner.zig:72:26: 0x102a126e3 in main (test) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + ~/zig/zig-aarch64-macos-0.17.0/lib/std/start.zig:789:88: 0x102a1062f in callMain (test) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x180dabe7f in start (/usr/lib/dyld) +failed command: ./.zc/o/d3c1ec455551ad69f0e7e23d1150d4b1/test --cache-dir=./.zc --seed=0x45ae9845 --listen=- + +test ++- run test 431 pass, 1 crash (432 total) +error: 'benchmark_run_test.test.the draft counts beside the receipt show whether a passed drafter drafted, and the receipt keeps the board's fields' terminated with signal ABRT with stderr: + thread 30041193 panic: integer overflow + ~/tmp/rv1001-3/zig/src/server/benchmark_drafts.zig:18:18: 0x100510a83 in totals (test) + t.rounds += d.rounds; + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark_drafts.zig:50:21: 0x100500c2b in value (test) + const t = totals(samples); + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark.zig:417:34: 0x100508db7 in run (test) + const kept = try drafts.value(a, &run_id, std.fs.path.basename(dest), facts.model.drafter_repo_id, passed, samples); + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark.zig:127:27: 0x100509ce7 in command (test) + .run => return run(ctx, args), + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark_cli_test.zig:50:33: 0x10095e2ab in command (test) + return benchmark.command(&ctx, argv); + ^ + ~/tmp/rv1001-3/zig/src/server/benchmark_run_test.zig:400:62: 0x10096e5cf in test.the draft counts beside the receipt show whether a passed drafter drafted, and the receipt keeps the board's fields (test) + try std.testing.expectEqual(@as(u8, 0), try r.command(try argv(&r, &.{ big, "--drafter", c.drafter, "--output", out }))); + ^ + ~/zig/zig-aarch64-macos-0.17.0/lib/compiler/test_runner.zig:143:68: 0x1004c66cb in mainServer (test) + const status: TestResults.Status = if (test_fn.func()) |v| s: { + ^ + ~/zig/zig-aarch64-macos-0.17.0/lib/compiler/test_runner.zig:72:26: 0x1004c6cd7 in main (test) + return mainServer(init) catch |err| panic("internal test runner failure: {t}", .{err}); + ^ + ~/zig/zig-aarch64-macos-0.17.0/lib/std/start.zig:789:88: 0x1004c4c23 in callMain (test) + if (fn_info.param_types[0].? == std.process.Init.Minimal) return wrapMain(root.main(.{ + ^ + ???:?:?: 0x180dabe7f in start (/usr/lib/dyld) +failed command: ./.zc/o/3a0f9fb66d4cedcbb2334a504f790741/test --cache-dir=./.zc --seed=0x45ae9845 --listen=- + +Build Summary: 288/292 steps succeeded (2 failed); 953/956 tests passed (1 skipped, 2 crashed) +test transitive failure ++- run test cached +| +- compile test debug native-native cached 76ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 110ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 77ms MaxRSS:32M +| +- options cached +| +- compile exe digest-writer debug native cached 100ms MaxRSS:32M ++- run test 6 pass (6 total) 17ms MaxRSS:3M +| +- compile test debug native cached 136ms MaxRSS:32M ++- test-flashnext-batch-host success +| +- run test 6 pass (6 total) 2s MaxRSS:30M +| +- compile test debug native cached 103ms MaxRSS:32M +| +- WriteFile flashnext-test-empty.zig cached ++- run test cached +| +- compile test debug native-native cached 146ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 94ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 119ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 87ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 132ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 90ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 141ms MaxRSS:32M +| +- WriteFile drama3.metallib cached +| +- run xcrun (drama3.metallib) cached +| +- run xcrun (d3_dense.air) cached +| +- run xcrun (d3_gdn.air) cached +| +- run xcrun (d3_attn.air) cached +| +- run xcrun (d3_moe.air) cached +| +- run xcrun (d3_sample.air) cached +| +- run xcrun (d3_codec.air) cached ++- run test cached +| +- compile test fast native-native cached 120ms MaxRSS:32M +| +- WriteFile draft_ids.txt cached +| +- WriteFile nemotron_packed.metallib cached +| +- run xcrun (nemotron_packed.metallib) cached +| +- run xcrun (coop_down_1_0.air) cached +| +- run xcrun (coop_down_2_0.air) cached +| +- run xcrun (coop_down_2_1.air) cached +| +- run xcrun (coop_draft_1_0.air) cached +| +- run xcrun (coop_draft_2_0.air) cached +| +- run xcrun (coop_draft_2_1.air) cached +| +- run xcrun (coop_eh_1_0.air) cached +| +- run xcrun (coop_eh_1_0_sk.air) cached +| +- run xcrun (coop_eh_2_0.air) cached +| +- run xcrun (coop_eh_2_0_sk.air) cached +| +- run xcrun (coop_eh_2_1.air) cached +| +- run xcrun (coop_eh_2_1_sk.air) cached +| +- run xcrun (coop_head_1_0.air) cached +| +- run xcrun (coop_head_2_0.air) cached +| +- run xcrun (coop_head_2_1.air) cached +| +- run xcrun (coop_in_1_0.air) cached +| +- run xcrun (coop_in_1_0_sk.air) cached +| +- run xcrun (coop_in_2_0.air) cached +| +- run xcrun (coop_in_2_0_sk.air) cached +| +- run xcrun (coop_in_2_1.air) cached +| +- run xcrun (coop_in_2_1_sk.air) cached +| +- run xcrun (coop_out_1_0.air) cached +| +- run xcrun (coop_out_1_0_sk.air) cached +| +- run xcrun (coop_out_2_0.air) cached +| +- run xcrun (coop_out_2_0_sk.air) cached +| +- run xcrun (coop_out_2_1.air) cached +| +- run xcrun (coop_out_2_1_sk.air) cached +| +- run xcrun (coop_qkv_1_0.air) cached +| +- run xcrun (coop_qkv_2_0.air) cached +| +- run xcrun (coop_qkv_2_1.air) cached +| +- run xcrun (coop_up_1_0.air) cached +| +- run xcrun (coop_up_2_0.air) cached +| +- run xcrun (coop_up_2_1.air) cached +| +- run xcrun (up_relu2_1_0.air) cached +| +- run xcrun (up_relu2_2_0.air) cached +| +- run xcrun (up_relu2_2_1.air) cached ++- run test cached +| +- compile test fast native-native cached 107ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 147ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 115ms MaxRSS:31M ++- run test cached +| +- compile test fast native-native cached 116ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test fast native-native cached 130ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- test-exl3 success +| +- compile obj exl3_metal_loader debug native-native cached 150ms MaxRSS:31M +| | +- WriteFile exl3_projection.metal cached +| +- run test cached +| | +- compile test fast native-native cached 150ms MaxRSS:32M +| | +- WriteFile exl3_projection.metal (reused) +| +- run test cached +| +- compile test fast native-native cached 157ms MaxRSS:32M +| +- WriteFile exl3_projection.metal (reused) ++- run test cached +| +- compile test fast native-native cached 125ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 119ms MaxRSS:32M +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile drama3_core.zig cached +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal cached +| +- WriteFile drama3.metallib cached +| | +- run xcrun (drama3.metallib) cached +| | +- run xcrun (d3_dense.air) cached +| | +- run xcrun (d3_gdn.air) cached +| | +- run xcrun (d3_attn.air) cached +| | +- run xcrun (d3_moe.air) cached +| | +- run xcrun (d3_sample.air) cached +| | +- run xcrun (d3_codec.air) cached +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 101ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test fast native-native cached 97ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test fast native-native cached 119ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 132ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 111ms MaxRSS:31M ++- run test cached +| +- compile test safe native-native cached 103ms MaxRSS:32M +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 88ms MaxRSS:32M ++- run test cached +| +- compile test safe native-native cached 110ms MaxRSS:31M ++- run test cached +| +- compile test safe native-native cached 163ms MaxRSS:32M ++- run test cached +| +- compile test safe native-native cached 108ms MaxRSS:32M +| +- WriteFile xsum.metal (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test safe native-native cached 90ms MaxRSS:31M ++- run test cached +| +- compile test safe native-native cached 123ms MaxRSS:32M ++- run sh success 139ms +| +- compile exe tensorfold-native safe native-native success 3m MaxRSS:4G +| +- options cached +| +- WriteFile drama3_core.zig (reused) +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile drama3.metallib (+1 more reused dependencies) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test 431 pass, 1 crash (432 total) +| +- compile test debug native-native success 16s MaxRSS:1G ++- run test cached +| +- compile test debug native-native cached 83ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 125ms MaxRSS:32M ++- run test cached +| +- compile test debug native-native cached 81ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 126ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 95ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 130ms MaxRSS:32M +| +- WriteFile tensorfold.metallib cached +| +- run xcrun (tensorfold.metallib) cached +| +- run xcrun (runtime_tests.air) cached +| +- run xcrun (nemotron_expert_up.air) cached +| +- run xcrun (nemotron_expert_down.air) cached ++- run test cached +| +- compile test fast native-native cached 111ms MaxRSS:32M +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile xsum.metal (reused) +| +- WriteFile nemotron_packed.metallib (+1 more reused dependencies) ++- run test cached +| +- compile test fast native-native cached 147ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 135ms MaxRSS:31M ++- run test cached +| +- compile test fast native-native cached 90ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 114ms MaxRSS:32M ++- run test cached +| +- compile test fast native-native cached 105ms MaxRSS:32M ++- test-cuda-host transitive failure +| +- run test cached +| | +- compile test debug native cached 139ms MaxRSS:32M +| +- run test 30 pass (30 total) 264ms MaxRSS:4M +| | +- compile test debug native cached 114ms MaxRSS:32M +| | +- WriteFile h3-compile-only.fatbin cached +| | +- options cached +| +- run test cached +| | +- compile test debug native cached 100ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 77ms MaxRSS:32M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- compile obj exl3_host_compile debug native cached 72ms MaxRSS:31M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 114ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 110ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 115ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 131ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 96ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 94ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 109ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test fast native cached 95ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test fast native cached 112ms MaxRSS:32M +| | +- options (reused) +| +- compile obj bolt_host_compile debug aarch64-linux-gnu cached 113ms MaxRSS:31M +| | +- options cached +| | +- WriteFile draft_ids.txt (reused) +| +- run test 26 pass (26 total) 768ms MaxRSS:6M +| | +- compile test debug native cached 82ms MaxRSS:32M +| | +- options (reused) +| +- compile obj drama_cuda_native_host debug native cached 94ms MaxRSS:31M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 85ms MaxRSS:32M +| +- run test 431 pass, 1 crash (432 total) +| | +- compile test debug native success 28s MaxRSS:1G +| +- run test cached +| | +- compile test debug native cached 82ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 75ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 94ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 129ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 111ms MaxRSS:32M +| | +- WriteFile draft_ids.txt (reused) +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 91ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 106ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 90ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 106ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 103ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 114ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 85ms MaxRSS:31M +| +- run test cached +| | +- compile test debug native cached 125ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 107ms MaxRSS:32M +| | +- options (reused) +| | +- WriteFile draft_ids.txt (reused) +| +- run test cached +| | +- compile test debug native cached 82ms MaxRSS:31M +| +- run test cached +| | +- compile test debug native cached 114ms MaxRSS:32M +| | +- options (reused) +| +- run test cached +| | +- compile test debug native cached 109ms MaxRSS:32M +| +- run test cached +| | +- compile test debug native cached 130ms MaxRSS:32M +| +- run test cached +| +- compile test debug native cached 123ms MaxRSS:32M +| +- options (reused) +| +- options (reused) +| +- WriteFile draft_ids.txt (reused) +| +- WriteFile exlr8_alias.zig cached ++- run test cached +| +- compile test debug native cached 124ms MaxRSS:31M +| +- options cached ++- run test cached +| +- compile test debug native cached 83ms MaxRSS:32M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 105ms MaxRSS:32M +| +- options (reused) ++- run test cached +| +- compile test debug native cached 105ms MaxRSS:32M ++- test-kv-pool success +| +- run test cached +| | +- compile test debug native cached 132ms MaxRSS:32M +| +- run test cached +| +- compile test debug native cached 120ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 84ms MaxRSS:32M +| +- options cached +| | +- compile lib hip-mock-success debug native cached 96ms MaxRSS:31M +| | +- compile lib hip-mock-failed debug native cached 106ms MaxRSS:31M +| | +- compile lib hip-mock-missing debug native cached 95ms MaxRSS:31M +| +- WriteFile kernels.zig cached ++- run test 23 pass, 1 skip (24 total) 887ms MaxRSS:5M +| +- compile test debug native cached 90ms MaxRSS:32M ++- run test cached +| +- compile test debug native cached 114ms MaxRSS:32M +| +- WriteFile kernels.zig cached ++- run test cached +| +- compile test debug native cached 117ms MaxRSS:32M +| +- WriteFile kernels.zig (reused) ++- run test cached +| +- compile test debug native cached 130ms MaxRSS:32M +| +- WriteFile kernels.zig (reused) ++- run test cached +| +- compile test debug native cached 78ms MaxRSS:32M +| +- WriteFile kernels.zig (reused) ++- run test cached +| +- compile test debug native cached 126ms MaxRSS:32M +| +- WriteFile kernels.zig (reused) ++- compile exe tf-qwen35-test debug native cached 117ms MaxRSS:31M +| +- WriteFile kernels.zig (reused) ++- compile exe tensorfold-native safe native success 3s MaxRSS:257M +| +- options (reused) +| +- WriteFile exlr8_alias.zig cached +| +- WriteFile kernels.zig (reused) ++- run test cached +| +- compile test safe native-native cached 102ms MaxRSS:31M ++- run test cached +| +- compile test debug native cached 95ms MaxRSS:31M ++- run test cached +| +- compile test debug native cached 152ms MaxRSS:32M ++- run exe learned-warning-test success 95ms + +- compile exe learned-warning-test debug native cached 99ms MaxRSS:31M + +env ZIG_GLOBAL_CACHE_DIR=$HOME/tmp/zig-global-1001 build test -Dcpu=apple_m1 235.87s user 24.03s system 78% cpu 5:31.99 total +exit=1 diff --git a/docs/receipts/drafter-counts/match.py b/docs/receipts/drafter-counts/match.py new file mode 100644 index 0000000000..d23351cf88 --- /dev/null +++ b/docs/receipts/drafter-counts/match.py @@ -0,0 +1,117 @@ +"""Matches each run's draft counts file against its receipt, its run log and, when kept, the server's own log lines.""" +import copy +import json +import re +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +RUNS = ("out/27b-auto", "out/35b", "out/35b-serial", "out/35b-attached", "out-r5b/35b-attached") +DONE = re.compile(r"\] done (\S+) .* tokens=(\d+) sha=([0-9a-f]+) finish=(\w+) rounds=(\d+) accepted=(\d+)/(\d+)$") +POST = re.compile(r'"POST (\S+) HTTP/1\.1" (\d+)') +SAID = re.compile(r"Draft counts: (\d+) of (\d+) drafted tokens accepted over (\d+) rounds \((\d+) of (\d+) attempts") +ROUTE = {"code": "/v1/completions", "chat": "/v1/chat/completions"} +KEYS = ("rounds", "drafted", "accepted") + + +def served(log): + """Every finished request in a server log: its line, the POST line before it and the counters it printed.""" + out, post = [], None + for n, line in enumerate(log.read_text().splitlines(), 1): + if m := POST.search(line): + post = (n, m[1], m[2]) + elif m := DONE.search(line): + out.append({"line": n, "post_line": post[0], "route": post[1], "http": post[2], "id": m[1], "tokens": int(m[2]), + "sha": m[3], "finish": m[4], "rounds": int(m[5]), "accepted": int(m[6]), "drafted": int(m[7])}) + return out + + +def cells(samples): + """The receipt's samples grouped by cell, in run order: [(fixture, temperature, [index, ...]), ...].""" + out = [] + for i, s in enumerate(samples): + if not out or out[-1][:2] != (s["fixture_id"], s["temperature"]): + out.append((s["fixture_id"], s["temperature"], [])) + out[-1][2].append(i) + return out + + +def check(run, drafts, rec, log_text, serve): + """Problems with one run, plus the lines to print; an empty problem list means every count matches.""" + bad, lines = [], [] + rows, kept = drafts["samples"], rec["samples"] + if drafts["run_id"] != rec["run_id"] or drafts["receipt"] != Path(run).name + ".json" or len(rows) != len(kept): + bad.append(f"{run}: the draft counts file is not this receipt's") + return bad, lines + for d, r in zip(rows, kept): + if any(d[k] != r[k] for k in ("fixture_id", "temperature", "repeat", "status", "completion_tokens")): + bad.append(f"{run}: sample {d['fixture_id']} T={d['temperature']} r{d['repeat']} differs from the receipt's") + sums = {k: sum(d[k] for d in rows) for k in KEYS} + t = drafts["totals"] + if any(t[k] != sums[k] for k in KEYS) or t["attempts"] != len(rows) or t["reported"] != sum(d["rounds"] is not None for d in rows): + bad.append(f"{run}: totals {t} are not the sum of its samples {sums}") + said = SAID.findall(log_text) + if [tuple(map(int, x)) for x in said] != [(t["accepted"], t["drafted"], t["rounds"], t["reported"], t["attempts"])]: + bad.append(f"{run}: the run's printed line {said} is not its totals") + lines.append(f"{run}: drafter_passed={drafts['drafter_passed']} attempts={t['attempts']} reported={t['reported']} " + f"rounds={t['rounds']} drafted={t['drafted']} accepted={t['accepted']} (sum of samples, and the run's line)") + if serve is None: + lines.append(" no server log: a managed run unlinks its server's log as it starts") + return bad, lines + groups, done, at, log = cells(kept), serve, 0, f"{Path(run).parent}/35b-serve.log" + if len(done) != len(kept) + len(groups): + bad.append(f"{run}: {len(done)} finished requests in the server log for {len(kept)} samples and {len(groups)} warm-ups") + return bad, lines + from_log = dict.fromkeys(KEYS, 0) + for fixture, temp, members in groups: + w = done[at] + lines.append(f" warm-up {fixture} T={temp}: {log}:{w['line']} rounds={w['rounds']} accepted={w['accepted']}/{w['drafted']} (not a sample)") + at += 1 + for i in members: + d, r, s = rows[i], kept[i], done[at] + at += 1 + same = all(d[k] == s[k] for k in KEYS) and d["completion_tokens"] == s["tokens"] and r["token_sha"] == s["sha"] + if not same or s["route"] != ROUTE[fixture] or s["http"] != "200": + bad.append(f"{run}: sample {fixture} T={temp} r{d['repeat']} does not match {log}:{s['line']}") + for k in KEYS: + from_log[k] += s[k] + lines.append(f" {fixture} T={temp} r{d['repeat']}: rounds={d['rounds']} drafted={d['drafted']} accepted={d['accepted']} " + f"sha={r['token_sha']} = {log}:{s['line']} ({s['route']} at line {s['post_line']}, finish={s['finish']}, " + f"rounds={s['rounds']} accepted={s['accepted']}/{s['drafted']} sha={s['sha']})") + if from_log != sums: + bad.append(f"{run}: the server log's samples sum to {from_log}, the file to {sums}") + lines.append(f" server log, warm-ups left out: rounds={from_log['rounds']} drafted={from_log['drafted']} accepted={from_log['accepted']}") + return bad, lines + + +def load(run): + """One run's draft counts, receipt, run log and, for an attached run, its server's log.""" + folder = (HERE / run).parent + serve = folder / "35b-serve.log" if run.endswith("attached") else None + return (json.loads((HERE / f"{run}.drafts.json").read_text()), json.loads((HERE / f"{run}.json").read_text()), + (HERE / f"{run}.log").read_text(), served(serve) if serve else None) + + +def main(): + problems = [] + for run in RUNS: + bad, lines = check(run, *load(run)) + problems += bad + print("\n".join(lines)) + # controls: the same check refuses a count off by one and a server log read one request out of step + drafts, rec, text, serve = load("out-r5b/35b-attached") + off = copy.deepcopy(drafts) + off["samples"][7]["drafted"] += 1 + off["totals"]["drafted"] += 1 + shifted = serve[1:] + serve[:1] + for name, args in (("one drafted count +1", (off, rec, text, serve)), ("log one request out of step", (drafts, rec, text, shifted))): + bad, _ = check("out-r5b/35b-attached", *args) + print(f"control, {name}: {'refused: ' + '; '.join(bad) if bad else 'NOT REFUSED'}") + if not bad: + problems.append(f"control {name} was not refused") + print("\n".join(problems) if problems else "every count matches") + return 1 if problems else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/docs/receipts/drafter-counts/match.txt b/docs/receipts/drafter-counts/match.txt new file mode 100644 index 0000000000..a9e511acfc --- /dev/null +++ b/docs/receipts/drafter-counts/match.txt @@ -0,0 +1,61 @@ +out/27b-auto: drafter_passed=False attempts=20 reported=20 rounds=5030 drafted=262 accepted=70 (sum of samples, and the run's line) + no server log: a managed run unlinks its server's log as it starts +out/35b: drafter_passed=False attempts=20 reported=20 rounds=2495 drafted=4490 accepted=2615 (sum of samples, and the run's line) + no server log: a managed run unlinks its server's log as it starts +out/35b-serial: drafter_passed=False attempts=20 reported=20 rounds=5100 drafted=0 accepted=0 (sum of samples, and the run's line) + no server log: a managed run unlinks its server's log as it starts +out/35b-attached: drafter_passed=None attempts=20 reported=20 rounds=2441 drafted=4626 accepted=2668 (sum of samples, and the run's line) + warm-up code T=1.0: out/35b-serve.log:8 rounds=106 accepted=150/247 (not a sample) + code T=1.0 r0: rounds=106 drafted=246 accepted=150 sha=5d7effdcf06f = out/35b-serve.log:10 (/v1/completions at line 9, finish=length, rounds=106 accepted=150/246 sha=5d7effdcf06f) + code T=1.0 r1: rounds=89 drafted=249 accepted=166 sha=bd3964ef7930 = out/35b-serve.log:12 (/v1/completions at line 11, finish=length, rounds=89 accepted=166/249 sha=bd3964ef7930) + code T=1.0 r2: rounds=89 drafted=228 accepted=166 sha=bc55cc799f87 = out/35b-serve.log:14 (/v1/completions at line 13, finish=length, rounds=89 accepted=166/228 sha=bc55cc799f87) + code T=1.0 r3: rounds=118 drafted=220 accepted=138 sha=306e0fe05bc2 = out/35b-serve.log:16 (/v1/completions at line 15, finish=length, rounds=118 accepted=138/220 sha=306e0fe05bc2) + code T=1.0 r4: rounds=119 drafted=218 accepted=136 sha=5a89476cf1f5 = out/35b-serve.log:18 (/v1/completions at line 17, finish=length, rounds=119 accepted=136/218 sha=5a89476cf1f5) + warm-up chat T=1.0: out/35b-serve.log:20 rounds=155 accepted=100/155 (not a sample) + chat T=1.0 r0: rounds=141 drafted=210 accepted=114 sha=460edda16244 = out/35b-serve.log:22 (/v1/chat/completions at line 21, finish=length, rounds=141 accepted=114/210 sha=460edda16244) + chat T=1.0 r1: rounds=112 drafted=252 accepted=144 sha=0eba9c9d3070 = out/35b-serve.log:24 (/v1/chat/completions at line 23, finish=length, rounds=112 accepted=144/252 sha=0eba9c9d3070) + chat T=1.0 r2: rounds=165 drafted=150 accepted=90 sha=15d18697d5f1 = out/35b-serve.log:26 (/v1/chat/completions at line 25, finish=length, rounds=165 accepted=90/150 sha=15d18697d5f1) + chat T=1.0 r3: rounds=136 drafted=213 accepted=120 sha=d813ade7fc53 = out/35b-serve.log:28 (/v1/chat/completions at line 27, finish=length, rounds=136 accepted=120/213 sha=d813ade7fc53) + chat T=1.0 r4: rounds=131 drafted=242 accepted=125 sha=7675315607f0 = out/35b-serve.log:30 (/v1/chat/completions at line 29, finish=length, rounds=131 accepted=125/242 sha=7675315607f0) + warm-up code T=0.0: out/35b-serve.log:32 rounds=83 accepted=172/290 (not a sample) + code T=0.0 r0: rounds=83 drafted=290 accepted=172 sha=9f2923ac14d0 = out/35b-serve.log:34 (/v1/completions at line 33, finish=length, rounds=83 accepted=172/290 sha=9f2923ac14d0) + code T=0.0 r1: rounds=83 drafted=290 accepted=172 sha=9f2923ac14d0 = out/35b-serve.log:36 (/v1/completions at line 35, finish=length, rounds=83 accepted=172/290 sha=9f2923ac14d0) + code T=0.0 r2: rounds=83 drafted=290 accepted=172 sha=9f2923ac14d0 = out/35b-serve.log:38 (/v1/completions at line 37, finish=length, rounds=83 accepted=172/290 sha=9f2923ac14d0) + code T=0.0 r3: rounds=82 drafted=290 accepted=173 sha=9f2923ac14d0 = out/35b-serve.log:40 (/v1/completions at line 39, finish=length, rounds=82 accepted=173/290 sha=9f2923ac14d0) + code T=0.0 r4: rounds=82 drafted=293 accepted=173 sha=9f2923ac14d0 = out/35b-serve.log:42 (/v1/completions at line 41, finish=length, rounds=82 accepted=173/293 sha=9f2923ac14d0) + warm-up chat T=0.0: out/35b-serve.log:44 rounds=135 accepted=121/245 (not a sample) + chat T=0.0 r0: rounds=143 drafted=235 accepted=113 sha=a8c28b8297b4 = out/35b-serve.log:46 (/v1/chat/completions at line 45, finish=length, rounds=143 accepted=113/235 sha=a8c28b8297b4) + chat T=0.0 r1: rounds=207 drafted=117 accepted=48 sha=a8c28b8297b4 = out/35b-serve.log:48 (/v1/chat/completions at line 47, finish=length, rounds=207 accepted=48/117 sha=a8c28b8297b4) + chat T=0.0 r2: rounds=148 drafted=202 accepted=108 sha=a8c28b8297b4 = out/35b-serve.log:50 (/v1/chat/completions at line 49, finish=length, rounds=148 accepted=108/202 sha=a8c28b8297b4) + chat T=0.0 r3: rounds=135 drafted=246 accepted=121 sha=a8c28b8297b4 = out/35b-serve.log:52 (/v1/chat/completions at line 51, finish=length, rounds=135 accepted=121/246 sha=a8c28b8297b4) + chat T=0.0 r4: rounds=189 drafted=145 accepted=67 sha=a8c28b8297b4 = out/35b-serve.log:54 (/v1/chat/completions at line 53, finish=length, rounds=189 accepted=67/145 sha=a8c28b8297b4) + server log, warm-ups left out: rounds=2441 drafted=4626 accepted=2668 +out-r5b/35b-attached: drafter_passed=None attempts=20 reported=20 rounds=2473 drafted=4576 accepted=2636 (sum of samples, and the run's line) + warm-up code T=1.0: out-r5b/35b-serve.log:8 rounds=106 accepted=150/248 (not a sample) + code T=1.0 r0: rounds=102 drafted=264 accepted=154 sha=5d7effdcf06f = out-r5b/35b-serve.log:10 (/v1/completions at line 9, finish=length, rounds=102 accepted=154/264 sha=5d7effdcf06f) + code T=1.0 r1: rounds=89 drafted=249 accepted=166 sha=bd3964ef7930 = out-r5b/35b-serve.log:12 (/v1/completions at line 11, finish=length, rounds=89 accepted=166/249 sha=bd3964ef7930) + code T=1.0 r2: rounds=89 drafted=228 accepted=166 sha=bc55cc799f87 = out-r5b/35b-serve.log:14 (/v1/completions at line 13, finish=length, rounds=89 accepted=166/228 sha=bc55cc799f87) + code T=1.0 r3: rounds=118 drafted=220 accepted=138 sha=306e0fe05bc2 = out-r5b/35b-serve.log:16 (/v1/completions at line 15, finish=length, rounds=118 accepted=138/220 sha=306e0fe05bc2) + code T=1.0 r4: rounds=111 drafted=250 accepted=144 sha=5a89476cf1f5 = out-r5b/35b-serve.log:18 (/v1/completions at line 17, finish=length, rounds=111 accepted=144/250 sha=5a89476cf1f5) + warm-up chat T=1.0: out-r5b/35b-serve.log:20 rounds=152 accepted=103/172 (not a sample) + chat T=1.0 r0: rounds=141 drafted=209 accepted=114 sha=460edda16244 = out-r5b/35b-serve.log:22 (/v1/chat/completions at line 21, finish=length, rounds=141 accepted=114/209 sha=460edda16244) + chat T=1.0 r1: rounds=112 drafted=252 accepted=144 sha=0eba9c9d3070 = out-r5b/35b-serve.log:24 (/v1/chat/completions at line 23, finish=length, rounds=112 accepted=144/252 sha=0eba9c9d3070) + chat T=1.0 r2: rounds=159 drafted=170 accepted=96 sha=15d18697d5f1 = out-r5b/35b-serve.log:26 (/v1/chat/completions at line 25, finish=length, rounds=159 accepted=96/170 sha=15d18697d5f1) + chat T=1.0 r3: rounds=201 drafted=97 accepted=54 sha=d813ade7fc53 = out-r5b/35b-serve.log:28 (/v1/chat/completions at line 27, finish=length, rounds=201 accepted=54/97 sha=d813ade7fc53) + chat T=1.0 r4: rounds=201 drafted=105 accepted=55 sha=7675315607f0 = out-r5b/35b-serve.log:30 (/v1/chat/completions at line 29, finish=length, rounds=201 accepted=55/105 sha=7675315607f0) + warm-up code T=0.0: out-r5b/35b-serve.log:32 rounds=83 accepted=172/290 (not a sample) + code T=0.0 r0: rounds=83 drafted=290 accepted=172 sha=9f2923ac14d0 = out-r5b/35b-serve.log:34 (/v1/completions at line 33, finish=length, rounds=83 accepted=172/290 sha=9f2923ac14d0) + code T=0.0 r1: rounds=83 drafted=290 accepted=172 sha=9f2923ac14d0 = out-r5b/35b-serve.log:36 (/v1/completions at line 35, finish=length, rounds=83 accepted=172/290 sha=9f2923ac14d0) + code T=0.0 r2: rounds=83 drafted=290 accepted=172 sha=9f2923ac14d0 = out-r5b/35b-serve.log:38 (/v1/completions at line 37, finish=length, rounds=83 accepted=172/290 sha=9f2923ac14d0) + code T=0.0 r3: rounds=82 drafted=290 accepted=173 sha=9f2923ac14d0 = out-r5b/35b-serve.log:40 (/v1/completions at line 39, finish=length, rounds=82 accepted=173/290 sha=9f2923ac14d0) + code T=0.0 r4: rounds=83 drafted=290 accepted=172 sha=9f2923ac14d0 = out-r5b/35b-serve.log:42 (/v1/completions at line 41, finish=length, rounds=83 accepted=172/290 sha=9f2923ac14d0) + warm-up chat T=0.0: out-r5b/35b-serve.log:44 rounds=191 accepted=65/140 (not a sample) + chat T=0.0 r0: rounds=142 drafted=218 accepted=114 sha=a8c28b8297b4 = out-r5b/35b-serve.log:46 (/v1/chat/completions at line 45, finish=length, rounds=142 accepted=114/218 sha=a8c28b8297b4) + chat T=0.0 r1: rounds=172 drafted=168 accepted=84 sha=a8c28b8297b4 = out-r5b/35b-serve.log:48 (/v1/chat/completions at line 47, finish=length, rounds=172 accepted=84/168 sha=a8c28b8297b4) + chat T=0.0 r2: rounds=134 drafted=251 accepted=122 sha=a8c28b8297b4 = out-r5b/35b-serve.log:50 (/v1/chat/completions at line 49, finish=length, rounds=134 accepted=122/251 sha=a8c28b8297b4) + chat T=0.0 r3: rounds=146 drafted=235 accepted=110 sha=a8c28b8297b4 = out-r5b/35b-serve.log:52 (/v1/chat/completions at line 51, finish=length, rounds=146 accepted=110/235 sha=a8c28b8297b4) + chat T=0.0 r4: rounds=142 drafted=210 accepted=114 sha=a8c28b8297b4 = out-r5b/35b-serve.log:54 (/v1/chat/completions at line 53, finish=length, rounds=142 accepted=114/210 sha=a8c28b8297b4) + server log, warm-ups left out: rounds=2473 drafted=4576 accepted=2636 +control, one drafted count +1: refused: out-r5b/35b-attached: the run's printed line [('2636', '4576', '2473', '20', '20')] is not its totals; out-r5b/35b-attached: sample chat T=1.0 r2 does not match out-r5b/35b-serve.log:26; out-r5b/35b-attached: the server log's samples sum to {'rounds': 2473, 'drafted': 4576, 'accepted': 2636}, the file to {'rounds': 2473, 'drafted': 4577, 'accepted': 2636} +control, log one request out of step: refused: out-r5b/35b-attached: sample code T=1.0 r0 does not match out-r5b/35b-serve.log:12; out-r5b/35b-attached: sample code T=1.0 r1 does not match out-r5b/35b-serve.log:14; out-r5b/35b-attached: sample code T=1.0 r2 does not match out-r5b/35b-serve.log:16; out-r5b/35b-attached: sample code T=1.0 r3 does not match out-r5b/35b-serve.log:18; out-r5b/35b-attached: sample code T=1.0 r4 does not match out-r5b/35b-serve.log:20; out-r5b/35b-attached: sample chat T=1.0 r0 does not match out-r5b/35b-serve.log:24; out-r5b/35b-attached: sample chat T=1.0 r1 does not match out-r5b/35b-serve.log:26; out-r5b/35b-attached: sample chat T=1.0 r2 does not match out-r5b/35b-serve.log:28; out-r5b/35b-attached: sample chat T=1.0 r3 does not match out-r5b/35b-serve.log:30; out-r5b/35b-attached: sample chat T=1.0 r4 does not match out-r5b/35b-serve.log:32; out-r5b/35b-attached: sample code T=0.0 r2 does not match out-r5b/35b-serve.log:40; out-r5b/35b-attached: sample code T=0.0 r3 does not match out-r5b/35b-serve.log:42; out-r5b/35b-attached: sample code T=0.0 r4 does not match out-r5b/35b-serve.log:44; out-r5b/35b-attached: sample chat T=0.0 r0 does not match out-r5b/35b-serve.log:48; out-r5b/35b-attached: sample chat T=0.0 r1 does not match out-r5b/35b-serve.log:50; out-r5b/35b-attached: sample chat T=0.0 r2 does not match out-r5b/35b-serve.log:52; out-r5b/35b-attached: sample chat T=0.0 r3 does not match out-r5b/35b-serve.log:54; out-r5b/35b-attached: sample chat T=0.0 r4 does not match out-r5b/35b-serve.log:8; out-r5b/35b-attached: the server log's samples sum to {'rounds': 2537, 'drafted': 4445, 'accepted': 2572}, the file to {'rounds': 2473, 'drafted': 4576, 'accepted': 2636} +every count matches diff --git a/docs/receipts/drafter-counts/out-r5b/35b-attached.drafts.json b/docs/receipts/drafter-counts/out-r5b/35b-attached.drafts.json new file mode 100644 index 0000000000..08b94a1046 --- /dev/null +++ b/docs/receipts/drafter-counts/out-r5b/35b-attached.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":null,"drafter_repo_id":null,"receipt":"35b-attached.json","run_id":"23c5e792-c249-4f55-904a-c2aef1070d16","samples":[{"accepted":154,"completion_tokens":256,"drafted":264,"fixture_id":"code","repeat":0,"rounds":102,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":249,"fixture_id":"code","repeat":1,"rounds":89,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":228,"fixture_id":"code","repeat":2,"rounds":89,"status":"ok","temperature":1.0},{"accepted":138,"completion_tokens":256,"drafted":220,"fixture_id":"code","repeat":3,"rounds":118,"status":"ok","temperature":1.0},{"accepted":144,"completion_tokens":256,"drafted":250,"fixture_id":"code","repeat":4,"rounds":111,"status":"ok","temperature":1.0},{"accepted":114,"completion_tokens":256,"drafted":209,"fixture_id":"chat","repeat":0,"rounds":141,"status":"ok","temperature":1.0},{"accepted":144,"completion_tokens":256,"drafted":252,"fixture_id":"chat","repeat":1,"rounds":112,"status":"ok","temperature":1.0},{"accepted":96,"completion_tokens":256,"drafted":170,"fixture_id":"chat","repeat":2,"rounds":159,"status":"ok","temperature":1.0},{"accepted":54,"completion_tokens":256,"drafted":97,"fixture_id":"chat","repeat":3,"rounds":201,"status":"ok","temperature":1.0},{"accepted":55,"completion_tokens":256,"drafted":105,"fixture_id":"chat","repeat":4,"rounds":201,"status":"ok","temperature":1.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":0,"rounds":83,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":1,"rounds":83,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":2,"rounds":83,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":3,"rounds":82,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":4,"rounds":83,"status":"ok","temperature":0.0},{"accepted":114,"completion_tokens":256,"drafted":218,"fixture_id":"chat","repeat":0,"rounds":142,"status":"ok","temperature":0.0},{"accepted":84,"completion_tokens":256,"drafted":168,"fixture_id":"chat","repeat":1,"rounds":172,"status":"ok","temperature":0.0},{"accepted":122,"completion_tokens":256,"drafted":251,"fixture_id":"chat","repeat":2,"rounds":134,"status":"ok","temperature":0.0},{"accepted":110,"completion_tokens":256,"drafted":235,"fixture_id":"chat","repeat":3,"rounds":146,"status":"ok","temperature":0.0},{"accepted":114,"completion_tokens":256,"drafted":210,"fixture_id":"chat","repeat":4,"rounds":142,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":2636,"attempts":20,"drafted":4576,"reported":20,"rounds":2473}} diff --git a/docs/receipts/drafter-counts/out-r5b/35b-attached.json b/docs/receipts/drafter-counts/out-r5b/35b-attached.json new file mode 100644 index 0000000000..07cebf3bfa --- /dev/null +++ b/docs/receipts/drafter-counts/out-r5b/35b-attached.json @@ -0,0 +1 @@ +{"created_at":"2026-10-18T18:18:44.299680Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767","drafter_repo_id":null,"family":null,"local":false,"quantization":null,"repo_id":"TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP","revision":"f84b054c5677b6e59bdb969ae61915b0983d68bb","tokenizer_sha256":"4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"},"run_id":"23c5e792-c249-4f55-904a-c2aef1070d16","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.237397889,"delivery_tps":78.76696308057672,"end_to_end_seconds":3.295957964,"error_code":null,"fixture_id":"code","prefill_seconds":0.056955031,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":78.76603615452429,"status":"ok","temperature":1.0,"token_sha":"5d7effdcf06f","ttft_seconds":0.058184278},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.930787403,"delivery_tps":87.00733452688448,"end_to_end_seconds":2.992671783,"error_code":null,"fixture_id":"code","prefill_seconds":0.058689351,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":86.98033222319357,"status":"ok","temperature":1.0,"token_sha":"bd3964ef7930","ttft_seconds":0.060928333},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.862874243,"delivery_tps":89.07132425516045,"end_to_end_seconds":2.941010748,"error_code":null,"fixture_id":"code","prefill_seconds":0.075046983,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":89.05244580043684,"status":"ok","temperature":1.0,"token_sha":"bc55cc799f87","ttft_seconds":0.077185048},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.218032322,"delivery_tps":79.24096916513196,"end_to_end_seconds":3.323499297,"error_code":null,"fixture_id":"code","prefill_seconds":0.102966972,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":79.2291227575826,"status":"ok","temperature":1.0,"token_sha":"306e0fe05bc2","ttft_seconds":0.10475618},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.206673426,"delivery_tps":79.52166189810187,"end_to_end_seconds":3.304778007,"error_code":null,"fixture_id":"code","prefill_seconds":0.09582675,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":79.5097264388664,"status":"ok","temperature":1.0,"token_sha":"5a89476cf1f5","ttft_seconds":0.097737354},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.637380711,"delivery_tps":70.1053918356252,"end_to_end_seconds":3.820668011,"error_code":null,"fixture_id":"chat","prefill_seconds":0.180313155,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":70.0920132161344,"status":"ok","temperature":1.0,"token_sha":"460edda16244","ttft_seconds":0.18244713},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.436585827,"delivery_tps":74.20155143414668,"end_to_end_seconds":3.53925292,"error_code":null,"fixture_id":"chat","prefill_seconds":0.100365123,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":74.19102561723564,"status":"ok","temperature":1.0,"token_sha":"0eba9c9d3070","ttft_seconds":0.102349214},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.61322829,"delivery_tps":70.5740073788695,"end_to_end_seconds":3.70876401,"error_code":null,"fixture_id":"chat","prefill_seconds":0.093882408,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":70.57266853606137,"status":"ok","temperature":1.0,"token_sha":"15d18697d5f1","ttft_seconds":0.095217431},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.682552571,"delivery_tps":69.24544730416558,"end_to_end_seconds":3.77583423,"error_code":null,"fixture_id":"chat","prefill_seconds":0.090835443,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":69.23584801543545,"status":"ok","temperature":1.0,"token_sha":"d813ade7fc53","ttft_seconds":0.092912002},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.72286765,"delivery_tps":68.49558565424694,"end_to_end_seconds":3.839050813,"error_code":null,"fixture_id":"chat","prefill_seconds":0.114075116,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":68.49331210322869,"status":"ok","temperature":1.0,"token_sha":"7675315607f0","ttft_seconds":0.115662301},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.510962061,"delivery_tps":101.55470047143815,"end_to_end_seconds":2.611563315,"error_code":null,"fixture_id":"code","prefill_seconds":0.098673421,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":101.55335534436598,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.100088602},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.502856842,"delivery_tps":101.88357389079947,"end_to_end_seconds":2.597287421,"error_code":null,"fixture_id":"code","prefill_seconds":0.09179928,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":101.85016154597912,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.094121749},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.525631071,"delivery_tps":100.96486495117244,"end_to_end_seconds":2.641316541,"error_code":null,"fixture_id":"code","prefill_seconds":0.113842104,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":100.94540622086969,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.115316383},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.479571621,"delivery_tps":102.84034461451034,"end_to_end_seconds":2.538006101,"error_code":null,"fixture_id":"code","prefill_seconds":0.056229296,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":102.82330302032602,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.05815271},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.497114522,"delivery_tps":102.11786353946005,"end_to_end_seconds":2.561751746,"error_code":null,"fixture_id":"code","prefill_seconds":0.062604535,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":102.10079157354336,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.064316305},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.250974187,"delivery_tps":78.43802667510998,"end_to_end_seconds":3.355127348,"error_code":null,"fixture_id":"chat","prefill_seconds":0.101222832,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":78.42744998246934,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.103859361},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.361044953,"delivery_tps":75.86926196044841,"end_to_end_seconds":3.465773003,"error_code":null,"fixture_id":"chat","prefill_seconds":0.102188539,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":75.8576053686123,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.104393002},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.223296169,"delivery_tps":79.11156363863131,"end_to_end_seconds":3.335259399,"error_code":null,"fixture_id":"chat","prefill_seconds":0.109445527,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":79.10203064606544,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.111606642},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.321044262,"delivery_tps":76.78307781614264,"end_to_end_seconds":3.470007536,"error_code":null,"fixture_id":"chat","prefill_seconds":0.146390853,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":76.76958425632151,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.148648645},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.245269776,"delivery_tps":78.57590203619485,"end_to_end_seconds":3.381288719,"error_code":null,"fixture_id":"chat","prefill_seconds":0.134429549,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":78.57404094259647,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.135674285}],"schema_version":1,"settings":{"context_tokens":null,"managed":false,"rank_count":null,"repetitions":5,"serial":false,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/out-r5b/35b-attached.log b/docs/receipts/drafter-counts/out-r5b/35b-attached.log new file mode 100644 index 0000000000..b5433f6cc6 --- /dev/null +++ b/docs/receipts/drafter-counts/out-r5b/35b-attached.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +code T=1 warm-up: ok 82.70 delivery tok/s +code T=1 repeat 1/5: ok 78.77 delivery tok/s +code T=1 repeat 2/5: ok 87.01 delivery tok/s +code T=1 repeat 3/5: ok 89.07 delivery tok/s +code T=1 repeat 4/5: ok 79.24 delivery tok/s +code T=1 repeat 5/5: ok 79.52 delivery tok/s +chat T=1 warm-up: ok 73.30 delivery tok/s +chat T=1 repeat 1/5: ok 70.11 delivery tok/s +chat T=1 repeat 2/5: ok 74.20 delivery tok/s +chat T=1 repeat 3/5: ok 70.57 delivery tok/s +chat T=1 repeat 4/5: ok 69.25 delivery tok/s +chat T=1 repeat 5/5: ok 68.50 delivery tok/s +code T=0 warm-up: ok 101.78 delivery tok/s +code T=0 repeat 1/5: ok 101.55 delivery tok/s +code T=0 repeat 2/5: ok 101.88 delivery tok/s +code T=0 repeat 3/5: ok 100.96 delivery tok/s +code T=0 repeat 4/5: ok 102.84 delivery tok/s +code T=0 repeat 5/5: ok 102.12 delivery tok/s +chat T=0 warm-up: ok 73.39 delivery tok/s +chat T=0 repeat 1/5: ok 78.44 delivery tok/s +chat T=0 repeat 2/5: ok 75.87 delivery tok/s +chat T=0 repeat 3/5: ok 79.11 delivery tok/s +chat T=0 repeat 4/5: ok 76.78 delivery tok/s +chat T=0 repeat 5/5: ok 78.58 delivery tok/s +Run 23c5e792-c249-4f55-904a-c2aef1070d16 | rocm | TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +Workload Temp Stream delivery tok/s First text, s Status +code 1 79.52 0.077 complete +chat 1 70.11 0.102 complete +code 0 101.88 0.094 complete +chat 0 78.44 0.112 complete +Stream delivery includes transport buffering; it is not an engine token clock. +The attached server's manifest does not attest a rank count and checkpoint revision; this result is unranked. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-runs/out-r5b/35b-attached.json +Draft counts: 2636 of 4576 drafted tokens accepted over 2473 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-runs/out-r5b/35b-attached.drafts.json diff --git a/docs/receipts/drafter-counts/out-r5b/35b-manifest.json b/docs/receipts/drafter-counts/out-r5b/35b-manifest.json new file mode 100644 index 0000000000..f945169c43 --- /dev/null +++ b/docs/receipts/drafter-counts/out-r5b/35b-manifest.json @@ -0,0 +1 @@ +{"schema_version": 1, "runtime": {"tensorfold_version": "1.0.12", "backend": "rocm", "python_version": "none", "platform": "linux"}, "model": {"revision": "f84b054c5677b6e59bdb969ae61915b0983d68bb", "config_sha256": "a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767", "tokenizer_sha256": "4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"}, "hardware": {"cpu_model": "AMD RYZEN AI MAX+ 395 with Radeon 8060S", "system_memory_bytes": 131209781248, "memory_type": "unified", "gpus": [{"name": "AMD Radeon 8060S Graphics", "memory_bytes": 128849018880, "cores": 40}], "gpu_count": 1, "source": "detected"}} \ No newline at end of file diff --git a/docs/receipts/drafter-counts/out-r5b/35b-serve.log b/docs/receipts/drafter-counts/out-r5b/35b-serve.log new file mode 100644 index 0000000000..08d2996a87 --- /dev/null +++ b/docs/receipts/drafter-counts/out-r5b/35b-serve.log @@ -0,0 +1,59 @@ +[tensorfold] HIP rank 0 of 1: policy matrix=auto,attention=auto,kernels=auto,graphs=auto,prefill_step=1024,fill_rows=512; mtp drafts 3 under 0.3, 8 snapshots +[tensorfold] open-file limit raised: soft 1024 -> 524288 +[tensorfold] HIP gfx1151 device 0, rank 0 of 1: model 18.71 GiB; 8 streams at once, 0.69 GiB each at a 32768-token window, of 84.4 GiB left after a 12.2 GiB reserve; prompt cache 77.24 GiB (62871 pages, 8 snapshots); prompts in 1024-row chunks +[tensorfold] serving amd-35b at http://127.0.0.1:8092/v1 (sampling: temperature 1.0, top_k 20, top_p 0.95; drafts: on; context: 32768; loaded in 43.5s) +[tensorfold] 127.0.0.1 "GET /v1/benchmark/manifest HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/benchmark/manifest HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-c0d885ed8529d3103c371745e93c2960 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5d7effdcf06f finish=length rounds=106 accepted=150/248 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-42cb3647d1350e43794428cd930fa548 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5d7effdcf06f finish=length rounds=102 accepted=154/264 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-8d0b719ca3724f339a4ceb4e90645ea9 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=bd3964ef7930 finish=length rounds=89 accepted=166/249 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-1e9f070f6d1be3507c865aa7d22431d2 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=bc55cc799f87 finish=length rounds=89 accepted=166/228 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-a29860269a9585c55acf222d9432aa22 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=306e0fe05bc2 finish=length rounds=118 accepted=138/220 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-21afea1a90951b87b242e02b2a0e236b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5a89476cf1f5 finish=length rounds=111 accepted=144/250 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d0b184cdb952c75350a292e74bcb3226 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=460edda16244 finish=length rounds=152 accepted=103/172 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-26d6191b931b254f55f48673d52ff400 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=460edda16244 finish=length rounds=141 accepted=114/209 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1e1968dbdc59a5c3007fb11798219079 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=0eba9c9d3070 finish=length rounds=112 accepted=144/252 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-483ba283e0de600b27774b09881ab007 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=15d18697d5f1 finish=length rounds=159 accepted=96/170 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-c97363ef32b24caeab949856eff4cb1a prompt=31 cached=0 thinking=False effort=none tokens=256 sha=d813ade7fc53 finish=length rounds=201 accepted=54/97 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d8f7bc52a258b85e726897b50ab01d69 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=7675315607f0 finish=length rounds=201 accepted=55/105 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-2f45b558de15475ff1516a723d309758 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4bdf6ae20a440eea6f56c469e0f2d5e1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-8d28644083b8e172b19bb9334990534b prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-1ef81db777ca1141a6d05bb7ccd105b0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-1c8708a1866e78781ffca5257591178d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=82 accepted=173/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-eb9325a6f7b2e36304b8dde6c2273455 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-9df108317c5aba009c0b0833241bc9b0 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=191 accepted=65/140 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-07fcec2c48a10e12aad5fa8fa3ff1a2a prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=142 accepted=114/218 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1e513af7badb380246e6212925f9707d prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=172 accepted=84/168 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-10b388202127bbdafb268c848b9a2597 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=134 accepted=122/251 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-89e37005e7550e83e88fe3bd5b8a976f prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=146 accepted=110/235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-32e6b75026ad6f582ba658cda730c1fa prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=142 accepted=114/210 +info: backend verify: 3005 calls, 21393 us each +info: backend keep: 1323 calls, 12 us each +info: backend draft: 3029 calls, 3530 us each +info: head graphs: 4 captured, 1186 of 1190 batches replayed +info: graphs: 17 captured (5 ms each), 3156 of 3173 rounds replayed diff --git a/docs/receipts/drafter-counts/out-r5b/heads-r5b.txt b/docs/receipts/drafter-counts/out-r5b/heads-r5b.txt new file mode 100644 index 0000000000..80629f6110 --- /dev/null +++ b/docs/receipts/drafter-counts/out-r5b/heads-r5b.txt @@ -0,0 +1,3 @@ +25a7201f Merge branch 'benchmark-rocm-host-identity' of https://github.com/ThinkOffApp/TensorFold into tf1001-gpu +624e2ce6 Benchmark: a HIP or Metal server's manifest names the receipt's backend +7b84f909 Merge remote-tracking branches 'origin/hip-refuses-drafter' and 'origin/benchmark-drafter-counts' into tf1001-gpu diff --git a/docs/receipts/drafter-counts/out-r5b/run-r5b.log b/docs/receipts/drafter-counts/out-r5b/run-r5b.log new file mode 100644 index 0000000000..8585a5db02 --- /dev/null +++ b/docs/receipts/drafter-counts/out-r5b/run-r5b.log @@ -0,0 +1,7 @@ +Auto-merging docs/community-benchmarks.md +build_exit=0 +7fcaa74ec93561902ef97675f00db1b6d022392c665a7578960e18c92bcd2407 ~/tf-1001/zig-out/native-hip/bin/tensorfold-native +R5b exit=0 +RUNDONE +llm-server: active +llm-server chat OK after 210s diff --git a/docs/receipts/drafter-counts/out/27b-auto.drafts.json b/docs/receipts/drafter-counts/out/27b-auto.drafts.json new file mode 100644 index 0000000000..442eff9e4a --- /dev/null +++ b/docs/receipts/drafter-counts/out/27b-auto.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":false,"drafter_repo_id":null,"receipt":"27b-auto.json","run_id":"c33268ac-a928-40da-a87b-9963f8260cec","samples":[{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":0,"rounds":254,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":1,"rounds":255,"status":"ok","temperature":1.0},{"accepted":21,"completion_tokens":256,"drafted":48,"fixture_id":"code","repeat":2,"rounds":234,"status":"ok","temperature":1.0},{"accepted":6,"completion_tokens":256,"drafted":34,"fixture_id":"code","repeat":3,"rounds":249,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":4,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":0,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":1,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":2,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":3,"rounds":255,"status":"ok","temperature":1.0},{"accepted":2,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":4,"rounds":253,"status":"ok","temperature":1.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":0,"rounds":254,"status":"ok","temperature":0.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":1,"rounds":254,"status":"ok","temperature":0.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":2,"rounds":254,"status":"ok","temperature":0.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":3,"rounds":254,"status":"ok","temperature":0.0},{"accepted":1,"completion_tokens":256,"drafted":15,"fixture_id":"code","repeat":4,"rounds":254,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":0,"rounds":248,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":1,"rounds":248,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":2,"rounds":248,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":3,"rounds":248,"status":"ok","temperature":0.0},{"accepted":7,"completion_tokens":256,"drafted":15,"fixture_id":"chat","repeat":4,"rounds":248,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":70,"attempts":20,"drafted":262,"reported":20,"rounds":5030}} diff --git a/docs/receipts/drafter-counts/out/27b-auto.json b/docs/receipts/drafter-counts/out/27b-auto.json new file mode 100644 index 0000000000..e565dd215e --- /dev/null +++ b/docs/receipts/drafter-counts/out/27b-auto.json @@ -0,0 +1 @@ +{"created_at":"2026-10-18T17:42:02.550299Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"14b65a0ee06517060a6bbd979bb1a8ff54e7b304b1a1f01d54344b88b8285e85","drafter_repo_id":null,"family":"qwen3_5","local":false,"quantization":"mlx-affine-4bit-g64","repo_id":"TensorFold/Qwen3.8-27B-MLX-4bit","revision":"70ae7fac63274ff2eac54152031433374cb80f2f","tokenizer_sha256":"49d8f5dc9c3e0a89091f4f189ecfd308c8f0250929b815e125d4700bded4df2c"},"run_id":"c33268ac-a928-40da-a87b-9963f8260cec","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.903755167,"delivery_tps":12.811652769060611,"end_to_end_seconds":20.146377346,"error_code":null,"fixture_id":"code","prefill_seconds":0.239668512,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":12.811316595952226,"status":"ok","temperature":1.0,"token_sha":"94624c419588","ttft_seconds":0.242095358},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":20.285006059,"delivery_tps":12.570861416472797,"end_to_end_seconds":20.524461929,"error_code":null,"fixture_id":"code","prefill_seconds":0.237587234,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":12.570792048154868,"status":"ok","temperature":1.0,"token_sha":"4e0a0a5f5689","ttft_seconds":0.239065053},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":18.86791311,"delivery_tps":13.5150081788775,"end_to_end_seconds":19.122833572,"error_code":null,"fixture_id":"code","prefill_seconds":0.25297176,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":13.514924920740311,"status":"ok","temperature":1.0,"token_sha":"ed99f8b35926","ttft_seconds":0.254460918},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.860349375,"delivery_tps":12.839653280268642,"end_to_end_seconds":20.106354547,"error_code":null,"fixture_id":"code","prefill_seconds":0.2431791,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":12.839322306980971,"status":"ok","temperature":1.0,"token_sha":"42b152d2efcc","ttft_seconds":0.245406803},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.808580603,"delivery_tps":12.873209096131824,"end_to_end_seconds":20.056947924,"error_code":null,"fixture_id":"code","prefill_seconds":0.245725482,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":12.873062015571138,"status":"ok","temperature":1.0,"token_sha":"f9ef9076534b","ttft_seconds":0.248004393},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.809258081,"delivery_tps":12.872768831488072,"end_to_end_seconds":20.305741719,"error_code":null,"fixture_id":"chat","prefill_seconds":0.49413369,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":12.872719602051264,"status":"ok","temperature":1.0,"token_sha":"388b0ee93e9f","ttft_seconds":0.495791022},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.803966824,"delivery_tps":12.876208199408364,"end_to_end_seconds":20.325205822,"error_code":null,"fixture_id":"chat","prefill_seconds":0.518820272,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":12.876097269993101,"status":"ok","temperature":1.0,"token_sha":"6a931bb6524d","ttft_seconds":0.520883761},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.80579648,"delivery_tps":12.875018697556564,"end_to_end_seconds":20.299056921,"error_code":null,"fixture_id":"chat","prefill_seconds":0.491224782,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":12.874983230858177,"status":"ok","temperature":1.0,"token_sha":"90107a7ec252","ttft_seconds":0.492907113},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.801318935,"delivery_tps":12.8779300427949,"end_to_end_seconds":20.29418779,"error_code":null,"fixture_id":"chat","prefill_seconds":0.490269684,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":12.877867733675195,"status":"ok","temperature":1.0,"token_sha":"2dab910075e1","ttft_seconds":0.492228067},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.877419162,"delivery_tps":12.828627193588988,"end_to_end_seconds":20.390664197,"error_code":null,"fixture_id":"chat","prefill_seconds":0.510393965,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":12.828093925409993,"status":"ok","temperature":1.0,"token_sha":"b1aba87c5a14","ttft_seconds":0.512769042},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.85645107,"delivery_tps":12.842174016950352,"end_to_end_seconds":20.10626902,"error_code":null,"fixture_id":"code","prefill_seconds":0.247056896,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":12.841844472284416,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.249392374},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.852772758,"delivery_tps":12.84455340865389,"end_to_end_seconds":20.102390354,"error_code":null,"fixture_id":"code","prefill_seconds":0.24749416,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":12.844214923842012,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.249258889},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.875707891,"delivery_tps":12.829731720673333,"end_to_end_seconds":20.124128539,"error_code":null,"fixture_id":"code","prefill_seconds":0.245450701,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":12.829398212092743,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.247752721},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.85645976,"delivery_tps":12.842168396689058,"end_to_end_seconds":20.104480532,"error_code":null,"fixture_id":"code","prefill_seconds":0.245327385,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":12.841827839377876,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.247428063},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.860533468,"delivery_tps":12.839534265827506,"end_to_end_seconds":20.108976676,"error_code":null,"fixture_id":"code","prefill_seconds":0.246409828,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":12.839478851323637,"status":"ok","temperature":0.0,"token_sha":"2b28aa2108de","ttft_seconds":0.248107459},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.439866206,"delivery_tps":13.117374229730848,"end_to_end_seconds":19.936563847,"error_code":null,"fixture_id":"chat","prefill_seconds":0.493991165,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":13.117315278328896,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.495944867},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.42160197,"delivery_tps":13.129709917538793,"end_to_end_seconds":19.91775181,"error_code":null,"fixture_id":"chat","prefill_seconds":0.493917198,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":13.12941917762138,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.495757104},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.442134316,"delivery_tps":13.115843963188059,"end_to_end_seconds":19.943971719,"error_code":null,"fixture_id":"chat","prefill_seconds":0.499177995,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":13.115375102223444,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.501499024},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.41672064,"delivery_tps":13.133010703912563,"end_to_end_seconds":19.921349835,"error_code":null,"fixture_id":"chat","prefill_seconds":0.502336755,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":13.132676445197223,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.504314446},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":19.425647849,"delivery_tps":13.126975325722636,"end_to_end_seconds":19.921862237,"error_code":null,"fixture_id":"chat","prefill_seconds":0.494480838,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":13.126935011664902,"status":"ok","temperature":0.0,"token_sha":"a06fab348fbe","ttft_seconds":0.495917118}],"schema_version":1,"settings":{"context_tokens":null,"managed":true,"rank_count":1,"repetitions":5,"serial":false,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/out/27b-auto.log b/docs/receipts/drafter-counts/out/27b-auto.log new file mode 100644 index 0000000000..4bee460aff --- /dev/null +++ b/docs/receipts/drafter-counts/out/27b-auto.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +This run passes no drafter: the HIP engine runs no DFlash2 drafter yet. +code T=1 warm-up: ok 12.81 delivery tok/s +code T=1 repeat 1/5: ok 12.81 delivery tok/s +code T=1 repeat 2/5: ok 12.57 delivery tok/s +code T=1 repeat 3/5: ok 13.52 delivery tok/s +code T=1 repeat 4/5: ok 12.84 delivery tok/s +code T=1 repeat 5/5: ok 12.87 delivery tok/s +chat T=1 warm-up: ok 12.89 delivery tok/s +chat T=1 repeat 1/5: ok 12.87 delivery tok/s +chat T=1 repeat 2/5: ok 12.88 delivery tok/s +chat T=1 repeat 3/5: ok 12.88 delivery tok/s +chat T=1 repeat 4/5: ok 12.88 delivery tok/s +chat T=1 repeat 5/5: ok 12.83 delivery tok/s +code T=0 warm-up: ok 12.85 delivery tok/s +code T=0 repeat 1/5: ok 12.84 delivery tok/s +code T=0 repeat 2/5: ok 12.84 delivery tok/s +code T=0 repeat 3/5: ok 12.83 delivery tok/s +code T=0 repeat 4/5: ok 12.84 delivery tok/s +code T=0 repeat 5/5: ok 12.84 delivery tok/s +chat T=0 warm-up: ok 13.13 delivery tok/s +chat T=0 repeat 1/5: ok 13.12 delivery tok/s +chat T=0 repeat 2/5: ok 13.13 delivery tok/s +chat T=0 repeat 3/5: ok 13.12 delivery tok/s +chat T=0 repeat 4/5: ok 13.13 delivery tok/s +chat T=0 repeat 5/5: ok 13.13 delivery tok/s +Run c33268ac-a928-40da-a87b-9963f8260cec | rocm | TensorFold/Qwen3.8-27B-MLX-4bit +Workload Temp Stream delivery tok/s First text, s Status +code 1 12.84 0.245 complete +chat 1 12.88 0.496 complete +code 0 12.84 0.248 complete +chat 0 13.13 0.496 complete +Stream delivery includes transport buffering; it is not an engine token clock. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-runs/out/27b-auto.json +Draft counts: 70 of 262 drafted tokens accepted over 5030 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-runs/out/27b-auto.drafts.json diff --git a/docs/receipts/drafter-counts/out/27b-dflash2.log b/docs/receipts/drafter-counts/out/27b-dflash2.log new file mode 100644 index 0000000000..83a51751ba --- /dev/null +++ b/docs/receipts/drafter-counts/out/27b-dflash2.log @@ -0,0 +1 @@ +tensorfold benchmark: --drafter: the HIP engine runs no DFlash2 drafter yet, so a rocm run takes none; --drafter none runs without it diff --git a/docs/receipts/drafter-counts/out/27b-serve-drafter.log b/docs/receipts/drafter-counts/out/27b-serve-drafter.log new file mode 100644 index 0000000000..40d7c5404e --- /dev/null +++ b/docs/receipts/drafter-counts/out/27b-serve-drafter.log @@ -0,0 +1 @@ +tensorfold: --drafter: the HIP engine runs no DFlash2 drafter yet; serve without --drafter diff --git a/docs/receipts/drafter-counts/out/35b-attached.drafts.json b/docs/receipts/drafter-counts/out/35b-attached.drafts.json new file mode 100644 index 0000000000..85de76df20 --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b-attached.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":null,"drafter_repo_id":null,"receipt":"35b-attached.json","run_id":"8381ef4d-d37c-4276-9b47-9b8dfd777cb7","samples":[{"accepted":150,"completion_tokens":256,"drafted":246,"fixture_id":"code","repeat":0,"rounds":106,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":249,"fixture_id":"code","repeat":1,"rounds":89,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":228,"fixture_id":"code","repeat":2,"rounds":89,"status":"ok","temperature":1.0},{"accepted":138,"completion_tokens":256,"drafted":220,"fixture_id":"code","repeat":3,"rounds":118,"status":"ok","temperature":1.0},{"accepted":136,"completion_tokens":256,"drafted":218,"fixture_id":"code","repeat":4,"rounds":119,"status":"ok","temperature":1.0},{"accepted":114,"completion_tokens":256,"drafted":210,"fixture_id":"chat","repeat":0,"rounds":141,"status":"ok","temperature":1.0},{"accepted":144,"completion_tokens":256,"drafted":252,"fixture_id":"chat","repeat":1,"rounds":112,"status":"ok","temperature":1.0},{"accepted":90,"completion_tokens":256,"drafted":150,"fixture_id":"chat","repeat":2,"rounds":165,"status":"ok","temperature":1.0},{"accepted":120,"completion_tokens":256,"drafted":213,"fixture_id":"chat","repeat":3,"rounds":136,"status":"ok","temperature":1.0},{"accepted":125,"completion_tokens":256,"drafted":242,"fixture_id":"chat","repeat":4,"rounds":131,"status":"ok","temperature":1.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":0,"rounds":83,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":1,"rounds":83,"status":"ok","temperature":0.0},{"accepted":172,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":2,"rounds":83,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":290,"fixture_id":"code","repeat":3,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":4,"rounds":82,"status":"ok","temperature":0.0},{"accepted":113,"completion_tokens":256,"drafted":235,"fixture_id":"chat","repeat":0,"rounds":143,"status":"ok","temperature":0.0},{"accepted":48,"completion_tokens":256,"drafted":117,"fixture_id":"chat","repeat":1,"rounds":207,"status":"ok","temperature":0.0},{"accepted":108,"completion_tokens":256,"drafted":202,"fixture_id":"chat","repeat":2,"rounds":148,"status":"ok","temperature":0.0},{"accepted":121,"completion_tokens":256,"drafted":246,"fixture_id":"chat","repeat":3,"rounds":135,"status":"ok","temperature":0.0},{"accepted":67,"completion_tokens":256,"drafted":145,"fixture_id":"chat","repeat":4,"rounds":189,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":2668,"attempts":20,"drafted":4626,"reported":20,"rounds":2441}} diff --git a/docs/receipts/drafter-counts/out/35b-attached.json b/docs/receipts/drafter-counts/out/35b-attached.json new file mode 100644 index 0000000000..f77e6dc641 --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b-attached.json @@ -0,0 +1 @@ +{"created_at":"2026-10-18T17:47:00.563138Z","hardware":{"cpu_model":null,"gpu_count":null,"gpus":[],"memory_type":"unknown","source":"unavailable","system_memory_bytes":null},"model":{"config_sha256":null,"drafter_repo_id":null,"family":null,"local":false,"quantization":null,"repo_id":"TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP","revision":null,"tokenizer_sha256":null},"run_id":"8381ef4d-d37c-4276-9b47-9b8dfd777cb7","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"unknown","tensorfold_version":"unknown"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.243378231,"delivery_tps":78.62172766738287,"end_to_end_seconds":3.302250066,"error_code":null,"fixture_id":"code","prefill_seconds":0.057585289,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":78.62041146840825,"status":"ok","temperature":1.0,"token_sha":"5d7effdcf06f","ttft_seconds":0.058603384},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.978453736,"delivery_tps":85.61489370066884,"end_to_end_seconds":3.040722942,"error_code":null,"fixture_id":"code","prefill_seconds":0.060811547,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":85.61376953719021,"status":"ok","temperature":1.0,"token_sha":"bd3964ef7930","ttft_seconds":0.062018675},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.893639504,"delivery_tps":88.1243152948053,"end_to_end_seconds":2.953564602,"error_code":null,"fixture_id":"code","prefill_seconds":0.058518767,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":88.1218867075229,"status":"ok","temperature":1.0,"token_sha":"bc55cc799f87","ttft_seconds":0.059632448},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.261464412,"delivery_tps":78.18573738280607,"end_to_end_seconds":3.352942274,"error_code":null,"fixture_id":"code","prefill_seconds":0.089279378,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":78.17404187683177,"status":"ok","temperature":1.0,"token_sha":"306e0fe05bc2","ttft_seconds":0.090984739},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.252107398,"delivery_tps":78.410694602774,"end_to_end_seconds":3.352111473,"error_code":null,"fixture_id":"code","prefill_seconds":0.097018668,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":78.38847687963506,"status":"ok","temperature":1.0,"token_sha":"5a89476cf1f5","ttft_seconds":0.099439085},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.673197325,"delivery_tps":69.42180815183949,"end_to_end_seconds":3.81150248,"error_code":null,"fixture_id":"chat","prefill_seconds":0.135500551,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":69.4060938540061,"status":"ok","temperature":1.0,"token_sha":"460edda16244","ttft_seconds":0.137710275},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.472544189,"delivery_tps":73.43319080222653,"end_to_end_seconds":3.579571649,"error_code":null,"fixture_id":"chat","prefill_seconds":0.104786219,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":73.42358033289602,"status":"ok","temperature":1.0,"token_sha":"0eba9c9d3070","ttft_seconds":0.106693052},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.516765132,"delivery_tps":72.50981809381747,"end_to_end_seconds":3.611294276,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092850914,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":72.50699875013886,"status":"ok","temperature":1.0,"token_sha":"15d18697d5f1","ttft_seconds":0.094055901},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.626211939,"delivery_tps":70.32131720086976,"end_to_end_seconds":3.719590424,"error_code":null,"fixture_id":"chat","prefill_seconds":0.090951429,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":70.3081347889163,"status":"ok","temperature":1.0,"token_sha":"d813ade7fc53","ttft_seconds":0.092991089},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.740061093,"delivery_tps":68.18070444818801,"end_to_end_seconds":3.853783732,"error_code":null,"fixture_id":"chat","prefill_seconds":0.112197111,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":68.17870914153332,"status":"ok","temperature":1.0,"token_sha":"7675315607f0","ttft_seconds":0.113449128},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.539497059,"delivery_tps":100.41358350712703,"end_to_end_seconds":2.63675358,"error_code":null,"fixture_id":"code","prefill_seconds":0.094807405,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":100.41118075316031,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.096286124},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.533257186,"delivery_tps":100.66092041868187,"end_to_end_seconds":2.632491988,"error_code":null,"fixture_id":"code","prefill_seconds":0.096488327,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":100.63805330902348,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.098856445},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.563490995,"delivery_tps":99.47372567228386,"end_to_end_seconds":2.664991688,"error_code":null,"fixture_id":"code","prefill_seconds":0.099719195,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":99.47020250953632,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.100818457},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.510903393,"delivery_tps":101.55707332703422,"end_to_end_seconds":2.580178065,"error_code":null,"fixture_id":"code","prefill_seconds":0.066937384,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":101.53330088286565,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.069008132},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.501479191,"delivery_tps":101.93968469434292,"end_to_end_seconds":2.557375648,"error_code":null,"fixture_id":"code","prefill_seconds":0.053957035,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":101.93776585224423,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055007038},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.309267232,"delivery_tps":77.05633366027298,"end_to_end_seconds":3.428299676,"error_code":null,"fixture_id":"chat","prefill_seconds":0.117582335,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":77.05459714963683,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.118747854},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.641391661,"delivery_tps":70.02817157272553,"end_to_end_seconds":3.7573917,"error_code":null,"fixture_id":"chat","prefill_seconds":0.114361886,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":70.02573593536196,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.11568451},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.276768971,"delivery_tps":77.82056112493626,"end_to_end_seconds":3.403395281,"error_code":null,"fixture_id":"chat","prefill_seconds":0.124187325,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":77.80986231003247,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.126139917},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.299548019,"delivery_tps":77.28331229962924,"end_to_end_seconds":3.423608808,"error_code":null,"fixture_id":"chat","prefill_seconds":0.122055129,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":77.28079925220916,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.123800468},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.538275495,"delivery_tps":72.06900659949883,"end_to_end_seconds":3.6730617,"error_code":null,"fixture_id":"chat","prefill_seconds":0.132858633,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":72.06727538828737,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.13408949}],"schema_version":1,"settings":{"context_tokens":null,"managed":false,"rank_count":null,"repetitions":5,"serial":false,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/out/35b-attached.log b/docs/receipts/drafter-counts/out/35b-attached.log new file mode 100644 index 0000000000..589809167c --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b-attached.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +code T=1 warm-up: ok 83.54 delivery tok/s +code T=1 repeat 1/5: ok 78.62 delivery tok/s +code T=1 repeat 2/5: ok 85.61 delivery tok/s +code T=1 repeat 3/5: ok 88.12 delivery tok/s +code T=1 repeat 4/5: ok 78.19 delivery tok/s +code T=1 repeat 5/5: ok 78.41 delivery tok/s +chat T=1 warm-up: ok 73.76 delivery tok/s +chat T=1 repeat 1/5: ok 69.42 delivery tok/s +chat T=1 repeat 2/5: ok 73.43 delivery tok/s +chat T=1 repeat 3/5: ok 72.51 delivery tok/s +chat T=1 repeat 4/5: ok 70.32 delivery tok/s +chat T=1 repeat 5/5: ok 68.18 delivery tok/s +code T=0 warm-up: ok 100.00 delivery tok/s +code T=0 repeat 1/5: ok 100.41 delivery tok/s +code T=0 repeat 2/5: ok 100.66 delivery tok/s +code T=0 repeat 3/5: ok 99.47 delivery tok/s +code T=0 repeat 4/5: ok 101.56 delivery tok/s +code T=0 repeat 5/5: ok 101.94 delivery tok/s +chat T=0 warm-up: ok 78.82 delivery tok/s +chat T=0 repeat 1/5: ok 77.06 delivery tok/s +chat T=0 repeat 2/5: ok 70.03 delivery tok/s +chat T=0 repeat 3/5: ok 77.82 delivery tok/s +chat T=0 repeat 4/5: ok 77.28 delivery tok/s +chat T=0 repeat 5/5: ok 72.07 delivery tok/s +Run 8381ef4d-d37c-4276-9b47-9b8dfd777cb7 | rocm | TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +Workload Temp Stream delivery tok/s First text, s Status +code 1 78.62 0.062 complete +chat 1 70.32 0.107 complete +code 0 100.66 0.096 complete +chat 0 77.06 0.124 complete +Stream delivery includes transport buffering; it is not an engine token clock. +Attached-server hardware and checkpoint revisions are unknown; this result is unranked. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-runs/out/35b-attached.json +Draft counts: 2668 of 4626 drafted tokens accepted over 2441 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-runs/out/35b-attached.drafts.json diff --git a/docs/receipts/drafter-counts/out/35b-manifest.json b/docs/receipts/drafter-counts/out/35b-manifest.json new file mode 100644 index 0000000000..428da3dc8b --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b-manifest.json @@ -0,0 +1 @@ +{"schema_version": 1, "runtime": {"tensorfold_version": "1.0.12", "backend": "hip", "python_version": "none", "platform": "linux"}, "model": {"revision": "f84b054c5677b6e59bdb969ae61915b0983d68bb", "config_sha256": "a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767", "tokenizer_sha256": "4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"}, "hardware": {"cpu_model": "AMD RYZEN AI MAX+ 395 with Radeon 8060S", "system_memory_bytes": 131209781248, "memory_type": "unknown", "gpus": [], "gpu_count": null, "source": "partial"}} \ No newline at end of file diff --git a/docs/receipts/drafter-counts/out/35b-serial.drafts.json b/docs/receipts/drafter-counts/out/35b-serial.drafts.json new file mode 100644 index 0000000000..65f63f88ae --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b-serial.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":false,"drafter_repo_id":null,"receipt":"35b-serial.json","run_id":"94ed04e1-d318-482a-8152-ca9588272c44","samples":[{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":0,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":1,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":2,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":3,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":4,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":0,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":1,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":2,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":3,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":4,"rounds":255,"status":"ok","temperature":1.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":0,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":1,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":2,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":3,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"code","repeat":4,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":0,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":1,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":2,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":3,"rounds":255,"status":"ok","temperature":0.0},{"accepted":0,"completion_tokens":256,"drafted":0,"fixture_id":"chat","repeat":4,"rounds":255,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":0,"attempts":20,"drafted":0,"reported":20,"rounds":5100}} diff --git a/docs/receipts/drafter-counts/out/35b-serial.json b/docs/receipts/drafter-counts/out/35b-serial.json new file mode 100644 index 0000000000..796f3f1676 --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b-serial.json @@ -0,0 +1 @@ +{"created_at":"2026-10-18T17:45:19.821863Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767","drafter_repo_id":null,"family":"qwen3_5_moe","local":false,"quantization":"mlx-affine-4bit-g64","repo_id":"TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP","revision":"f84b054c5677b6e59bdb969ae61915b0983d68bb","tokenizer_sha256":"4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"},"run_id":"94ed04e1-d318-482a-8152-ca9588272c44","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.759774118,"delivery_tps":67.82322341631695,"end_to_end_seconds":3.812049981,"error_code":null,"fixture_id":"code","prefill_seconds":0.05065055,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":67.82192521585628,"status":"ok","temperature":1.0,"token_sha":"5d7effdcf06f","ttft_seconds":0.05178199},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.765770599,"delivery_tps":67.71522409456253,"end_to_end_seconds":3.818825506,"error_code":null,"fixture_id":"code","prefill_seconds":0.050688539,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":67.70774619405101,"status":"ok","temperature":1.0,"token_sha":"bd3964ef7930","ttft_seconds":0.052756407},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.750469511,"delivery_tps":67.99148726635256,"end_to_end_seconds":3.802543212,"error_code":null,"fixture_id":"code","prefill_seconds":0.050120338,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":67.99089375296336,"status":"ok","temperature":1.0,"token_sha":"bc55cc799f87","ttft_seconds":0.051782331},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.757577024,"delivery_tps":67.86288035382664,"end_to_end_seconds":3.810577822,"error_code":null,"fixture_id":"code","prefill_seconds":0.051629016,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":67.86147625055469,"status":"ok","temperature":1.0,"token_sha":"306e0fe05bc2","ttft_seconds":0.052721208},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.76430671,"delivery_tps":67.74155764794203,"end_to_end_seconds":3.816769987,"error_code":null,"fixture_id":"code","prefill_seconds":0.051200081,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":67.74056719382524,"status":"ok","temperature":1.0,"token_sha":"5a89476cf1f5","ttft_seconds":0.052168078},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.774047542,"delivery_tps":67.56671641313416,"end_to_end_seconds":3.860697371,"error_code":null,"fixture_id":"chat","prefill_seconds":0.084277281,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":67.55822852857109,"status":"ok","temperature":1.0,"token_sha":"460edda16244","ttft_seconds":0.086375769},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.777132115,"delivery_tps":67.51153844667675,"end_to_end_seconds":3.863561471,"error_code":null,"fixture_id":"chat","prefill_seconds":0.084966138,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":67.51080441523933,"status":"ok","temperature":1.0,"token_sha":"0eba9c9d3070","ttft_seconds":0.086157876},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.768336651,"delivery_tps":67.66911335597653,"end_to_end_seconds":3.856368231,"error_code":null,"fixture_id":"chat","prefill_seconds":0.085810948,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":67.65989961192439,"status":"ok","temperature":1.0,"token_sha":"15d18697d5f1","ttft_seconds":0.08775632},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.768488215,"delivery_tps":67.66639178676587,"end_to_end_seconds":3.855664985,"error_code":null,"fixture_id":"chat","prefill_seconds":0.085679503,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":67.66519007724261,"status":"ok","temperature":1.0,"token_sha":"d813ade7fc53","ttft_seconds":0.086873521},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.787361919,"delivery_tps":67.32918729544843,"end_to_end_seconds":3.873478157,"error_code":null,"fixture_id":"chat","prefill_seconds":0.084100217,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":67.32745819217918,"status":"ok","temperature":1.0,"token_sha":"7675315607f0","ttft_seconds":0.08576239},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.68833721,"delivery_tps":69.13684554346916,"end_to_end_seconds":3.741133435,"error_code":null,"fixture_id":"code","prefill_seconds":0.050393629,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":69.13407940220092,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.052297563},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.687622845,"delivery_tps":69.15023870886124,"end_to_end_seconds":3.739775363,"error_code":null,"fixture_id":"code","prefill_seconds":0.050492205,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":69.14692100479135,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.051910066},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.683352113,"delivery_tps":69.23041625588945,"end_to_end_seconds":3.735806331,"error_code":null,"fixture_id":"code","prefill_seconds":0.050909091,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":69.22959360576273,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.051924666},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.690186125,"delivery_tps":69.10220551544673,"end_to_end_seconds":3.74245429,"error_code":null,"fixture_id":"code","prefill_seconds":0.049272778,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":69.08648303003007,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.051699174},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.683403752,"delivery_tps":69.22944568906982,"end_to_end_seconds":3.73556045,"error_code":null,"fixture_id":"code","prefill_seconds":0.050575493,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":69.22866499260533,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.051836489},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.699669996,"delivery_tps":68.92506636421633,"end_to_end_seconds":3.786365283,"error_code":null,"fixture_id":"chat","prefill_seconds":0.08520682,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":68.92445886201169,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.0863457},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.702860125,"delivery_tps":68.86568527888966,"end_to_end_seconds":3.789798745,"error_code":null,"fixture_id":"chat","prefill_seconds":0.08433075,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":68.85634621675966,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.08636053},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.706321415,"delivery_tps":68.80137242495468,"end_to_end_seconds":3.793343091,"error_code":null,"fixture_id":"chat","prefill_seconds":0.084680788,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":68.79199866677762,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.086742156},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.701143705,"delivery_tps":68.89762201222068,"end_to_end_seconds":3.788443451,"error_code":null,"fixture_id":"chat","prefill_seconds":0.084888201,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":68.88600196629339,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.086986137},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.697890868,"delivery_tps":68.95822756876447,"end_to_end_seconds":3.783991086,"error_code":null,"fixture_id":"chat","prefill_seconds":0.083931734,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":68.94859961392461,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.085842167}],"schema_version":1,"settings":{"context_tokens":null,"managed":true,"rank_count":1,"repetitions":5,"serial":true,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/out/35b-serial.log b/docs/receipts/drafter-counts/out/35b-serial.log new file mode 100644 index 0000000000..ca4de65088 --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b-serial.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +The server gets up to 606 s to load 21 GB of weights; --startup-timeout sets another limit. +code T=1 warm-up: ok 67.56 delivery tok/s +code T=1 repeat 1/5: ok 67.82 delivery tok/s +code T=1 repeat 2/5: ok 67.72 delivery tok/s +code T=1 repeat 3/5: ok 67.99 delivery tok/s +code T=1 repeat 4/5: ok 67.86 delivery tok/s +code T=1 repeat 5/5: ok 67.74 delivery tok/s +chat T=1 warm-up: ok 67.60 delivery tok/s +chat T=1 repeat 1/5: ok 67.57 delivery tok/s +chat T=1 repeat 2/5: ok 67.51 delivery tok/s +chat T=1 repeat 3/5: ok 67.67 delivery tok/s +chat T=1 repeat 4/5: ok 67.67 delivery tok/s +chat T=1 repeat 5/5: ok 67.33 delivery tok/s +code T=0 warm-up: ok 69.12 delivery tok/s +code T=0 repeat 1/5: ok 69.14 delivery tok/s +code T=0 repeat 2/5: ok 69.15 delivery tok/s +code T=0 repeat 3/5: ok 69.23 delivery tok/s +code T=0 repeat 4/5: ok 69.10 delivery tok/s +code T=0 repeat 5/5: ok 69.23 delivery tok/s +chat T=0 warm-up: ok 68.94 delivery tok/s +chat T=0 repeat 1/5: ok 68.93 delivery tok/s +chat T=0 repeat 2/5: ok 68.87 delivery tok/s +chat T=0 repeat 3/5: ok 68.80 delivery tok/s +chat T=0 repeat 4/5: ok 68.90 delivery tok/s +chat T=0 repeat 5/5: ok 68.96 delivery tok/s +Run 94ed04e1-d318-482a-8152-ca9588272c44 | rocm | TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +Workload Temp Stream delivery tok/s First text, s Status +code 1 67.82 0.052 complete +chat 1 67.57 0.086 complete +code 0 69.15 0.052 complete +chat 0 68.90 0.086 complete +Stream delivery includes transport buffering; it is not an engine token clock. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-runs/out/35b-serial.json +Draft counts: 0 of 0 drafted tokens accepted over 5100 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-runs/out/35b-serial.drafts.json diff --git a/docs/receipts/drafter-counts/out/35b-serve.log b/docs/receipts/drafter-counts/out/35b-serve.log new file mode 100644 index 0000000000..e88b47d49d --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b-serve.log @@ -0,0 +1,59 @@ +[tensorfold] HIP rank 0 of 1: policy matrix=auto,attention=auto,kernels=auto,graphs=auto,prefill_step=1024,fill_rows=512; mtp drafts 3 under 0.3, 8 snapshots +[tensorfold] open-file limit raised: soft 1024 -> 524288 +[tensorfold] HIP gfx1151 device 0, rank 0 of 1: model 18.71 GiB; 8 streams at once, 0.69 GiB each at a 32768-token window, of 84.7 GiB left after a 12.2 GiB reserve; prompt cache 77.54 GiB (63118 pages, 8 snapshots); prompts in 1024-row chunks +[tensorfold] serving amd-35b at http://127.0.0.1:8092/v1 (sampling: temperature 1.0, top_k 20, top_p 0.95; drafts: on; context: 32768; loaded in 20.3s) +[tensorfold] 127.0.0.1 "GET /v1/benchmark/manifest HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "GET /v1/benchmark/manifest HTTP/1.1" 200 - +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-16ae8babc582d71239d55d876bb6f6d1 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5d7effdcf06f finish=length rounds=106 accepted=150/247 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-d43fc7dbb25ce8d14d2397a703841f21 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5d7effdcf06f finish=length rounds=106 accepted=150/246 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4ffea70fec8911c107c79efac1d752ba prompt=14 cached=0 thinking=False effort=none tokens=256 sha=bd3964ef7930 finish=length rounds=89 accepted=166/249 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-b70640d2e4b24fa7942ead5bc3aeca64 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=bc55cc799f87 finish=length rounds=89 accepted=166/228 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-a30da46b31b807bf47abdc6019945988 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=306e0fe05bc2 finish=length rounds=118 accepted=138/220 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-4145895c4fb7626a3ecb2acaa1ff43ec prompt=14 cached=0 thinking=False effort=none tokens=256 sha=5a89476cf1f5 finish=length rounds=119 accepted=136/218 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-0c6951ef33cbc190d30325f7deabd8e5 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=460edda16244 finish=length rounds=155 accepted=100/155 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-d4790b7404faaa4cd6e9fae807afc4aa prompt=31 cached=0 thinking=False effort=none tokens=256 sha=460edda16244 finish=length rounds=141 accepted=114/210 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-e0e51ddb0279975edd9a044cc4bd3bf7 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=0eba9c9d3070 finish=length rounds=112 accepted=144/252 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-6bb775d0cc1b0c9356b9424908f65bd3 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=15d18697d5f1 finish=length rounds=165 accepted=90/150 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-b5cb8f90cc4415de1430fa8c46bc3a43 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=d813ade7fc53 finish=length rounds=136 accepted=120/213 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-a9b88c08b4332a012299061949d1bbe3 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=7675315607f0 finish=length rounds=131 accepted=125/242 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-5f0f121f875a331afff54268d4da88e0 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-8f301048e70a9b5ff46ff7e77149f8ca prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-cc4f7a9b74d73884968f9dd4e58dba6e prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-7ecea917473773bcf0c6f5461575098d prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=83 accepted=172/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-ed7714c9b9b4cca218aabb97aa9ca689 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=82 accepted=173/290 +[tensorfold] 127.0.0.1 "POST /v1/completions HTTP/1.1" 200 - +[tensorfold] done cmpl-c13cd03a52942ce8723142063bf4aa20 prompt=14 cached=0 thinking=False effort=none tokens=256 sha=9f2923ac14d0 finish=length rounds=82 accepted=173/293 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-78285e4804185a721b9890b34749301a prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=135 accepted=121/245 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-651b1f4704c7c43ea965645bce264c4f prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=143 accepted=113/235 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-482ac2ca8b84bc6467416fd139789a3a prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=207 accepted=48/117 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-cf60e03f9861b01bb663957707ded4a0 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=148 accepted=108/202 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-1ca737e5b005484a9218ca86062384f0 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=135 accepted=121/246 +[tensorfold] 127.0.0.1 "POST /v1/chat/completions HTTP/1.1" 200 - +[tensorfold] done chatcmpl-eca1f9f0435a852e7c2b6502d8190670 prompt=31 cached=0 thinking=False effort=none tokens=256 sha=a8c28b8297b4 finish=length rounds=189 accepted=67/145 +info: backend verify: 2920 calls, 22048 us each +info: backend keep: 1351 calls, 11 us each +info: backend draft: 2944 calls, 3828 us each +info: head graphs: 4 captured, 1119 of 1123 batches replayed +info: graphs: 17 captured (4 ms each), 3071 of 3088 rounds replayed diff --git a/docs/receipts/drafter-counts/out/35b.drafts.json b/docs/receipts/drafter-counts/out/35b.drafts.json new file mode 100644 index 0000000000..6361c928b7 --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b.drafts.json @@ -0,0 +1 @@ +{"drafter_passed":false,"drafter_repo_id":null,"receipt":"35b.json","run_id":"b46ba73a-e62a-4f31-83cd-98daa4ad5c91","samples":[{"accepted":145,"completion_tokens":256,"drafted":240,"fixture_id":"code","repeat":0,"rounds":111,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":249,"fixture_id":"code","repeat":1,"rounds":89,"status":"ok","temperature":1.0},{"accepted":166,"completion_tokens":256,"drafted":228,"fixture_id":"code","repeat":2,"rounds":89,"status":"ok","temperature":1.0},{"accepted":138,"completion_tokens":256,"drafted":220,"fixture_id":"code","repeat":3,"rounds":118,"status":"ok","temperature":1.0},{"accepted":144,"completion_tokens":256,"drafted":250,"fixture_id":"code","repeat":4,"rounds":111,"status":"ok","temperature":1.0},{"accepted":108,"completion_tokens":256,"drafted":174,"fixture_id":"chat","repeat":0,"rounds":147,"status":"ok","temperature":1.0},{"accepted":105,"completion_tokens":256,"drafted":172,"fixture_id":"chat","repeat":1,"rounds":151,"status":"ok","temperature":1.0},{"accepted":106,"completion_tokens":256,"drafted":169,"fixture_id":"chat","repeat":2,"rounds":149,"status":"ok","temperature":1.0},{"accepted":120,"completion_tokens":256,"drafted":213,"fixture_id":"chat","repeat":3,"rounds":136,"status":"ok","temperature":1.0},{"accepted":64,"completion_tokens":256,"drafted":117,"fixture_id":"chat","repeat":4,"rounds":192,"status":"ok","temperature":1.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":0,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":1,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":2,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":3,"rounds":82,"status":"ok","temperature":0.0},{"accepted":173,"completion_tokens":256,"drafted":293,"fixture_id":"code","repeat":4,"rounds":82,"status":"ok","temperature":0.0},{"accepted":108,"completion_tokens":256,"drafted":219,"fixture_id":"chat","repeat":0,"rounds":148,"status":"ok","temperature":0.0},{"accepted":108,"completion_tokens":256,"drafted":219,"fixture_id":"chat","repeat":1,"rounds":148,"status":"ok","temperature":0.0},{"accepted":122,"completion_tokens":256,"drafted":247,"fixture_id":"chat","repeat":2,"rounds":134,"status":"ok","temperature":0.0},{"accepted":65,"completion_tokens":256,"drafted":140,"fixture_id":"chat","repeat":3,"rounds":191,"status":"ok","temperature":0.0},{"accepted":85,"completion_tokens":256,"drafted":168,"fixture_id":"chat","repeat":4,"rounds":171,"status":"ok","temperature":0.0}],"schema":"tensorfold-benchmark-drafts/1","totals":{"accepted":2615,"attempts":20,"drafted":4490,"reported":20,"rounds":2495}} diff --git a/docs/receipts/drafter-counts/out/35b.json b/docs/receipts/drafter-counts/out/35b.json new file mode 100644 index 0000000000..d5b90a5e56 --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b.json @@ -0,0 +1 @@ +{"created_at":"2026-10-18T17:43:35.960224Z","hardware":{"cpu_model":"AMD RYZEN AI MAX+ 395 with Radeon 8060S","gpu_count":1,"gpus":[{"cores":40,"memory_bytes":128849018880,"name":"AMD Radeon 8060S Graphics"}],"memory_type":"unified","source":"detected","system_memory_bytes":131209781248},"model":{"config_sha256":"a822a9e48b0aafbe3144ec37d4fb067e178ed96615ce6e4420b3149893cc5767","drafter_repo_id":null,"family":"qwen3_5_moe","local":false,"quantization":"mlx-affine-4bit-g64","repo_id":"TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP","revision":"f84b054c5677b6e59bdb969ae61915b0983d68bb","tokenizer_sha256":"4f990165752add61f3708e59f04d0f80b276b20ec960e06fc80133303f3ec141"},"run_id":"b46ba73a-e62a-4f31-83cd-98daa4ad5c91","runtime":{"backend":"rocm","dependencies":{},"platform":"linux","python_version":"none","tensorfold_version":"1.0.12"},"samples":[{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.242303078,"delivery_tps":78.64779876077951,"end_to_end_seconds":3.302757688,"error_code":null,"fixture_id":"code","prefill_seconds":0.058524826,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":78.64690566092122,"status":"ok","temperature":1.0,"token_sha":"5d7effdcf06f","ttft_seconds":0.059956997},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.96510749,"delivery_tps":86.00025491824582,"end_to_end_seconds":3.023972944,"error_code":null,"fixture_id":"code","prefill_seconds":0.056392831,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":85.99890079644958,"status":"ok","temperature":1.0,"token_sha":"bd3964ef7930","ttft_seconds":0.058147789},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.878249098,"delivery_tps":88.59552850279395,"end_to_end_seconds":2.938296122,"error_code":null,"fixture_id":"code","prefill_seconds":0.058279035,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":88.59444074463444,"status":"ok","temperature":1.0,"token_sha":"bc55cc799f87","ttft_seconds":0.05956116},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.259204921,"delivery_tps":78.23994077726174,"end_to_end_seconds":3.318896947,"error_code":null,"fixture_id":"code","prefill_seconds":0.058096791,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":78.23906891899784,"status":"ok","temperature":1.0,"token_sha":"306e0fe05bc2","ttft_seconds":0.059404526},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.24801356,"delivery_tps":78.50952444915285,"end_to_end_seconds":3.307986506,"error_code":null,"fixture_id":"code","prefill_seconds":0.057949467,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":78.50840126160378,"status":"ok","temperature":1.0,"token_sha":"5a89476cf1f5","ttft_seconds":0.059037988},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.448749345,"delivery_tps":73.93984731586806,"end_to_end_seconds":3.544540757,"error_code":null,"fixture_id":"chat","prefill_seconds":0.094050582,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":73.93876038640492,"status":"ok","temperature":1.0,"token_sha":"460edda16244","ttft_seconds":0.095195552},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.488952348,"delivery_tps":73.08784258580536,"end_to_end_seconds":3.58242967,"error_code":null,"fixture_id":"chat","prefill_seconds":0.091283518,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":73.08653254052066,"status":"ok","temperature":1.0,"token_sha":"0eba9c9d3070","ttft_seconds":0.092819875},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.429184184,"delivery_tps":74.36171005039256,"end_to_end_seconds":3.521660621,"error_code":null,"fixture_id":"chat","prefill_seconds":0.090493505,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":74.36075963257915,"status":"ok","temperature":1.0,"token_sha":"15d18697d5f1","ttft_seconds":0.092000783},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.616247215,"delivery_tps":70.51509060062975,"end_to_end_seconds":3.710006137,"error_code":null,"fixture_id":"chat","prefill_seconds":0.091761291,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":70.51476632507715,"status":"ok","temperature":1.0,"token_sha":"d813ade7fc53","ttft_seconds":0.093469072},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.662028584,"delivery_tps":69.63353620835636,"end_to_end_seconds":3.757150529,"error_code":null,"fixture_id":"chat","prefill_seconds":0.092135778,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":69.6242956990888,"status":"ok","temperature":1.0,"token_sha":"7675315607f0","ttft_seconds":0.094256425},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.503125183,"delivery_tps":101.87265172826157,"end_to_end_seconds":2.558441621,"error_code":null,"fixture_id":"code","prefill_seconds":0.053257699,"prompt_tokens":14,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":101.85524463007886,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.054989359},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.500239603,"delivery_tps":101.99022513443485,"end_to_end_seconds":2.555581121,"error_code":null,"fixture_id":"code","prefill_seconds":0.052766597,"prompt_tokens":14,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":101.97124545480219,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.054677031},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.496235342,"delivery_tps":102.15382969287357,"end_to_end_seconds":2.552384181,"error_code":null,"fixture_id":"code","prefill_seconds":0.054032533,"prompt_tokens":14,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":102.15150866601722,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055659256},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.498657708,"delivery_tps":102.05479493392058,"end_to_end_seconds":2.55358196,"error_code":null,"fixture_id":"code","prefill_seconds":0.053328897,"prompt_tokens":14,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":102.05369506179902,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.0546727},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":2.489292493,"delivery_tps":102.43874543352024,"end_to_end_seconds":2.545203491,"error_code":null,"fixture_id":"code","prefill_seconds":0.05439677,"prompt_tokens":14,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":102.4373590909349,"status":"ok","temperature":0.0,"token_sha":"9f2923ac14d0","ttft_seconds":0.055443064},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.287998041,"delivery_tps":77.5547907329182,"end_to_end_seconds":3.380643672,"error_code":null,"fixture_id":"chat","prefill_seconds":0.090520614,"prompt_tokens":31,"repeat":0,"seed":1234,"server_decode_seconds":null,"server_decode_tps":77.55396120223493,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.092159808},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.286176494,"delivery_tps":77.59777981054478,"end_to_end_seconds":3.376711398,"error_code":null,"fixture_id":"chat","prefill_seconds":0.088128057,"prompt_tokens":31,"repeat":1,"seed":1235,"server_decode_seconds":null,"server_decode_tps":77.59672763559524,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.089660125},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.242861839,"delivery_tps":78.63424735931218,"end_to_end_seconds":3.331759,"error_code":null,"fixture_id":"chat","prefill_seconds":0.087261058,"prompt_tokens":31,"repeat":2,"seed":1236,"server_decode_seconds":null,"server_decode_tps":78.633206679773,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.088589421},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.490464885,"delivery_tps":73.05617114093958,"end_to_end_seconds":3.580284215,"error_code":null,"fixture_id":"chat","prefill_seconds":0.087617345,"prompt_tokens":31,"repeat":3,"seed":1237,"server_decode_seconds":null,"server_decode_tps":73.05537833403073,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.088891012},{"cached_tokens":0,"completion_tokens":256,"delivery_seconds":3.381315769,"delivery_tps":75.41442959508464,"end_to_end_seconds":3.47076663,"error_code":null,"fixture_id":"chat","prefill_seconds":0.08749438,"prompt_tokens":31,"repeat":4,"seed":1238,"server_decode_seconds":null,"server_decode_tps":75.41348265434758,"status":"ok","temperature":0.0,"token_sha":"a8c28b8297b4","ttft_seconds":0.088971758}],"schema_version":1,"settings":{"context_tokens":null,"managed":true,"rank_count":1,"repetitions":5,"serial":false,"temperatures":[1.0,0.0],"tokens":256},"suite":{"id":"community-v1","sha256":"9bb717da00666b90b4006174abd9ed46108affc37456b43c64a224a9c938d5be"}} diff --git a/docs/receipts/drafter-counts/out/35b.log b/docs/receipts/drafter-counts/out/35b.log new file mode 100644 index 0000000000..bf94c059e4 --- /dev/null +++ b/docs/receipts/drafter-counts/out/35b.log @@ -0,0 +1,37 @@ +community-v1: code and chat, 2 temperatures, 5 repeats after warm-up. +This generates model replies; use Ctrl-C to cancel. Nothing is uploaded unless you publish. +The server gets up to 606 s to load 21 GB of weights; --startup-timeout sets another limit. +code T=1 warm-up: ok 79.24 delivery tok/s +code T=1 repeat 1/5: ok 78.65 delivery tok/s +code T=1 repeat 2/5: ok 86.00 delivery tok/s +code T=1 repeat 3/5: ok 88.60 delivery tok/s +code T=1 repeat 4/5: ok 78.24 delivery tok/s +code T=1 repeat 5/5: ok 78.51 delivery tok/s +chat T=1 warm-up: ok 71.15 delivery tok/s +chat T=1 repeat 1/5: ok 73.94 delivery tok/s +chat T=1 repeat 2/5: ok 73.09 delivery tok/s +chat T=1 repeat 3/5: ok 74.36 delivery tok/s +chat T=1 repeat 4/5: ok 70.52 delivery tok/s +chat T=1 repeat 5/5: ok 69.63 delivery tok/s +code T=0 warm-up: ok 101.81 delivery tok/s +code T=0 repeat 1/5: ok 101.87 delivery tok/s +code T=0 repeat 2/5: ok 101.99 delivery tok/s +code T=0 repeat 3/5: ok 102.15 delivery tok/s +code T=0 repeat 4/5: ok 102.05 delivery tok/s +code T=0 repeat 5/5: ok 102.44 delivery tok/s +chat T=0 warm-up: ok 75.67 delivery tok/s +chat T=0 repeat 1/5: ok 77.55 delivery tok/s +chat T=0 repeat 2/5: ok 77.60 delivery tok/s +chat T=0 repeat 3/5: ok 78.63 delivery tok/s +chat T=0 repeat 4/5: ok 73.06 delivery tok/s +chat T=0 repeat 5/5: ok 75.41 delivery tok/s +Run b46ba73a-e62a-4f31-83cd-98daa4ad5c91 | rocm | TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP +Workload Temp Stream delivery tok/s First text, s Status +code 1 78.65 0.059 complete +chat 1 73.09 0.093 complete +code 0 102.05 0.055 complete +chat 0 77.55 0.089 complete +Stream delivery includes transport buffering; it is not an engine token clock. +This rocm receipt is saved locally; it cannot be published until the board accepts rocm. +Receipt saved: ~/tf-1001-runs/out/35b.json +Draft counts: 2615 of 4490 drafted tokens accepted over 2495 rounds (20 of 20 attempts reported them); saved: ~/tf-1001-runs/out/35b.drafts.json diff --git a/docs/receipts/drafter-counts/pull.log b/docs/receipts/drafter-counts/pull.log new file mode 100644 index 0000000000..e5f166bad9 --- /dev/null +++ b/docs/receipts/drafter-counts/pull.log @@ -0,0 +1,100 @@ +start 17:19:08 +[tensorfold] downloading TensorFold/Qwen3.8-27B-MLX-4bit@70ae7fac63274ff2eac54152031433374cb80f2f from Hugging Face +TensorFold/Qwen3.8-27B-MLX-4bit: Qwen3.5-2B or Qwen3.8-27B (qwen3_5) + .gitattributes: 0.00 MiB + LICENSE: 0.01 MiB + README.md: 0.01 MiB + chat_template.jinja: 0.01 MiB + config.json: 0.00 MiB + generation_config.json: 0.00 MiB + model.safetensors.index.json: 0.21 MiB + preprocessor_config.json: 0.00 MiB + processor_config.json: 0.00 MiB + qwen-logo.png: 0.01 MiB + tokenizer.json: 19.06 MiB + tokenizer_config.json: 0.00 MiB + video_preprocessor_config.json: 0.00 MiB + vocab.json: 6.41 MiB + 0.54 of 14.95 GiB, 55.1 MiB/s + 1.13 of 14.95 GiB, 57.8 MiB/s + 1.72 of 14.95 GiB, 58.6 MiB/s + 2.30 of 14.95 GiB, 58.7 MiB/s + 2.86 of 14.95 GiB, 58.5 MiB/s + 3.43 of 14.95 GiB, 58.6 MiB/s + 4.04 of 14.95 GiB, 59.0 MiB/s + 4.67 of 14.95 GiB, 59.7 MiB/s + 5.30 of 14.95 GiB, 60.2 MiB/s + 5.94 of 14.95 GiB, 60.8 MiB/s + 6.57 of 14.95 GiB, 61.2 MiB/s + 7.19 of 14.95 GiB, 61.3 MiB/s + 7.82 of 14.95 GiB, 61.6 MiB/s + 8.45 of 14.95 GiB, 61.8 MiB/s + 9.07 of 14.95 GiB, 61.9 MiB/s + 9.68 of 14.95 GiB, 61.9 MiB/s + 10.31 of 14.95 GiB, 62.1 MiB/s + 10.96 of 14.95 GiB, 62.3 MiB/s + 11.60 of 14.95 GiB, 62.5 MiB/s + 12.25 of 14.95 GiB, 62.7 MiB/s + 12.90 of 14.95 GiB, 62.9 MiB/s + 13.53 of 14.95 GiB, 63.0 MiB/s + 14.17 of 14.95 GiB, 63.1 MiB/s + 14.81 of 14.95 GiB, 63.2 MiB/s + model-00001-of-00003.safetensors: 5095.74 MiB in 160 pieces + model-00002-of-00003.safetensors: 5106.15 MiB in 160 pieces + model-00003-of-00003.safetensors: 5108.92 MiB in 160 pieces +stored TensorFold/Qwen3.8-27B-MLX-4bit@70ae7fac63274ff2eac54152031433374cb80f2f at ~/.cache/huggingface/hub/models--TensorFold--Qwen3.8-27B-MLX-4bit/snapshots/70ae7fac63274ff2eac54152031433374cb80f2f +27b exit=0 17:23:41 +[tensorfold] downloading TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP@f84b054c5677b6e59bdb969ae61915b0983d68bb from Hugging Face +TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP: qwen3_5_moe (qwen3_5_moe) + .gitattributes: 0.00 MiB + README.md: 0.00 MiB + chat_template.jinja: 0.01 MiB + config.json: 0.02 MiB + configuration.json: 0.00 MiB + generation_config.json: 0.00 MiB + model.safetensors.index.json: 0.21 MiB + preprocessor_config.json: 0.00 MiB + processor_config.json: 0.00 MiB + tokenizer.json: 19.06 MiB + tokenizer_config.json: 0.00 MiB + video_preprocessor_config.json: 0.00 MiB + vocab.json: 6.41 MiB + 0.58 of 19.44 GiB, 58.9 MiB/s + 1.21 of 19.44 GiB, 61.9 MiB/s + 1.84 of 19.44 GiB, 62.8 MiB/s + 2.46 of 19.44 GiB, 63.0 MiB/s + 3.10 of 19.44 GiB, 63.5 MiB/s + 3.73 of 19.44 GiB, 63.7 MiB/s + 4.37 of 19.44 GiB, 63.9 MiB/s + 5.00 of 19.44 GiB, 63.9 MiB/s + 5.63 of 19.44 GiB, 64.1 MiB/s + 6.29 of 19.44 GiB, 64.4 MiB/s + 6.94 of 19.44 GiB, 64.6 MiB/s + 7.57 of 19.44 GiB, 64.5 MiB/s + 8.20 of 19.44 GiB, 64.6 MiB/s + 8.82 of 19.44 GiB, 64.5 MiB/s + 9.41 of 19.44 GiB, 64.2 MiB/s + 10.04 of 19.44 GiB, 64.3 MiB/s + 10.70 of 19.44 GiB, 64.4 MiB/s + 11.28 of 19.44 GiB, 64.1 MiB/s + 11.93 of 19.44 GiB, 64.3 MiB/s + 12.59 of 19.44 GiB, 64.4 MiB/s + 13.21 of 19.44 GiB, 64.4 MiB/s + 13.82 of 19.44 GiB, 64.3 MiB/s + 14.46 of 19.44 GiB, 64.4 MiB/s + 15.11 of 19.44 GiB, 64.4 MiB/s + 15.75 of 19.44 GiB, 64.5 MiB/s + 16.37 of 19.44 GiB, 64.5 MiB/s + 17.00 of 19.44 GiB, 64.5 MiB/s + 17.63 of 19.44 GiB, 64.4 MiB/s + 18.26 of 19.44 GiB, 64.5 MiB/s + 18.89 of 19.44 GiB, 64.5 MiB/s + 19.41 of 19.44 GiB, 64.1 MiB/s + model-00001-of-00004.safetensors: 5043.22 MiB in 158 pieces + model-00002-of-00004.safetensors: 5119.77 MiB in 160 pieces + model-00003-of-00004.safetensors: 5119.63 MiB in 160 pieces + model-00004-of-00004.safetensors: 4174.43 MiB in 131 pieces + mtp-4bit.safetensors: 453.37 MiB in 15 pieces +stored TensorFold/Qwen3.6-35B-A3B-MLX-4bit-MTP@f84b054c5677b6e59bdb969ae61915b0983d68bb at ~/.cache/huggingface/hub/models--TensorFold--Qwen3.6-35B-A3B-MLX-4bit-MTP/snapshots/f84b054c5677b6e59bdb969ae61915b0983d68bb +35b exit=0 17:29:21 +PULLDONE diff --git a/docs/receipts/drafter-counts/run.log b/docs/receipts/drafter-counts/run.log new file mode 100644 index 0000000000..b90e37a8c1 --- /dev/null +++ b/docs/receipts/drafter-counts/run.log @@ -0,0 +1,57 @@ +hint: Using 'master' as the name for the initial branch. This default branch name +hint: will change to "main" in Git 3.0. To configure the initial branch name +hint: to use in all of your new repositories, which will suppress this warning, +hint: call: +hint: +hint: git config --global init.defaultBranch +hint: +hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and +hint: 'development'. The just-created branch can be renamed via this command: +hint: +hint: git branch -m +hint: +hint: Disable this message with "git config set advice.defaultBranchName false" +warning: remote HEAD refers to nonexistent ref, unable to checkout +Trying simple merge with origin/hip-refuses-drafter +Simple merge did not work, trying automatic merge. +Auto-merging docs/community-benchmarks.md +Auto-merging zig/src/server/benchmark.zig +Trying simple merge with origin/benchmark-drafter-counts +Simple merge did not work, trying automatic merge. +Auto-merging docs/community-benchmarks.md +Auto-merging zig/src/server/benchmark.zig +Auto-merging zig/src/server/benchmark_run_test.zig +fatal: ambiguous argument 'origin/main': unknown revision or path not in the working tree. +Use '--' to separate paths from revisions, like this: +'git [...] -- [...]' +Filesystem Size Used Avail Use% Mounted on +/dev/mapper/ubuntu--vg-ubuntu--lv 1.9T 1.7T 76G 96% / + total used free shared buff/cache available +Mem: 122 96 2 0 24 26 +~/.cache/huggingface/hub/models--TensorFold--Qwen3.6-35B-A3B-MLX-4bit-MTP/snapshots: +f84b054c5677b6e59bdb969ae61915b0983d68bb + +~/.cache/huggingface/hub/models--TensorFold--Qwen3.8-27B-MLX-4bit/snapshots: +70ae7fac63274ff2eac54152031433374cb80f2f +llm-server: failed +gtt_used=1129861120 busy=0 t=17:31:19 +build_exit=0 +e279e29a54b56dc81ff70985f8c4cffaa0d9aff4a7456a234eb7247667492247 ~/tf-1001/zig-out/native-hip/bin/tensorfold-native +gtt_used=1129861120 busy=0 t=17:33:19 +R1 exit=0 +gtt_used=1129906176 busy=81 t=17:42:02 +R2a exit=1 +ls: cannot access '~/tf-1001-runs/out/27b-dflash2.json': No such file or directory +gtt_used=1129906176 busy=78 t=17:42:02 +R2b exit=1 +gtt_used=1129861120 busy=57 t=17:42:02 +gtt_used=1129861120 busy=56 t=17:42:02 +R3 exit=0 +gtt_used=1129906176 busy=76 t=17:43:35 +R4 exit=0 +gtt_used=1129906176 busy=84 t=17:45:19 +R5 exit=0 +gtt_used=1682337792 busy=42 t=17:47:01 +RUNDONE +llm-server: active +llm-server chat OK after 200s: {"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"","reasoning_content":"The user's request is simple. They"}}],"created":1792345812,"model":"Qwen3.8-Flash-Next","system_fingerprint":"b111-d807f04","object":"chat.completion","usag diff --git a/zig/src/server/benchmark.zig b/zig/src/server/benchmark.zig index 1dcbf07b5d..63803b4194 100644 --- a/zig/src/server/benchmark.zig +++ b/zig/src/server/benchmark.zig @@ -15,6 +15,7 @@ const manifest = @import("benchmark_manifest.zig"); const upload = @import("benchmark_upload.zig"); const check = @import("benchmark_check.zig"); const stop = @import("benchmark_stop.zig"); +const drafts = @import("benchmark_drafts.zig"); const Allocator = std.mem.Allocator; const Args = args_mod.Args; @@ -410,6 +411,12 @@ fn run(ctx: *const Context, args: Args) !u8 { try show(ctx, v); if (!try receipt.save(a, ctx.io, v, dest, nowMs(ctx), &problem)) return fail(ctx, problem); try say(ctx, "Receipt saved: {s}", .{dest}); + // whether a drafter ran is the server's counters, kept beside the receipt; the board's receipt has no field for it + const passed: ?bool = if (plan) |p| p.drafter_dir != null else null; + const counts = try drafts.path(a, dest); + const kept = try drafts.value(a, &run_id, std.fs.path.basename(dest), facts.model.drafter_repo_id, passed, samples); + if (!try drafts.save(a, ctx.io, kept, counts, &problem)) return fail(ctx, problem); + try say(ctx, "{s}", .{try drafts.verdict(a, drafts.totals(samples), passed, counts)}); if (cleanup) |c| return fail(ctx, c); const was_cancelled = stop.requested() != null or for (samples) |s| { if (s.error_code == .cancelled) break true; diff --git a/zig/src/server/benchmark_cli_test.zig b/zig/src/server/benchmark_cli_test.zig index 3db6472d18..2d58bd6a94 100644 --- a/zig/src/server/benchmark_cli_test.zig +++ b/zig/src/server/benchmark_cli_test.zig @@ -291,6 +291,16 @@ pub fn managedWorld(a: std.mem.Allocator, chat_trickles: bool) !*fake.Fake { })); } +/// ``managedWorld`` whose replies end with ``speculative`` draft counters (members of a JSON object). +pub fn draftedWorld(a: std.mem.Allocator, speculative: []const u8) !*fake.Fake { + return fake.Fake.start(std.testing.allocator, try a.dupe(fake.Route, &.{ + .{ .method = "GET", .path = "/health", .replies = try a.dupe([]const fake.Write, &.{try fake.jsonReply(a, "200 OK", "{\"status\": \"ok\", \"warming\": false}")}) }, + .{ .method = "GET", .path = "/v1/models", .replies = try a.dupe([]const fake.Write, &.{try fake.jsonReply(a, "200 OK", "{\"data\": [{\"id\": \".tensorfold-benchmark\"}]}")}) }, + .{ .method = "POST", .path = "/v1/completions", .replies = try a.dupe([]const fake.Write, &.{try fake.drafted(a, false, 4, 15, speculative)}) }, + .{ .method = "POST", .path = "/v1/chat/completions", .replies = try a.dupe([]const fake.Write, &.{try fake.drafted(a, true, 4, 15, speculative)}) }, + })); +} + pub fn childGone(r: *Run) !bool { const text = try std.Io.Dir.cwd().readFileAlloc(std.testing.io, try r.path("pid"), r.a, .limited(64)); const pid = try std.fmt.parseInt(std.posix.pid_t, std.mem.trim(u8, text, " \n"), 10); diff --git a/zig/src/server/benchmark_drafts.zig b/zig/src/server/benchmark_drafts.zig new file mode 100644 index 0000000000..a469f6a8f1 --- /dev/null +++ b/zig/src/server/benchmark_drafts.zig @@ -0,0 +1,159 @@ +//! The local draft counts beside a receipt (rounds, drafted, accepted per attempt); never uploaded to the board. +const std = @import("std"); +const json = @import("json"); +const suite = @import("benchmark_suite.zig"); +const receipt = @import("benchmark_receipt.zig"); +const Allocator = std.mem.Allocator; +const Value = json.Value; + +pub const schema = "tensorfold-benchmark-drafts/1"; + +/// The attempts' counts summed over those whose server reported them; ``sums`` is null once a sum passes 64 bits. +pub const Totals = struct { attempts: usize = 0, reported: usize = 0, sums: ?suite.Drafts = .{ .rounds = 0, .drafted = 0, .accepted = 0 } }; + +pub fn totals(samples: []const suite.Sample) Totals { + var t: Totals = .{ .attempts = samples.len }; + for (samples) |s| if (s.drafts) |d| { + t.reported += 1; + if (t.sums) |sum| t.sums = add(sum, d); + }; + return t; +} + +/// Each counter of ``x`` plus ``y``; null when one passes 64 bits, so two large counts give no totals, not a panic. +fn add(x: suite.Drafts, y: suite.Drafts) ?suite.Drafts { + return .{ + .rounds = std.math.add(u64, x.rounds, y.rounds) catch return null, + .drafted = std.math.add(u64, x.drafted, y.drafted) catch return null, + .accepted = std.math.add(u64, x.accepted, y.accepted) catch return null, + }; +} + +/// ``run.json`` -> ``run.drafts.json``; any other name gets ``.drafts.json`` appended. +pub fn path(a: Allocator, receipt_path: []const u8) ![]const u8 { + const stem = if (std.mem.endsWith(u8, receipt_path, ".json")) receipt_path[0 .. receipt_path.len - ".json".len] else receipt_path; + return std.mem.concat(a, u8, &.{ stem, ".drafts.json" }); +} + +/// What a run's draft counts show; ``passed``: a drafter went to the server (null when attached and unknown). +pub fn verdict(a: Allocator, t: Totals, passed: ?bool, file: []const u8) ![]const u8 { + if (t.reported == 0) return std.fmt.allocPrint(a, "Draft counts: the server reported none; saved: {s}", .{file}); + const sum = t.sums orelse return std.fmt.allocPrint(a, "Draft counts: no totals, as a sum passes 64 bits ({d} of {d} attempts reported counts); saved: {s}", .{ t.reported, t.attempts, file }); + const line = try std.fmt.allocPrint(a, "Draft counts: {d} of {d} drafted tokens accepted over {d} rounds ({d} of {d} attempts reported them); saved: {s}", .{ sum.accepted, sum.drafted, sum.rounds, t.reported, t.attempts, file }); + if ((passed orelse false) and sum.drafted == 0) return std.mem.concat(a, u8, &.{ line, "\n", not_drafted }); + return line; +} + +/// The note when a drafter was passed and the server proposed no tokens with it. +pub const not_drafted = "A drafter was passed, but the server drafted no tokens: this run does not show that the drafter ran."; + +/// The file: the run it belongs to, the drafter, the totals and one entry per receipt sample, in receipt order. +pub fn value(a: Allocator, run_id: []const u8, receipt_name: []const u8, drafter_repo_id: ?[]const u8, passed: ?bool, samples: []const suite.Sample) !Value { + const top = try json.newObject(a); + try top.put(a, "schema", .{ .string = schema }); + try top.put(a, "run_id", .{ .string = run_id }); + try top.put(a, "receipt", .{ .string = receipt_name }); + try top.put(a, "drafter_repo_id", if (drafter_repo_id) |d| .{ .string = d } else .null); + try top.put(a, "drafter_passed", if (passed) |p| .{ .bool = p } else .null); + const t = totals(samples); + const sum = try json.newObject(a); + try sum.put(a, "attempts", try json.intValue(a, t.attempts)); + try sum.put(a, "reported", try json.intValue(a, t.reported)); + // a sum past 64 bits has no total: null, as an attempt without counts has + inline for (.{ "rounds", "drafted", "accepted" }) |k| try sum.put(a, k, if (t.sums) |d| try json.intValue(a, @field(d, k)) else .null); + try top.put(a, "totals", .{ .object = sum }); + const rows = try a.alloc(Value, samples.len); + for (samples, rows) |s, *slot| { + const o = try json.newObject(a); + try o.put(a, "fixture_id", .{ .string = s.fixture_id }); + try o.put(a, "temperature", .{ .float = s.temperature }); + try o.put(a, "repeat", try json.intValue(a, s.repeat)); + try o.put(a, "status", .{ .string = @tagName(s.status) }); + try o.put(a, "completion_tokens", if (s.completion_tokens) |n| try json.intValue(a, n) else .null); + inline for (.{ "rounds", "drafted", "accepted" }) |k| try o.put(a, k, if (s.drafts) |d| try json.intValue(a, @field(d, k)) else .null); + slot.* = .{ .object = o }; + } + try top.put(a, "samples", .{ .array = rows }); + return .{ .object = top }; +} + +/// Writes ``v`` as canonical JSON plus a newline, privately and atomically, as the receipt is written. +pub fn save(a: Allocator, io: std.Io, v: Value, file: []const u8, problem: *[]const u8) !bool { + const bytes = try std.mem.concat(a, u8, &.{ try receipt.canonical(a, v), "\n" }); + const dir = std.fs.path.dirname(file) orelse "."; + const cwd = std.Io.Dir.cwd(); + var name: [8]u8 = undefined; + io.random(&name); + const temp = try std.fmt.allocPrint(a, "{s}/.benchmark-{s}", .{ dir, std.fmt.bytesToHex(name, .lower) }); + problem.* = try std.fmt.allocPrint(a, "Could not save the draft counts to {s}", .{file}); + var f = cwd.createFile(io, temp, .{ .exclusive = true, .permissions = .fromMode(0o600) }) catch return false; + const written = if (f.writeStreamingAll(io, bytes)) true else |_| false; + f.close(io); + if (!written or (if (cwd.rename(temp, cwd, file, io)) false else |_| true)) { + cwd.deleteFile(io, temp) catch {}; + return false; + } + problem.* = ""; + return true; +} + +test "the file's name sits beside the receipt" { + var arena: std.heap.ArenaAllocator = .init(std.testing.allocator); + defer arena.deinit(); + const a = arena.allocator(); + try std.testing.expectEqualStrings("/r/run.drafts.json", try path(a, "/r/run.json")); + try std.testing.expectEqualStrings("/r/run.out.drafts.json", try path(a, "/r/run.out")); +} + +test "totals sum only the attempts that reported counts, and the verdict says when a passed drafter drafted nothing" { + var arena: std.heap.ArenaAllocator = .init(std.testing.allocator); + defer arena.deinit(); + const a = arena.allocator(); + const drafted = [_]suite.Sample{ + .{ .fixture_id = "code", .temperature = 0, .repeat = 0, .seed = 1, .status = .ok, .drafts = .{ .rounds = 14, .drafted = 13, .accepted = 10 } }, + .{ .fixture_id = "chat", .temperature = 0, .repeat = 0, .seed = 1, .status = .ok, .drafts = .{ .rounds = 6, .drafted = 5, .accepted = 2 } }, + .{ .fixture_id = "chat", .temperature = 0, .repeat = 1, .seed = 1, .status = .@"error" }, + }; + const t = totals(&drafted); + try std.testing.expect(t.attempts == 3 and t.reported == 2); + try std.testing.expectEqual(suite.Drafts{ .rounds = 20, .drafted = 18, .accepted = 12 }, t.sums.?); + const shown = try verdict(a, t, true, "f"); + try std.testing.expect(std.mem.startsWith(u8, shown, "Draft counts: 12 of 18 drafted tokens accepted over 20 rounds (2 of 3 attempts")); + try std.testing.expect(std.mem.indexOf(u8, shown, not_drafted) == null); + const plain = [_]suite.Sample{.{ .fixture_id = "code", .temperature = 0, .repeat = 0, .seed = 1, .status = .ok, .drafts = .{ .rounds = 32, .drafted = 0, .accepted = 0 } }}; + try std.testing.expect(std.mem.endsWith(u8, try verdict(a, totals(&plain), true, "f"), not_drafted)); + for ([_]?bool{ false, null }) |p| try std.testing.expect(std.mem.indexOf(u8, try verdict(a, totals(&plain), p, "f"), not_drafted) == null); + const silent = [_]suite.Sample{.{ .fixture_id = "code", .temperature = 0, .repeat = 0, .seed = 1, .status = .ok }}; + try std.testing.expectEqualStrings("Draft counts: the server reported none; saved: f", try verdict(a, totals(&silent), true, "f")); + const v = try value(a, "00000000-0000-4000-8000-000000000001", "run.json", "z-lab/Qwen3.8-27B-DFlash2", true, &drafted); + const text = try json.stringify(a, v, .{ .compact = true }); + try std.testing.expect(std.mem.indexOf(u8, text, "\"totals\":{\"attempts\":3,\"reported\":2,\"rounds\":20,\"drafted\":18,\"accepted\":12}") != null); + try std.testing.expect(std.mem.indexOf(u8, text, "\"repeat\":1,\"status\":\"error\",\"completion_tokens\":null,\"rounds\":null,\"drafted\":null,\"accepted\":null") != null); +} + +test "sums that would pass 64 bits give no totals rather than a panic, and the attempts keep their own counts" { + var arena: std.heap.ArenaAllocator = .init(std.testing.allocator); + defer arena.deinit(); + const a = arena.allocator(); + const max = std.math.maxInt(u64); + const at = struct { + fn sample(repeat: i32, d: suite.Drafts) suite.Sample { + return .{ .fixture_id = "code", .temperature = 0, .repeat = repeat, .seed = 1, .status = .ok, .drafts = d }; + } + }.sample; + // a sum of exactly 2^64 - 1 still fits + const fits = [_]suite.Sample{ at(0, .{ .rounds = 1, .drafted = max - 1, .accepted = 1 }), at(1, .{ .rounds = 1, .drafted = 1, .accepted = 0 }) }; + try std.testing.expectEqual(suite.Drafts{ .rounds = 2, .drafted = max, .accepted = 1 }, totals(&fits).sums.?); + // one past it gives no totals, whether every sum passes or only the rounds or only the drafted tokens + const big: suite.Drafts = .{ .rounds = max - 1, .drafted = max, .accepted = max - 1 }; + for ([_]suite.Drafts{ big, .{ .rounds = 2, .drafted = 0, .accepted = 0 }, .{ .rounds = 0, .drafted = 1, .accepted = 0 } }) |second| { + const over = [_]suite.Sample{ at(0, big), at(1, second), .{ .fixture_id = "chat", .temperature = 0, .repeat = 0, .seed = 1, .status = .@"error" } }; + const t = totals(&over); + try std.testing.expect(t.attempts == 3 and t.reported == 2 and t.sums == null); + const shown = try verdict(a, t, true, "f"); + try std.testing.expectEqualStrings("Draft counts: no totals, as a sum passes 64 bits (2 of 3 attempts reported counts); saved: f", shown); + const text = try json.stringify(a, try value(a, "00000000-0000-4000-8000-000000000001", "run.json", null, true, &over), .{ .compact = true }); + try std.testing.expect(std.mem.indexOf(u8, text, "\"totals\":{\"attempts\":3,\"reported\":2,\"rounds\":null,\"drafted\":null,\"accepted\":null}") != null); + try std.testing.expect(std.mem.indexOf(u8, text, "\"rounds\":18446744073709551614,\"drafted\":18446744073709551615,\"accepted\":18446744073709551614") != null); + } +} diff --git a/zig/src/server/benchmark_fake.zig b/zig/src/server/benchmark_fake.zig index b90a1ad7fa..b90c08f3a4 100644 --- a/zig/src/server/benchmark_fake.zig +++ b/zig/src/server/benchmark_fake.zig @@ -244,9 +244,15 @@ pub fn tmpPath(a: Allocator, tmp: *std.testing.TmpDir) ![]const u8 { /// A completion stream of two text deliveries ``gap_ms`` apart, a finish, usage and DONE. pub fn reply(a: Allocator, chat: bool, completion: u64, gap_ms: u32) ![]const Write { + return drafted(a, chat, completion, gap_ms, ""); +} + +/// ``reply`` whose finishing chunk carries ``speculative`` members (such as ``"rounds": 3``) when not empty. +pub fn drafted(a: Allocator, chat: bool, completion: u64, gap_ms: u32, speculative: []const u8) ![]const Write { const first = if (chat) "{\"choices\": [{\"index\": 0, \"delta\": {\"reasoning_content\": \"synthetic\"}}]}" else "{\"choices\": [{\"index\": 0, \"text\": \"synthetic\"}]}"; const second = if (chat) "{\"choices\": [{\"index\": 0, \"delta\": {\"content\": \" output\"}}]}" else "{\"choices\": [{\"index\": 0, \"text\": \" output\"}]}"; - const final = "{\"choices\": [{\"index\": 0, \"delta\": {}, \"finish_reason\": \"length\"}], \"tensorfold\": {\"tokens_per_second\": 42.0, \"prefill_seconds\": 0.03, \"seconds\": 0.2, \"token_sha\": \"0123456789ab\"}}"; + const plain = "{\"choices\": [{\"index\": 0, \"delta\": {}, \"finish_reason\": \"length\"}], \"tensorfold\": {\"tokens_per_second\": 42.0, \"prefill_seconds\": 0.03, \"seconds\": 0.2, \"token_sha\": \"0123456789ab\"}"; + const final = if (speculative.len > 0) try std.mem.concat(a, u8, &.{ plain, ", \"speculative\": {", speculative, "}}" }) else plain ++ "}"; const usage = try std.fmt.allocPrint(a, "{{\"choices\": [], \"usage\": {{\"prompt_tokens\": 24, \"completion_tokens\": {d}, \"prompt_tokens_details\": {{\"cached_tokens\": 0}}}}}}", .{completion}); const w = try a.alloc(Write, 3); w[0] = .{ .bytes = try std.mem.concat(a, u8, &.{ try headOf(a, "200 OK", "text/event-stream"), ": keepalive\n\n", try event(a, first) }), .pause_ms = gap_ms }; diff --git a/zig/src/server/benchmark_run_test.zig b/zig/src/server/benchmark_run_test.zig index 783e3bb996..f9c2e861bb 100644 --- a/zig/src/server/benchmark_run_test.zig +++ b/zig/src/server/benchmark_run_test.zig @@ -364,3 +364,55 @@ test "an attached run whose accepted manifest lacks its rank count or revision i try std.testing.expect(missing_field == .null); } } + +test "the draft counts beside the receipt show whether a passed drafter drafted, and the receipt keeps the board's fields" { + var arena: std.heap.ArenaAllocator = .init(std.testing.allocator); + defer arena.deinit(); + const a = arena.allocator(); + const io = std.testing.io; + var tmp = std.testing.tmpDir(.{}); + defer tmp.cleanup(); + var r = try Run.init(a, &tmp); + const hub = try r.path("hub"); + try r.env.put("HF_HUB_CACHE", hub); + const drafter = "z-lab/Qwen3.8-27B-DFlash2"; + try cache(a, hub, drafter); + const big = try r.path("big-checkpoint"); + try std.Io.Dir.cwd().createDirPath(io, big); + try std.Io.Dir.cwd().writeFile(io, .{ .sub_path = try std.fs.path.join(a, &.{ big, "config.json" }), .data = "{\"model_type\": \"qwen3_5\", \"hidden_size\": 5120, \"num_hidden_layers\": 64}" }); + const out = try r.path("run.json"); + const counts = try r.path("run.drafts.json"); + const Case = struct { speculative: []const u8, drafter: []const u8, line: []const u8, fields: []const []const u8, unproven: bool }; + const cases = [_]Case{ + .{ .speculative = "\"rounds\": 3, \"drafted\": 4, \"accepted\": 3, \"acceptance_rate\": 0.75", .drafter = "auto", .line = "Draft counts: 6 of 8 drafted tokens accepted over 6 rounds (2 of 2 attempts reported them); saved: ", .fields = &.{ "\"drafter_passed\":true", "\"drafter_repo_id\":\"" ++ drafter ++ "\"", "\"accepted\":3,\"completion_tokens\":4,\"drafted\":4", "\"totals\":{\"accepted\":6,\"attempts\":2,\"drafted\":8,\"reported\":2,\"rounds\":6}" }, .unproven = false }, + .{ .speculative = "\"rounds\": 4, \"drafted\": 0, \"accepted\": 0", .drafter = "auto", .line = "Draft counts: 0 of 0 drafted tokens accepted over 8 rounds (2 of 2 attempts reported them); saved: ", .fields = &.{"\"drafter_passed\":true"}, .unproven = true }, + .{ .speculative = "\"rounds\": 4, \"drafted\": 0, \"accepted\": 0", .drafter = "none", .line = "Draft counts: 0 of 0 drafted tokens accepted over 8 rounds", .fields = &.{ "\"drafter_passed\":false", "\"drafter_repo_id\":null" }, .unproven = false }, + .{ .speculative = "", .drafter = "auto", .line = "Draft counts: the server reported none; saved: ", .fields = &.{ "\"reported\":0", "\"rounds\":null" }, .unproven = false }, + // accepted past drafted is no count at all + .{ .speculative = "\"rounds\": 1, \"drafted\": 1, \"accepted\": 2", .drafter = "auto", .line = "Draft counts: the server reported none; saved: ", .fields = &.{"\"reported\":0"}, .unproven = false }, + // two attempts' valid counts near 2^64 sum past it: no totals, each attempt's counts kept, and the run goes on + .{ .speculative = "\"rounds\": 9223372036854775808, \"drafted\": 18446744073709551615, \"accepted\": 18446744073709551000", .drafter = "auto", .line = "Draft counts: no totals, as a sum passes 64 bits (2 of 2 attempts reported counts); saved: ", .fields = &.{ "\"totals\":{\"accepted\":null,\"attempts\":2,\"drafted\":null,\"reported\":2,\"rounds\":null}", "\"accepted\":18446744073709551000,\"completion_tokens\":4,\"drafted\":18446744073709551615" }, .unproven = false }, + }; + for (cases) |c| { + const m = try cli.draftedWorld(a, c.speculative); + defer m.stop(); + r.self_exe = try cli.standIn(&r, m.port); + try std.testing.expectEqual(@as(u8, 0), try r.command(try argv(&r, &.{ big, "--drafter", c.drafter, "--output", out }))); + try std.testing.expect(r.has(c.line) and r.has(counts)); + try std.testing.expectEqual(c.unproven, r.has(@import("benchmark_drafts.zig").not_drafted)); + const text = try std.Io.Dir.cwd().readFileAlloc(io, counts, a, .limited(1 << 20)); + for (c.fields) |f| if (std.mem.indexOf(u8, text, f) == null) { + std.debug.print("draft counts lack {s}: {s}\n", .{ f, text }); + return error.TestUnexpectedResult; + }; + const st = try std.Io.Dir.cwd().statFile(io, counts, .{}); + try std.testing.expectEqual(@as(u32, 0o600), @as(u32, @intCast(st.permissions.toMode() & 0o777))); + // the receipt itself holds exactly the board's fields, so it loads under the same rules as before + var problem: []const u8 = ""; + const v = (try receipt.load(a, io, out, std.Io.Clock.real.now(io).toMilliseconds(), &problem)).?; + for (v.get("samples").?.array) |s| { + try std.testing.expect(s.get("drafted") == null and s.get("rounds") == null); + try std.testing.expect(std.mem.eql(u8, s.get("status").?.string, "ok") and std.mem.eql(u8, s.get("completion_tokens").?.int, "4")); + } + } +} diff --git a/zig/src/server/benchmark_sse.zig b/zig/src/server/benchmark_sse.zig index bd692064cd..a5776f9abe 100644 --- a/zig/src/server/benchmark_sse.zig +++ b/zig/src/server/benchmark_sse.zig @@ -34,6 +34,7 @@ pub const Stream = struct { last: ?i96 = null, deliveries: u32 = 0, usage: ?Value = null, + speculative: ?Value = null, runtime: *json.Object, done: bool = false, finished: bool = false, @@ -86,6 +87,9 @@ pub const Stream = struct { if (u != .object) return error.InvalidUsage; s.usage = u; } + if (chunk.get("speculative")) |d| if (d == .object) { + s.speculative = d; + }; const reported = chunk.get("tensorfold") orelse chunk.get("runtime"); if (reported) |r| if (r == .object) for (r.object.keys(), r.object.values()) |k, v| try s.runtime.put(s.a, k, v); const choices = chunk.get("choices") orelse Value{ .array = &.{} }; @@ -148,6 +152,7 @@ pub const Stream = struct { sample.server_decode_seconds = runtimeNumber(s.runtime, "decode_seconds", "decode_s"); sample.prefill_seconds = runtimeNumber(s.runtime, "prefill_seconds", "prefill_s"); sample.token_sha = tokenSha(s.a, s.runtime.get("token_sha")); + sample.drafts = drafts(s.speculative); if (s.first) |f| sample.ttft_seconds = @max(0, seconds(f - start)); if (s.deliveries >= 2 and s.last.? > s.first.?) { const span = seconds(s.last.? - s.first.?); @@ -170,6 +175,17 @@ fn count(v: ?Value) ?u64 { return std.fmt.parseInt(u64, x.int, 10) catch null; } +/// The final chunk's speculative rounds, drafted and accepted; null unless all are counts and accepted <= drafted. +fn drafts(v: ?Value) ?suite.Drafts { + const d = v orelse return null; + const out: suite.Drafts = .{ + .rounds = count(d.get("rounds")) orelse return null, + .drafted = count(d.get("drafted")) orelse return null, + .accepted = count(d.get("accepted")) orelse return null, + }; + return if (out.accepted <= out.drafted) out else null; +} + /// ``key``, else ``fallback``, from the telemetry block: a finite non-negative number, else null. fn runtimeNumber(runtime: *const json.Object, key: []const u8, fallback: []const u8) ?f64 { const v = runtime.get(key) orelse runtime.get(fallback) orelse return null; diff --git a/zig/src/server/benchmark_sse_test.zig b/zig/src/server/benchmark_sse_test.zig index c6fb746b70..46eb6118f8 100644 --- a/zig/src/server/benchmark_sse_test.zig +++ b/zig/src/server/benchmark_sse_test.zig @@ -55,6 +55,22 @@ test "CRLF data lines of one event and a keepalive comment make one event" { try std.testing.expectEqual(@as(u32, 1), s.deliveries); } +test "the finishing chunk's speculative counters reach the sample only when all three are counts and accepted fits drafted" { + var arena: std.heap.ArenaAllocator = .init(std.testing.allocator); + defer arena.deinit(); + const a = arena.allocator(); + const counted = "data: {\"choices\": [{\"index\": 0, \"delta\": {}, \"finish_reason\": \"length\"}], \"speculative\": {\"rounds\": 14, \"drafted\": 13, \"accepted\": 10, \"acceptance_rate\": 0.769}}\n\n"; + const got = try play(a, try joined(a, &.{ text1, text2, counted, usage4, done }), &stamps, 4); + try std.testing.expectEqual(suite.Status.ok, got.status); + try std.testing.expectEqual(suite.Drafts{ .rounds = 14, .drafted = 13, .accepted = 10 }, got.drafts.?); + try std.testing.expect((try play(a, try joined(a, &.{ text1, text2, final, usage4, done }), &stamps, 4)).drafts == null); + for ([_][]const u8{ "{\"rounds\": 1, \"drafted\": 1, \"accepted\": 2}", "{\"rounds\": 1, \"drafted\": 2}", "{\"rounds\": -1, \"drafted\": 2, \"accepted\": 1}", "{\"rounds\": 1.5, \"drafted\": 2, \"accepted\": 1}", "[1, 2, 3]" }) |odd| { + const chunk = try std.fmt.allocPrint(a, "data: {{\"choices\": [{{\"index\": 0, \"delta\": {{}}, \"finish_reason\": \"length\"}}], \"speculative\": {s}}}\n\n", .{odd}); + const s = try play(a, try joined(a, &.{ text1, text2, chunk, usage4, done }), &stamps, 4); + try std.testing.expect(s.drafts == null and s.status == .ok); + } +} + test "one stamp is unmeasured, two are ok, and missing telemetry stays null" { var arena: std.heap.ArenaAllocator = .init(std.testing.allocator); defer arena.deinit(); diff --git a/zig/src/server/benchmark_suite.zig b/zig/src/server/benchmark_suite.zig index 56df6d983b..5db35ca877 100644 --- a/zig/src/server/benchmark_suite.zig +++ b/zig/src/server/benchmark_suite.zig @@ -86,8 +86,13 @@ pub const Sample = struct { prefill_seconds: ?f64 = null, token_sha: ?[]const u8 = null, error_code: ?ErrorCode = null, + /// The server's draft counters for this attempt; kept in the local draft counts file, never in the receipt. + drafts: ?Drafts = null, }; +/// A reply's ``speculative`` counters: verify rounds, tokens a drafter proposed and tokens the target kept. +pub const Drafts = struct { rounds: u64, drafted: u64, accepted: u64 }; + /// A cell's medians are set only when every declared repeat is ok. pub const Cell = struct { fixture_id: []const u8, diff --git a/zig/src/server/root.zig b/zig/src/server/root.zig index c7b31b5004..7d0d8e93b6 100644 --- a/zig/src/server/root.zig +++ b/zig/src/server/root.zig @@ -104,6 +104,7 @@ test { _ = @import("benchmark_server_test.zig"); _ = @import("benchmark_facts_test.zig"); _ = @import("benchmark_manifest.zig"); + _ = @import("benchmark_drafts.zig"); _ = @import("benchmark_upload.zig"); _ = @import("benchmark_upload_test.zig"); _ = @import("benchmark_args.zig");