[Bug]: Server rejects --prefill-chunk-tokens auto while the CLI accepts it

[13inks]

Commit

Reproduced on fd3c0b8be8dfbf00d34123dbe6efc39ee5424668.

Verified still present on main @ 614a8f6 by source inspection — I did not
rebuild main because this checkout is mid-benchmark-campaign and its binaries
are pinned for comparability. Happy to retest on main if useful.

Mac hardware and memory

M5 Pro MacBook Pro (Mac17,9), 24 GB — hardware-independent, this is argument parsing.

macOS and Swift versions

macOS 26.5.1 (25F80), Swift 6.3.3, Xcode 26.6 (17F113)

Reproduction steps

# CLI accepts auto
.build/release/TurboFieldfareCLI --model scratch/gemma4.gturbo \
  --prompt "hi" --max-new 8 --prefill-chunk-tokens auto        # runs

# Server refuses it
.build/release/TurboFieldfareServer --model scratch/gemma4.gturbo \
  --port 8098 --prefill-chunk-tokens auto                      # prints usage, exits

No model strictly needed for the server side — it fails during argument parsing.

Expected behavior

The server accepts auto, matching the CLI.

docs/RUNTIME_CONTROLS.md states the invariant directly:

The CLI and the local server accept these flags with the
same names and values.

and documents the value in the runtime-settings table:

--prefill-chunk-tokens 32\|64\|128\|256\|auto … auto picks the smallest
size that covers the prompt.

Sources/TurboFieldfareServer/Core/ServerArguments.swift:44 states the same
intent in code:

// Mirrors the CLI's runtime flags so both binaries accept the same options
// with the same validation, instead of the server pinning production defaults.

Observed behavior

auto is CLI-only.

Sources/TurboFieldfareCLI/Args.swift:301 handles it:

case "--prefill-chunk-tokens":
    let value = try takeValue(argv, &index, flag: flag)
    if value == "auto" {
        prefillChunkTokensAuto = true
        break
    }

ServerArguments.swift has no equivalent branch, so auto falls through to
Int(value) and is rejected. The two usage strings disagree accordingly:

• CLI: Prefill chunk size: 32, 64, 128, 256, or auto
• Server: Prefill chunk size: 32, 64, 128, or 256

Note that #165 improved the server's rejection messages by deriving them from
RuntimeConfiguration.allowedPrefillChunkTokens — which is the right fix for
that bug, but it cannot surface auto, since auto is a parse-level alias
rather than a member of the allowed-values array. So on main the server's
help and rejection text are now self-consistent and still both omit auto.

Suggested fix: give the server the same auto branch as the CLI and append it
to the server's usage string. Alternatively, if auto is intentionally
CLI-only, RUNTIME_CONTROLS.md should say so rather than asserting the flags
match.

Does this require the installed model?

No — argument parsing only. A model is needed only to confirm the CLI's auto
path then generates normally, which it does.

---

_{Found while sweeping --prefill-chunk-tokens against TTFT (M5 Pro, internal APFS, --max-context 4096, docs/benchmark-prompts/real-generation-v1). 256 was fastest at every prompt length here — 16.07 s vs 17.41 s TTFT at 3,015 prompt tokens, and 2.4x better than 32 — consistent in direction with the 16% figure in RUNTIME_CONTROLS.md. auto was the one setting I could not measure on the server, which is how this surfaced.}

[13inks]

Correcting the closing note in my own report — the "~5 s fixed first-token
overhead" observation was wrong, and I would rather retract it than leave it
in the record for someone to chase.

It was cold start, not a floor. Every measurement in that sweep started a
fresh server and ran the short prompt first, so the short case always paid
one-time warmup — which is also exactly why it appeared not to respond to chunk
size. It was never measuring chunking.

Re-measured with a warm server, same host (M5 Pro, internal APFS,
--max-context 4096, --prefill-chunk-tokens 256), three samples per point:

prompt tokens	44	96	174	330	538	798	1,188	1,968	2,878
TTFT (s)	0.29	0.48	0.73	1.48	2.43	3.71	5.74	10.30	15.48

Samples agreed to within 0.02 s at every point, so TTFT looks close to
deterministic once warm. It scales near-linearly at roughly 5.4 s per 1,000
prompt tokens, with cost per 1k rising gently from 4.2 s at 174 tokens to
5.4 s at 2,878.

So there is no fixed overhead worth investigating — 44 tokens is 0.29 s, not
~5 s. Apologies for the noise.

The auto asymmetry in the issue body is unaffected and still reproduces.