RTX 3090 runs on the 5090 cost profile: prefill predicted 2.3x too low, Host KV never used at --max-concurrency 1

[iamwavecut]

Related: #98, #51, #120, #144, #142.

What I see

On an RTX 3090 the planner starts with context-cost-transfer=generic-default context-cost-prefill=generic-default. The compiled registry has one hardware class, nvidia-geforce-rtx-5090-sm120, and the generic fallback is that same 5090 profile. On the 3090 those numbers are off by a lot:

	planner predicts	measured on the 3090
prefill 34,876 tokens from root	14.1 s	44.6 s
prefill 20,851 tokens after 73k reused	18.6 s	42.0 s
prefill 119,806 tokens from root	86.1 s	161.9 s
all 127 requests with >2k prefilled tokens, measured/predicted		p50 2.27× (p10 2.17, p90 2.63)

The transfer side is off too: generic assumes 0.025 ns/byte (~40 GB/s), this machine has a PCIe 3.0 x8 link (~6 GB/s).

Since FutureLoss = w · Recovery and Recovery comes from this model, every retained checkpoint is valued at less than half of what it costs to rebuild. Dropping a parked context looks cheap, keeping or demoting it looks expensive.

Over 19 hours of one agent session at --max-concurrency 1 with the default 8 GiB Host KV:

main_kv_transfers   d2h 0 B  h2d 0 B  d2d 0 B        (761 telemetry intervals)
state_transfers     d2h 121 ops  h2d 98 ops         (so the Host path itself works)
captures            121 completed, 0 aborted
pressure            checkpoints_dropped 5, private_owners_evicted 3

Not a single KV byte went to Host; the only Host traffic is StateImages. This is the #51 shape the planner was rebuilt for, and I think the card-specific constants are what decide it.

Environment

• Fork of release/v0.6.2-rtx3090 with upstream merged through 93cdc264, plus On-demand vision residency: stream the tower through evicted read-only text weights #72. context_cost_defaults.cpp is identical on master, so the constants argument does not depend on the newer runtime fixes (3d9fda22, 138d76ae etc., which this build lacks).
• RTX 3090 24 GiB, driver 580.126, CUDA 13.1, PCIe Gen 3 x8, 62 GiB host RAM.
• qwen3_8_27b.ninfer (groupwise-int), --kv-dtype rk8v4 --max-context 203200 --kv-capacity 203200 --max-concurrency 1 --prefill-chunk 1024 --spec mtp --draft-tokens 3 --lm-head-draft --vision --vision-residency overlay --vision-max-merged 12288, cache flags at defaults. Startup line ends with cost-profile=nvidia-geforce-rtx-3090-sm86/qwen3.8-27b/groupwise-int.
• Workload: one coding agent over Anthropic Messages, max_tokens=32000, prompts 86k → 170k tokens. Numbers above are from the request log (schema 18): materialization.predicted_now_ns vs timings_seconds.prefill. I checked the decode: 35·40.8 ms + 34,876·0.2357 ms + 6.08e8·7.10 ns = 13.97 s matches the logged 14.13 s.

What I would do, if you agree

1. Measure the 3090 with ninfer_context_cost_bench --suite all on a clean card and add a compiled nvidia-geforce-rtx-3090-sm86 preset next to the 5090 entry. Registry-only change, no planner semantics touched. I can attach the bench report.
2. One startup log line when both cost sources resolve to generic-default, pointing at --context-cost-presets and the bench.

If you would rather keep the registry to hardware you measure yourself, that is fine — the report stands on its own and I will carry the preset locally.

Not verified

The ratios come from a production card that shares the GPU with another service, not from the bench corpus. I have not re-measured on master.

[iamwavecut]

Closing this myself: the local override path (--context-cost-presets) covers our card, and the registry is your call for hardware you measure. Leaving the numbers here in case they are useful; no action needed.