Tracking issue for the coordinated fix of the TurboQuant contract defect across
backends. Contract corrected to version 2 in 613c73c.
The defect
specs/formats/turboquant.md version 1 assigned the rotated Lloyd-Max path to
values and plain uniform quantization to keys. That is inverted with respect to
TurboQuant (arXiv:2504.19874): attention
scores are inner products, so keys take the rotated path and values take uniform
quantization. Three backends implemented version 1 faithfully, which is why they
agree with each other and disagree with the algorithm.
Keys are the tensor whose quantization error biases every attention score, and
they are the ones that were getting vanilla uniform quantization. This is a
correctness defect.
Per-backend
Coordination
Stored codes change meaning, so this is a wire-format break. Caches written under
version 1 cannot be migrated in place.
Tracking issue for the coordinated fix of the TurboQuant contract defect across
backends. Contract corrected to version 2 in 613c73c.
The defect
specs/formats/turboquant.mdversion 1 assigned the rotated Lloyd-Max path tovalues and plain uniform quantization to keys. That is inverted with respect to
TurboQuant (arXiv:2504.19874): attention
scores are inner products, so keys take the rotated path and values take uniform
quantization. Three backends implemented version 1 faithfully, which is why they
agree with each other and disagree with the algorithm.
Keys are the tensor whose quantization error biases every attention score, and
they are the ones that were getting vanilla uniform quantization. This is a
correctness defect.
Per-backend
kernels/serving/; needs reconciliation, not a rewrite)decode)
to unwind
Coordination
Stored codes change meaning, so this is a wire-format break. Caches written under
version 1 cannot be migrated in place.
format_versionadded to cache metadata in every backend, with a hardrefusal on unsupported versions
assignment, so existing tests pass against the wrong behaviour
registry/quant-formats.yamlturboquant status reviewed once backends land(currently
plannedwhile Metal claimsimplemented)there (QuixiCore#14, QuixiCore-CUDA: TurboQuant roles inverted in kernels/quant, and two incompatible implementations coexist #15, QuixiCore-ROCm: TurboQuant roles inverted, plus scale-dtype seam and missing signed sub-8-bit decode #16 closed as duplicates)