IVF-SQ currently fits centers, assigns vectors to partitions, and encodes residual SQ8 values on CPU. Accelerating center fitting alone leaves full-corpus assignment and encoding on the construction critical path.
I propose an optional Python cuVS/CuPy adapter, backed by backend-independent native interfaces, to accelerate both training and construction while keeping the v1 index format and existing CPU readers.
Scope
- Prepare an owned, bounded training sample using native sampling and metric preprocessing; expose the resolved parameters.
- Accept validated external IVF centers and finish residual SQ calibration in Rust.
- Export an owned encoding-model snapshot and accept validated preassigned vectors or encoded SQ8 batches.
- Provide reusable GPU workers for center fitting, exact partition assignment, and optional SQ8 encoding. Keep serialization and querying on CPU.
GPU construction should use a model calibrated with exact assignment. CPU and GPU distance calculations may choose different partitions near ties, so validation must check retrieval quality. For identical assignments, GPU encoding should reproduce native SQ8 arithmetic, including rounding, clipping, scalar tails, constant bounds, and subnormal values.
Acceptance criteria
- No mandatory CUDA dependency, change to default CPU behavior, or file-format change.
- Explicit ownership and batch validation at the Rust/C/Python boundaries; invalid batches are rejected before append.
- GPU errors propagate without a silent CPU fallback; cleanup preserves the original operation error.
- CPU CI covers native interfaces and lifecycle/error contracts. Explicit GPU tests cover L2, cosine, inner product, numerical parity, and held-out retrieval.
- Controlled ablations compare CPU construction, GPU assignment with CPU encoding, and GPU assignment with GPU encoding using the same sample and centers. Report repeated stage timings, recall, hardware, and timing boundaries.
CPU SQ calibration remains a follow-up optimization, recorded as a TODO at its implementation entry point. GPU querying, multi-GPU execution, automatic batch sizing, and an out-of-core writer are outside this proposal. Device scratch is batch-sized; the native writer still retains the complete encoded index in host memory.
The GPU adapter would initially be experimental, with broader GPU/CUDA/cuVS version coverage left for follow-up.
IVF-SQ currently fits centers, assigns vectors to partitions, and encodes residual SQ8 values on CPU. Accelerating center fitting alone leaves full-corpus assignment and encoding on the construction critical path.
I propose an optional Python cuVS/CuPy adapter, backed by backend-independent native interfaces, to accelerate both training and construction while keeping the v1 index format and existing CPU readers.
Scope
GPU construction should use a model calibrated with exact assignment. CPU and GPU distance calculations may choose different partitions near ties, so validation must check retrieval quality. For identical assignments, GPU encoding should reproduce native SQ8 arithmetic, including rounding, clipping, scalar tails, constant bounds, and subnormal values.
Acceptance criteria
CPU SQ calibration remains a follow-up optimization, recorded as a TODO at its implementation entry point. GPU querying, multi-GPU execution, automatic batch sizing, and an out-of-core writer are outside this proposal. Device scratch is batch-sized; the native writer still retains the complete encoded index in host memory.
The GPU adapter would initially be experimental, with broader GPU/CUDA/cuVS version coverage left for follow-up.