xformers is compiled against torch 2.10 but the image ships 2.14

[blzee-maker]

Problem

Every container logs:

WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 2.10.0+cu128 with CUDA 1208 (you have 2.14.0+cu130)
    Python  3.10.19 (you have 3.11.12)

xformers arrives via sam-audio's dependency tree with a prebuilt binary
matching torch 2.10, while the image installs torch>=2.5 and resolves to
2.14. The extension fails to load and memory-efficient attention is silently
disabled, falling back to a slower, more memory-hungry path.

Why it matters

Two costs, both paid on every request:

• Speed — separations are slower than they should be
• Memory — vanilla attention allocates the full attention matrix. Given the
  OOM headroom problems in Cast weights to bfloat16 so base and large fit on one GPU #1, this may be part of why base does not fit

It is a warning, not an error, so it has been easy to ignore. It is not free.

The real fix

Pin torch to the version xformers was built against:

.pip_install("torch==2.10.*", "torchaudio==2.10.*")

Belongs with the lockfile work in #2 — this is exactly the class of drift a
resolved lockfile prevents.

Alternatives

• Install an xformers build matching torch 2.14, if one exists
• Drop xformers and use PyTorch's native scaled_dot_product_attention, which
  has its own memory-efficient kernels and no version coupling

The third option is probably best long-term: one less binary dependency pinned
to a specific torch build.

Done when

The warning is gone, and a before/after measurement shows the speed and
peak-VRAM difference — worth knowing whether this alone changes what fits.