fix(matmul): count the GEMM output tile in auto_tile's VRAM budget

[boskodev790]

Lane

fix / bug — corrects repository behavior. CPU-safe validation only; no GPU scorecard (does not enter the GPU queue).

Problem

auto_tile sizes the tile edge T so the per-(i, j, k) working set fits vram_fraction of free VRAM, using _tile_workspace_bytes_per_elem. That model counted only three T×T tiles — the accumulator plus the two operand tiles:

return cfg.acc_dtype.itemsize + 2 * _tile_operand_bytes(cfg)

But _gemm_tiled_sync also holds a fourth T×T tensor live at the same time — the freshly allocated product from backend.matmul (torch.bmm cannot write its result into either operand), so at each k-step acc, a_dev, b_dev and prod coexist. The missing term is one operand-sized tile (the output, produced in the compute dtype).

Per element (modelled vs actual): fp32 12 vs 16; fp16+accumulate_fp32 12 vs 16; fp16 raw 6 vs 8. Since T ∝ 1/√(per_elem), undercounting by 4/3 makes auto_tile pick T ≈ √(16/12) ≈ 1.15× too large, so the real working set is ≈1.33× the intended vram_fraction. At a still-valid vram_fraction ≥ 0.75 the overshoot pushes the per-step working set past 100% of free VRAM and the tiled/out-of-core path OOMs — on exactly the huge-N runs tiling exists to enable. Same class of under-budget as #74 (operand upcast), on the still-missing GEMM output term.

Fix

Budget the output tile as well:

return cfg.acc_dtype.itemsize + 3 * _tile_operand_bytes(cfg)

Only matmul/gemm.py (the model + its docstrings) and tests/ change — no scorer/eval/docs/CI paths.

Tests

• Updated the fp16 workspace assertions to the corrected values (12 → 16, 6 → 8).
• Added test_tile_workspace_counts_the_bmm_output_tile (regression for [bug] auto_tile's per-step VRAM model omits the GEMM output tile, over-sizing T by ~15% and eroding the OOM margin #95): asserts the model equals acc + 3 * operand and is exactly one operand tile larger than the old 3-term model, for fp32 / fp16+acc / fp16-raw. It fails on the pre-fix code and passes after.

uv run python -m strategy.smoke
uv run --extra test python -m pytest tests/ strategy/tests/ eval/tests/ -q
# 78 passed, 23 skipped

Fixes #95