CUDA MPS SM-percentage strong-scaling sweep

[philipnickel]

Summary

Adds the ability to clamp the process to a fraction of the GPU's SMs via CUDA MPS, and a strong-scaling sweep over that fraction.

• mps_thread_percent config option (conf/config.yaml, default null). When set, a preamble at the top of simulate.py exports CUDA_MPS_ACTIVE_THREAD_PERCENTAGE before import jax — required because CUDA reads the clamp once, when JAX creates its context. Recorded in metrics for the plot x-axis.
• conf/sweep/sm_scaling.yaml — Hydra-native sweep: fixed N=10M / G=128, backends swept in-process, a static sweep dir so repeated invocations aggregate, and the standard 3 plots (x = MPS thread %).
• run_sm_scaling.sh — the irreducible per-process loop. CUDA fixes the SM clamp at context creation and JAX keeps one context per process, so each percentage must be its own process (same constraint as MPI rank layout at MPI_Init — mirrors the run_solver trick from the LSM-P2 codebase). The script starts the MPS daemon only for the sweep and stops it after, so ordinary pixi run commands never depend on MPS.
• pixi run sweep-sm [overrides] task; mps-stop / mps-status helpers.

Why no Hydra multirun / activation hook

• An in-process multirun over mps_thread_percent can't work: all jobs share one CUDA context at the first job's clamp.
• An earlier activation hook that auto-started MPS on every pixi run coupled all commands (tests, sims) to a healthy daemon and caused intermittent "no supported devices" failures. Removed — MPS is now started only by run_sm_scaling.sh.

Result (cutile_v3, N=10M, G=128)

SM %	ms/substep	speedup vs 100%
10	276.3	0.12x
25	96.2	0.34x
50	49.4	0.66x
75	37.5	0.87x
100	32.8	1.00x

Near-linear to ~50% SMs, then strong-scaling saturation as fixed per-step overhead dominates.

Also in this branch

• PLOTTING_TODO.md — nsight-metric plots to add (atomic throughput/replay, warp stalls, L2 hit rate).
• Benchmark/default sim tweaks (steps_per_frame, num_grids).

Test plan

• pixi run lint clean
• config-contract + metrics tests pass (no MPS dependency)
• pixi run sweep-sm backend=cutile_v3 — MPS clamp varies correctly, daemon self-cleans

🤖 Generated with Claude Code