Collecting the small stuff so it doesn't get lost. Each line is independent, split
any of them out if it turns into real work.
| Where |
What |
src/nn/linear.rs:33 |
Linear::no_bias builds a full Linear and throws the bias away, so it still pulls out_features samples from the RNG. Harmless today (checked, seeded weights match), but it ties the RNG stream to an allocation we discard. |
src/nn/embedding.rs:60 |
Embedding::forward does v as usize on an f32, and a negative value saturates to 0 and silently reads row 0. |
src/data/loader.rs:26 |
Batch::labels() has the same as usize saturation. |
src/tensor/ops/reduce.rs:105 |
variance() divides by data.len() with no guard for an empty tensor. |
src/rng.rs:28 |
uniform(n, lo, hi) calls gen_range(lo..hi), which panics on an empty range. Reachable through kaiming_uniform with fan_in == 0. |
src/tensor/ops/mod.rs:41 |
PARALLEL_THRESHOLD is a flat element count with no regard for how expensive f is, so gelu (an erff per element) stays serial below 32k while neg goes parallel above it. |
src/serialize/safetensors.rs:70 |
The whole file is built in one Vec before fs::write, so peak memory is 2x the model. |
src/serialize/safetensors.rs:143 |
used.contains(&effective) inside a loop over all entries is O(n^2) in tensor count. |
src/serialize/checkpoint.rs:93 |
Values go out one write_all(&[u8; 4]) at a time and come back the same way. Fine through BufWriter, but a bulk cast would be a big constant factor win on large models. |
cuda/kernels.cu:180 |
LAUNCH_ELEMENTWISE casts size_t n to int for the block count, silently wrong past 2^31 elements. Same cast in fastnn_cuda_sum, _max, _mse_loss, _binary_cross_entropy. |
cuda/kernels.cu:679 |
fastnn_cuda_sum with n == 0 computes blocks == 0 and launches an invalid grid. |
src/autograd/engine.rs |
The graph and every intermediate gradient stay alive for the whole reverse pass, since order holds a clone of every node. Peak memory is activations plus gradients at the same time, where PyTorch frees each node's saved tensors as it retires them. Worth at least a note in the module docs. |
Collecting the small stuff so it doesn't get lost. Each line is independent, split
any of them out if it turns into real work.
src/nn/linear.rs:33Linear::no_biasbuilds a fullLinearand throws the bias away, so it still pullsout_featuressamples from the RNG. Harmless today (checked, seeded weights match), but it ties the RNG stream to an allocation we discard.src/nn/embedding.rs:60Embedding::forwarddoesv as usizeon anf32, and a negative value saturates to 0 and silently reads row 0.src/data/loader.rs:26Batch::labels()has the sameas usizesaturation.src/tensor/ops/reduce.rs:105variance()divides bydata.len()with no guard for an empty tensor.src/rng.rs:28uniform(n, lo, hi)callsgen_range(lo..hi), which panics on an empty range. Reachable throughkaiming_uniformwithfan_in == 0.src/tensor/ops/mod.rs:41PARALLEL_THRESHOLDis a flat element count with no regard for how expensivefis, sogelu(anerffper element) stays serial below 32k whileneggoes parallel above it.src/serialize/safetensors.rs:70Vecbeforefs::write, so peak memory is 2x the model.src/serialize/safetensors.rs:143used.contains(&effective)inside a loop over all entries is O(n^2) in tensor count.src/serialize/checkpoint.rs:93write_all(&[u8; 4])at a time and come back the same way. Fine throughBufWriter, but a bulk cast would be a big constant factor win on large models.cuda/kernels.cu:180LAUNCH_ELEMENTWISEcastssize_t ntointfor the block count, silently wrong past 2^31 elements. Same cast infastnn_cuda_sum,_max,_mse_loss,_binary_cross_entropy.cuda/kernels.cu:679fastnn_cuda_sumwithn == 0computesblocks == 0and launches an invalid grid.src/autograd/engine.rsorderholds a clone of every node. Peak memory is activations plus gradients at the same time, where PyTorch frees each node's saved tensors as it retires them. Worth at least a note in the module docs.