src/nn/dropout.rs:41. rng::bernoulli runs on the CPU and the mask goes up with
.to(device) on every call. In a transformer that's one full activation sized H2D
transfer per dropout, per layer, per step, sitting right on the critical path.
kernels.cu already has kernel_dropout_forward with a curand seed
(kernels.cu:1601), it's just never declared in ffi.rs.
src/nn/dropout.rs:41.rng::bernoulliruns on the CPU and the mask goes up with.to(device)on every call. In a transformer that's one full activation sized H2Dtransfer per dropout, per layer, per step, sitting right on the critical path.
kernels.cualready haskernel_dropout_forwardwith a curand seed(
kernels.cu:1601), it's just never declared inffi.rs.