No memory limit on Windows

[natedemoss]

eval/sandbox.py applies RLIMIT_AS, RLIMIT_NPROC, RLIMIT_FSIZE and
os.setsid() through preexec_fn, which is POSIX-only and returns None on
Windows. On Windows the only controls left are the subprocess timeout and
interpreter isolation — no memory cap at all.

This is documented in SECURITY.md, and test_memory_limit_is_enforced_on_posix
is skipped on Windows, so it is honest rather than hidden. But contributors do
develop on Windows, and a generated program that allocates without bound will
take the machine down rather than being killed.

Options: a Job Object via pywin32, or simply refusing to run the judge on
Windows without an explicit --i-know-there-are-no-limits flag. The second is
much less code and arguably the more honest interface.

preexec_fn is also unsafe in the presence of threads, and data/verify.py
calls into the judge from a ThreadPoolExecutor — worth confirming this is
actually safe as used, since Python's own docs warn about it.

[natedemoss]

Findings from the 12-hour data-prep failure, since they land squarely on this issue's preexec_fn note.

What happened: the first GPU run produced 446 examples and then made no further progress until Kaggle killed it at the 12h cap. I initially described this as "93 seconds per example," which was wrong — that number is just 12h divided by 446. The rate was almost certainly normal and then went to zero.

Measurements, all on Kaggle unless noted:

Test	Result
CPU kernel, all components	0.21 s/example projected
GPU kernel, full pipeline, 20 min	0.59 s/example, stable, 29 GB RAM free
GPU kernel, old vs new sandbox A/B, 60 traces	0.12 vs 0.08 s/trace
GitHub Linux CI, threaded verification	within budget
preexec_fn serial spawn ×10	0.107 s, same as bare spawn
RLIMIT_NPROC on Kaggle	unlimited; container user has 3 processes

So none of the steady-state explanations survive. Every arm is fast, including the old code.

The mechanism that does fit is an unbounded hang rather than a slowdown. subprocess.Popen.__init__ blocks reading the exec-status pipe from the child with no timeout. preexec_fn runs in the child between fork and exec, and a forked child can deadlock there on a lock another thread held at fork time — the documented reason preexec_fn is unsafe with threads. If that happens, the parent blocks in Popen.__init__ before subprocess.run's timeout logic is reachable, so the timeout never fires and the worker never returns.

That explains why every test above missed it: at 60–2000 traces the per-fork probability is low, and the failure only shows up after several hundred. It is consistent with all the evidence but not proven — a probabilistic hang cannot be confirmed by a run that does not hang.

The fix (#623c831, corrected in #021b035) removes preexec_fn entirely: rlimits are applied by a bootstrap inside the child, and isolation uses start_new_session, which is thread-safe. That eliminates the hazard whether or not it was the cause here.

RLIMIT_NPROC was also dropped, for the independent reason in this issue: it is a per-user limit, so it is simply wrong inside a shared container. Measured unlimited on Kaggle, so it was not implicated this time.

Still open here: the Windows memory-limit gap, which is unaffected by any of the above.