Skip to content

fix: make train --hf-jobs work on Windows - #35

Open
lboucher26 wants to merge 2 commits into
pollen-robotics:developfrom
lboucher26:fix/hf-jobs-windows
Open

lboucher26 wants to merge 2 commits into
pollen-robotics:developfrom
lboucher26:fix/hf-jobs-windows

Conversation

@lboucher26

Copy link
Copy Markdown

train <task> --hf-jobs is unusable on Windows. Two independent bugs, both
one-liners, both invisible in CI because they only trigger on nt. Found
while following scripts/hf/README.md from a clean clone.

1. The flag is never intercepted (train_hook.py)

_invoked_as_train() compares the console script's bare name to "train",
stripping .py and -script. Windows installs the script as train.exe, so
the comparison fails, the hook returns early, and the flag reaches tyro:

Unrecognized options: --hf-jobs, --dry-run

Which is the same symptom #20 fixed on Linux. The cause is easy to miss
because tyro prints the program without the extension —
C:\...\.venv\Scripts\train — so argv[0] looks like it should have matched.
Confirmed with a sitecustomize probe on the real invocation:

ARGV0 = 'C:\GitHub\microduck_rl\.venv\Scripts\train.exe'
name  = 'train.exe'

Every existing case in test_hf_jobs_flag.py uses posix argv[0]
(/x/.venv/bin/train), so the suite stayed green. Now parametrized over the
spellings a console script actually takes, with play.exe added to the
negative case so the suffix strip can't over-reach.

2. The wandb key is never found (hf_jobs.py)

wandb login writes ~/_netrc on Windows and says so:

Appending key for api.wandb.ai to your netrc file: C:\Users\...\_netrc

_wandb_api_key() reads only ~/.netrc, so submission reports

[wandb] x no API key found (checked $WANDB_API_KEY and ~/.netrc).
        Run `wandb login` locally, or pass --no-wandb to skip.

immediately after a successful login. The suggested remedy is a dead end:
wandb login again just rewrites _netrc. Now tries both spellings, with a
new test_hf_jobs_wandb_key.py covering both, env-var precedence, and the
no-key path.

Verification

uv run --with pytest pytest tests/test_hf_jobs_flag.py tests/test_hf_jobs_wandb_key.py
— 21 passed (see note below for the 1 unrelated pre-existing failure).

End-to-end on Windows, which previously could not get past argument parsing:

[hf] namespace: <user>
[wandb] forwarding API key from ~/.netrc
[src] building tarball -> src-20260902-223214.tar.gz
[src] HEAD=5946fd9, 9.9 MB
[dry-run] would submit job:
  flavor:    l4x1, timeout: 12h
  secrets:   {'HF_TOKEN': '***', 'WANDB_API_KEY': '***'}

Not fixed here

test_train_on_path_is_a_mjlab_trainer also fails on Windows, for an
unrelated reason: it looks for b"mjlab" in the first 8192 bytes of the
resolved script, but a Windows console script is a PE launcher stub with the
payload appended at the end. In this venv train.exe is 46080 bytes and
mjlab first appears at offset 43543, so the check can't pass on Windows
even when the script is correct. Left alone to keep this PR to the two
functional bugs — happy to add a commit or open a separate issue, whichever
you prefer.

🤖 Generated with Claude Code

https://claude.ai/code/session_019oSADSTKCB8mSdb6yFpvLt

lboucher26 and others added 2 commits September 2, 2026 22:43
argv[0] is however the OS spells the installed console script, and Windows
spells it `train.exe`. `_invoked_as_train()` stripped `.py` and `-script`
but not `.exe`, so the bare-name comparison never matched, the hook declined
to intercept, and the flag fell through to tyro:

    Unrecognized options: --hf-jobs, --dry-run

i.e. the exact failure this hook was added to prevent, one platform over.
Nothing warns about it: the submission path is simply absent on Windows.

tyro's error hides the cause, printing the program as `...\Scripts\train`
with the extension stripped, so the name looks like it should have matched.

The existing cases all used posix argv[0] (`/x/.venv/bin/train`), so they
passed throughout. Parametrized over the spellings a console script actually
takes, with `play.exe` added to the negative case so the suffix strip can't
over-reach into "every .exe is the trainer".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019oSADSTKCB8mSdb6yFpvLt
`wandb login` follows the platform's netrc convention and writes `~/_netrc`
on Windows, printing the path it chose:

    Appending key for api.wandb.ai to your netrc file: C:\Users\...\_netrc

`_wandb_api_key()` only looked at `~/.netrc`, so submission reported

    [wandb] x no API key found (checked $WANDB_API_KEY and ~/.netrc).
            Run `wandb login` locally, or pass --no-wandb to skip.

directly after a successful login, and refused to forward a key that was
sitting in the user's home directory. The advice in the message is also a
dead end on Windows: running `wandb login` again just rewrites `_netrc`.

Try both spellings. The stdlib `netrc` default path doesn't know about
`_netrc`, and this passes an explicit path anyway, so the fallback has to be
here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019oSADSTKCB8mSdb6yFpvLt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant