Skip to content

Add Windows wheels to the build and release pipeline - #7

Open
tomsch420 wants to merge 5 commits into
mainfrom
windows-wheels
Open

tomsch420 wants to merge 5 commits into
mainfrom
windows-wheels

Conversation

@tomsch420

@tomsch420 tomsch420 commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Adds Windows (AMD64) wheels to the release, next to the existing manylinux wheels, and fixes the code paths that hung or failed on Windows.

CI on this PR passes on both Linux and Windows. On Windows the extensions compile with MSVC, all 1,047 tests pass (4 skipped), and cibuildwheel builds pyjpt-…-cp311-cp311-win_amd64.whl, which passes an import check.

Build

  • setup.py: passes the numpy include dir to every extension. CPATH is only honored by GCC/Clang, not MSVC.
  • intset.pyx: drops redundant typing annotations on IntSet.from_list / from_set. Cython 3.3 rejects them as conflicting signatures, so the build on main currently fails with the unpinned Cython.

Windows runtime fixes

  • c45.py index buffer: allocated as c_int64 instead of c_long. It's read back as np.int64, but long is 32-bit on Windows.
  • Multiprocessing fallback: C4.5 learning, preprocessing, likelihood inference and leaf rendering keep their state in a module-level threading.local that only fork-started workers inherit. Under spawn (Windows; macOS default) the workers fail and the pool hangs. New multicore.workers_inherit_state(); when it is false, these use the existing in-process DummyPool (the same path multicore=0 takes). Windows therefore learns on a single core for now; real spawn support would need the state passed to the workers explicitly.
  • kaleido: the plotly extra installs kaleido==0.1.0.post1 on Windows. kaleido 0.2.x hangs indefinitely in write_image on Windows.
  • Plot output directory: defaults to tempfile.gettempdir() instead of /tmp, which doesn't exist on Windows.

pandas 3

  • c45.py training data: made writable and C-contiguous with np.require. Under pandas 3, DataFrame.values can be read-only, and the Cython impurity code then made the C4.5 worker pool hang.

CI

  • Windows job: compiles the package, runs the test suite (verbose, with a 20-minute faulthandler watchdog), and builds AMD64 wheels with cibuildwheel for the same python-versions as the manylinux build. The Linux job is unchanged apart from its artifact name (pyjpt-wheel-linux).
  • Publish: downloads all pyjpt-wheel-* artifacts.
  • Triggers: the workflow now runs on pull requests, with a per-ref concurrency group.
  • Test-install constraints (CI only): pandas<3 and plotly<6.1. Package metadata is unchanged.

Known issues on main, not addressed here

  • pandas 3 str dtype: variable inference doesn't recognise pandas 3's default str dtype (Unknown column type: … [str], about 50 test errors).
  • kaleido pin: plotly >= 6.1 needs kaleido>=1, while the extra pins kaleido<1.0.

Not yet exercised

The publish workflow only runs on version tags, so collecting several artifacts in the publish step is untested until the next release. The release workflow passes cp39–cp312 to cibuildwheel; this PR's run only built cp311.

🤖 Generated with Claude Code

- setup.py: pass the numpy include dir to every extension explicitly;
  CPATH is only honored by GCC/Clang, not by MSVC.
- intset.pyx: drop the redundant typing annotations on IntSet.from_list /
  from_set, which Cython 3.3 rejects as conflicting signatures (the build
  currently fails with the unpinned Cython).
- CI: add a windows-latest job that compiles the package, runs the test
  suite and builds AMD64 wheels with cibuildwheel for the same Python
  versions as the manylinux build; run on pull requests; scope the
  concurrency group per ref.
- Publish all pyjpt-wheel-* artifacts (Linux and Windows).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tomsch420 and others added 4 commits November 20, 2026 13:52
…as 3

- c45.py: allocate the shared index buffer as c_int64 instead of c_long.
  It is read back as np.int64, but C long is 32 bits on Windows.
- c45.py: make the training data writable and C-contiguous with
  np.require. Under pandas 3, DataFrame.values can be read-only; the
  Cython impurity then raised in a worker and the pool hung forever.
- CI: constrain pandas<3 and plotly<6.1 for the test install until the
  pandas 3 str dtype and the kaleido>=1 requirement are supported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
C4.5 learning, preprocessing, likelihood inference and leaf rendering
fill a module-level threading.local in the parent and let pool workers
read it. That only works with the "fork" start method. Under "spawn"
(Windows; macOS default) the workers start without that state, fail,
and the pool never completes, so the test suite hung on Windows CI.

Add multicore.workers_inherit_state() and use DummyPool (the existing
multicore=0 path) when it is False. Verified on Linux by running the full
suite with the start method forced to "spawn".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run the suite verbosely with a faulthandler watchdog that dumps all thread
stacks and exits after 20 minutes, cap the step at 30 minutes, and force
the non-interactive Agg matplotlib backend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- pyproject.toml: on Windows, install kaleido==0.1.0.post1 with the plotly
  extra. kaleido 0.2.x hangs indefinitely on Windows in write_image (seen
  as a 20 minute stall in test_pdf_grid_2_plotly on CI).
- Plotting engines and Distribution.plot: default the output directory to
  tempfile.gettempdir() instead of '/tmp', which does not exist on Windows
  (the matplotlib save_plot does not create it, so five plot tests failed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant