Skip to content

ci: standalone binary builds (one-file PyInstaller, release matrix) - #137

Merged
marzukia merged 3 commits into
mainfrom
binary/0.5.0-linux
Oct 16, 2026
Merged

marzukia merged 3 commits into
mainfrom
binary/0.5.0-linux

Conversation

@monkytheluffy

Copy link
Copy Markdown
Collaborator

Standalone one-file binaries (PyInstaller) for releases, plus the CI workflow to build them on every release publication.

What it does

.github/workflows/binary.yml (triggers: release: types: [published] + workflow_dispatch) builds the ppgrid console script as a self-contained one-file binary per platform:

  • ubuntu-latest (linux x86_64), macos-latest (macos arm64), macos-13 (macos x86_64)
  • Per leg: platform prebuilt of the C interp using ppgrid._cinterp's own cc recipe (the .so name carries the cinterp.c src hash, so a stale build is never loaded), PyInstaller with --collect-data pyproj + --collect-submodules pyproj (PROJ database), --collect-all rasterio (wheel-bundled GDAL, no PyInstaller hook), --hidden-import pandas (loaded via importlib.import_module, invisible to static analysis), --copy-metadata ppgrid (__version__ via importlib.metadata), --add-data for cinterp.c + the src-hash .so (the loader needs cinterp.c present to even look for the .so).
  • Gates: --version + clean-env e2e (fresh HOME, no venv) on all legs; no-compiler e2e (PATH without cc/gcc/clang, so the bundled tier-1 .so is the only way the C interp can load) on ubuntu; byte-exact anchor gate vs the committed examples/melb (10m/500m, exact bench tier2 config: --value-col price --cap-km 10 --skip-calibration --workers 4) on ubuntu only — the anchors are Linux-anchored, macOS legs get the functional e2e.
  • xz the binary as ppgrid-<ver>-<os>-<arch>.xz; upload to v<ver> via softprops/action-gh-release@v2 (action-release@v2 was renamed upstream; the successor keeps tag-based upload, overwrite by default). On workflow_dispatch with no matching release the build falls back to an actions/upload-artifact so it stays verifiable.

README: one new "Standalone binaries" paragraph (what it is, release-assets location, no-Python/no-GDAL claim, C interp fallback to bit-identical np.interp).

Goal-1 proof (built + tested on hydrogen, v0.5.0, PyInstaller 6.22.3, python 3.13.12)

  • dist/ppgrid --version -> ppgrid 0.5.0, exit 0; --help exit 0
  • clean-env e2e (env -i, fresh HOME, PATH without cc/gcc/clang, data/melb_houses.csv --res 500 --cap-km auto): exit 0, rasters written, no ~/.cache created (tier-1 bundled .so loaded, no JIT, no compiler touched)
  • anchor gate, binary output vs committed examples/melb — all four byte-exact:
    • 10m/value.tif 74d76b9b3a01dbc2ce76bf11c8fcd67d2c8b8b08ad1720b17983c28e00a5db43
    • 10m/support_km.tif 3d0822e1f15b545e359e51f748686d0500053f09b80e646ef7bf22b636a6c47d
    • 500m/value.tif 4a30bf476a85c9d30bb46aa84aa3aa140c532af0d03fda1dc90a72bfda3d45a7
    • 500m/support_km.tif 530cc25f3d4c5ebe2ab17359baeac56c635bc2f0096a14d70d7e790efaad9346
  • linux binary: ppgrid-0.5.0-linux-x86_64.xz, 179,993,860 bytes, sha256 3840181c350516d8066e705e69c0edfa68e4ecdf23b191fbe82acdba392976a
  • Attached to the v0.5.0 release and at https://drop.junkyard.sh/ppgrid-0.5.0-linux-x86_64.xz

Recipe note: the committed-era prebuilt _cinterp_c-314334463687...so in the working tree is a stale JIT artifact (gitignored) whose src hash no longer matches cinterp.c (current hash 450abce519f7, after the darwin-FMA fix 976ff46); this build recompiles with _cinterp._build's exact cc line (-O3 -ffp-contract=off -fPIC -shared -fvisibility=hidden -I<include>), verified by tests/test_cinterp.py (37 passed).

macOS binaries will be produced by running this workflow on the v0.5.0 release after merge (re-publish the release, or workflow_dispatch from the tag).

monkytheluffy added 3 commits October 16, 2026 21:55
Add .github/workflows/binary.yml: on release publication (or
workflow_dispatch), build one-file PyInstaller binaries for
ubuntu-latest (x86_64), macos-latest (arm64), macos-13 (x86_64).

Per-leg recipe (identical to the local 0.5.0 linux build that passed
the anchor gate byte-for-byte): platform prebuilt of the C interp via
ppgrid._cinterp's own cc recipe, --collect-data/--collect-submodules
pyproj (PROJ database), --collect-all rasterio (bundled GDAL),
--hidden-import pandas (lazy importlib.import_module), --copy-metadata
ppgrid (importlib.metadata --version), --add-data cinterp.c + the
src-hash-named .so (the loader needs cinterp.c present to look for it).

Per-leg gates: --version + clean-env e2e (fresh HOME, no venv) on all
legs; no-compiler e2e (PATH without cc/gcc/clang -> the bundled
tier-1 .so is the only way the C interp loads) on ubuntu; byte-exact
anchor gate vs the committed examples (10m/500m, exact bench tier2
config) on ubuntu only (anchors are Linux-anchored; macOS legs get
functional e2e). xz the binary, upload to v<version> via
softprops/action-gh-release@v2 (action-release@v2 was renamed; the
successor keeps tag-based upload with overwrite by default), falling
back to actions/upload-artifact when no release matches (the
workflow_dispatch verification path).

README: short 'Standalone binaries' section (what it is, release
assets location, no-Python/no-GDAL claim, C interp fallback to
bit-identical np.interp).

Goal-1 proof (this host, v0.5.0, pyinstaller 6.22.3, python 3.13.12):
- dist/ppgrid --version -> 'ppgrid 0.5.0', exit 0; --help exit 0
- clean-env e2e (env -i, fresh HOME, PATH without cc/gcc/clang):
  exit 0, rasters written, no ~/.cache (tier-1 .so, no JIT)
- anchor gate: all four shas byte-exact vs examples/melb
  10m/value.tif        74d76b9b3a01dbc2ce76bf11c8fcd67d2c8b8b08ad1720b17983c28e00a5db43
  10m/support_km.tif   3d0822e1f15b545e359e51f748686d0500053f09b80e646ef7bf22b636a6c47d
  500m/value.tif       4a30bf476a85c9d30bb46aa84aa3aa140c532af0d03fda1dc90a72bfda3d45a7
  500m/support_km.tif  530cc25f3d4c5ebe2ab17359baeac56c635bc2f0096a14d70d7e790efaad9346
- ppgrid-0.5.0-linux-x86_64.xz: 179993860 bytes, sha256
  3840181c350516d8066e705e69c0edfa68e4ecdf23b191fbe82acdba392976a,
  attached to the v0.5.0 release and published at
  https://drop.junkyard.sh/ppgrid-0.5.0-linux-x86_64.xz
@marzukia
marzukia merged commit 89f358f into main Oct 16, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants