Skip to content

Add ROM-free cross-platform CI and Nightly releases - #2

Merged
Zection6V merged 44 commits into
developfrom
agent/build-nightly-workflows
Aug 17, 2026
Merged

Add ROM-free cross-platform CI and Nightly releases#2
Zection6V merged 44 commits into
developfrom
agent/build-nightly-workflows

Conversation

@Zection6V

@Zection6V Zection6V commented Aug 17, 2026

Copy link
Copy Markdown
Owner

Summary

Adds reproducible Windows/Linux build workflows and a fixed nightly-release prerelease publisher for develop, following the build/release safety model used by melonPrimeDS while keeping the public pipeline completely ROM-free.

ROM-free build model

  • GitHub Actions does not receive or fetch a Metroid Prime Hunters ROM.
  • No private ROM download URL or repository/environment secret is used by the build or Nightly workflows.
  • No proprietary retail BIOS/firmware dump, save, or generated MPH title-bank source is uploaded or generated from private game content in CI.
  • The pinned BSD-2-Clause FreeBIOS source is recompiled into the native FreeBIOS ARM9/ARM7 banks using ndsrecomp's documented pipeline.
  • Public builds use NDS_RETAIL_BIOS_BANKS=OFF; if a user elects to supply retail BIOS dumps, immutable BIOS code can use the existing reference-interpreter path rather than requiring proprietary generated BIOS C in the distributed binary.
  • No MPH title-bank directory is linked into the Nightly runner. Direct-booted MPH ARM9/ARM7 code therefore uses the existing Tier-3 fallback when no content-specific native bank is present.
  • The Windows launcher already uses --freebios --generated-firmware --boot direct when no retail dumps are selected, so the ROM-free ZIP follows the existing no-dump startup path.

This is a correctness-first Nightly path and may be slower than an optimized tagged build, especially in known FMV/runtime-code hot paths.

Multi-ROM launcher validation semantics

The launcher deliberately does not use whole-ROM SHA-1 as a compatibility gate. Runtime base/version selection remains authoritative in nds_runner through the executable-compatible seven-version MPH detector.

Pinned recomp-ui normally renders a cartridge with no configured CRC/SHA fingerprints as ROM not recognized, even though its play gate permits such a host-delegated ROM. The MPH integration now patches only that presentation path:

  • fingerprint-free selected ROM: ROM selected - runtime validation
  • generic Nightly Region label: Auto (runtime detected)
  • clean or modified ROMs are not rejected by whole-ROM SHA-1 in the launcher
  • the runner still performs the authoritative fail-closed runtime compatibility decision

This avoids reintroducing a clean-ROM SHA list that would incorrectly reject compatible modified ROMs.

Portable local optimization cache direction

Nightly packages reserve a portable-first cache root:

cache/banks/<content-sha1>/

Whole-ROM SHA-1 is used only as exact cache/content identity. Runtime base-profile selection remains the executable-compatible seven-version MPH detector and does not use whole-ROM SHA-1.

The current Nightly does not yet generate native title banks in this directory. docs/LOCAL_BANK_CACHE.md defines the intended progression from Tier-3 to a compiler-free local bank/IR layer and ultimately a validated hot-block JIT/persistent cache. The project intentionally avoids requiring a C/C++ compiler on the player's first launch.

CI

  • Windows: builds the pinned ndsrecomp runner with the MPH runtime patches, generates only redistributable FreeBIOS banks, patches the pinned recomp-ui delegated-validation presentation, builds/tests recomp-ui, creates the ROM-free ZIP, and rejects title-bank/private-content leakage.
  • Windows CI reuses the windows-2025 image's existing C:\msys64, installs only MinGW GCC/SDL2/ccache, uses setup-msys2's package cache, and persists compiler output with actions/cache + ccache.
  • Linux: performs the same ROM-free runner build and produces/scans an AppImage; compiler output is also persisted with actions/cache + ccache.
  • recomp-ui.pin pins launcher UI for reproducibility.
  • A source-policy job parses the new Python/Bash/PowerShell helpers and rejects reintroduction of repository/environment secret references into the public build/Nightly workflows.
  • Main Build concurrency uses cancel-in-progress: true, so stale PR builds are cancelled when a newer commit arrives.

Nightly release

  • Runs on pushes to develop and by manual dispatch.
  • Reuses the same Windows/Linux workflows as PR CI so Nightly and normal build steps cannot drift.
  • Publishes only after both platform builds and the payload verifier succeed.
  • Uses the fixed nightly-release prerelease tag.
  • Generates SHA256SUMS.txt.
  • Replaces same-named assets safely and removes stale assets only after new uploads succeed.

Release safety

  • Windows ZIP is inspected for required payload and forbidden ROM/save/BIOS/firmware/generated material.
  • Linux AppImage is extracted and scanned before publication.
  • The Windows package includes the portable cache/banks/README.txt contract; Linux resolves the sibling cache/banks directory and falls back to the XDG cache directory when the AppImage location is not writable.
  • Stable/tagged optimized release packaging remains separate and retains its existing native-bank quality requirements.

Validation

Current PR head: 35bc28c9ac32644cecd0dcfe310012e72a387639

Warm-cache measurements from repeated hosted-runner builds:

  • Linux: about 1m32 cold -> 35s warm
    • FreeBIOS recompiler/generation build stage: about 15s -> 2s
    • runner build: about 30s -> 2s
    • ccache: 129/129 cacheable calls hit (100%)
  • Windows: persistent ccache is active and warm runs show near-total compiler-cache reuse; MinGW setup was reduced by reusing the runner image MSYS2 installation and trimming the install set to GCC/SDL2/ccache.

The remaining dominant setup cost is package installation/unpacking (especially Windows MinGW and Linux SDL/OpenGL development packages), not C/C++ recompilation. Caching the entire installed toolchain prefix was intentionally avoided because it would be large, image-version-sensitive, and more fragile than the current package-cache + compiler-cache split.

The earlier private-ROM-URL approach was removed from this branch's PR history before this implementation was rebuilt; the current public build/Nightly path contains no MPH_US10_ROM_URL or secrets.* release dependency.

@Zection6V
Zection6V force-pushed the agent/build-nightly-workflows branch from 92f13d2 to fef5e8f Compare August 17, 2026 06:13
@Zection6V Zection6V changed the title Add cross-platform build CI and Nightly releases Add ROM-free cross-platform CI and Nightly releases Aug 17, 2026

Copy link
Copy Markdown
Owner Author

Multi-ROM startup fix + persistent diagnostics

Follow-up for the reported issue where opening a ROM other than USA v1.0 immediately exited.

Root cause found in the runner startup path: the seven-version executable/header detector ran first, but the legacy game.toml USA v1.0 whole-ROM SHA-1 gate was still applied immediately afterwards. A correctly detected US1.1/EU/JP/KR ROM could therefore be rejected with return 1 before reaching Tier-3 execution.

Changes at head cece5e32161b858fe40cf705ea439628d7925b31:

  • added tools/patch_ndsrecomp_mph_diagnostics.py to the MPH patch stack;
  • ROM-free builds now treat a successfully selected MPH runtime profile as the compatibility authority instead of re-applying the legacy USA v1.0 whole-ROM SHA gate;
  • optimized/native-bank builds retain the existing stricter exact-content reuse policy;
  • authoritative known executable checksums keep host Aim/Morph RAM access enabled;
  • header-fallback modified ROMs may enter Tier-3, but dangerous host RAM writes remain disabled;
  • interactive runner launches create/overwrite MetroidPrimeHuntersRecomp.log beside nds_runner;
  • log includes cartridge SHA, gameCode, revision, melonPrime-compatible executable CRC32, detector source (executable-checksum or header-fallback), selected runtime profile, variant name, host-write safety state, and explicit rejection reasons.

Validation:

  • MPH Multi-ROM Static Checks #121: SUCCESS (32009439592)
  • Build Expose Widescreen mode choices mstan/MetroidPrimeHuntersRecomp#34: SUCCESS (32009439683)
    • Windows ROM-free runner + launcher + ZIP: SUCCESS
    • Linux ROM-free runner + AppImage: SUCCESS
  • Windows artifact: mph-nightly-windows, artifact 9281244856, SHA-256 9d6c1e6954dd9c66cdf2f2bd936eeb499e4a32d1bdc1d4b61ebb9f55447324dc
  • Linux artifact: mph-nightly-linux, artifact 9281219781, SHA-256 edba3209b058444d2720b35cf3dde4cd6dd2624e1106ca5b7a9d937c969606d0

Actual retail/modded non-US ROM execution is not possible in public CI, so the next real-device/runtime check should use the new Windows artifact. If it still exits/crashes, MetroidPrimeHuntersRecomp.log should now identify the exact last successful stage and selected profile.

@Zection6V
Zection6V marked this pull request as ready for review August 17, 2026 08:30
@Zection6V
Zection6V merged commit ccd94bf into develop Aug 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant