Build-system DX: preset restructure + tiered dependency acquisition (+ Windows dev path) - #98
Open
painfulexistence wants to merge 17 commits into
Open
Build-system DX: preset restructure + tiered dependency acquisition (+ Windows dev path)#98painfulexistence wants to merge 17 commits into
painfulexistence wants to merge 17 commits into
Conversation
First slice of proper Windows development support. Today a Windows dev
hits a wall: the documented `cmake --preset=dev` lives in
CMakeUserPresets.json and uses ccache + GCC -g/-O3 flags that cl.exe
rejects, and there's no MSVC-aware preset. This adds committed,
Windows-gated presets that use the bundled vcpkg toolchain:
- windows-vs : Visual Studio 2022 generator (x64) — open build/*.sln
and F5 to build + debug
- windows-ninja : Ninja Multi-Config + MSVC — fast CLI / VS Code builds
Both export compile_commands.json, plus matching buildPresets. A
.vscode/launch.json debugs the selected CMake Tools launch target
(cppvsdbg on Windows) with cwd at the target dir so the build-copied
assets/ resolve via SDL_GetBasePath.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
The project's documented workflow (README: cmake --preset=dev) lived in CMakeUserPresets.json — the file CMake convention reserves for machine-local personal overrides (and recommends gitignoring) — while the shared CMakePresets.json held only plumbing. Committing the user file also meant nobody had a place for personal config without dirtying a tracked file, and the "personal" bits it carried (ccache launcher, GCC-only -g/-O3 flags) are exactly what made the documented preset unusable with MSVC. Now: - dev/release (configure + build) move into the shared CMakePresets.json, made portable: no ccache, no compiler-specific flag overrides — each compiler keeps its default Debug/Release flags (the dropped -g and "-O3 -DNDEBUG" were redundant restatements of GCC/Clang defaults, and broke cl.exe). The ASan-on-macOS history note is preserved in the dev description. CMAKE_BUILD_TYPE stays for configure-time gates even though the multi-config generator ignores it at compile time. - ccache becomes an opt-in hidden mixin; enable it from a personal CMakeUserPresets.json via inherits: ["dev", "ccache"]. - windows-ninja is folded away: a portable dev already IS Ninja+MSVC on Windows (run from a VS x64 prompt). windows-vs stays for the .sln workflow, with host conditions now on its build presets too. - CMakeUserPresets.json is deleted from tracking and gitignored, per CMake convention. Migration for existing checkouts: if your local copy survives the pull (local edits), delete or rename its presets — a user preset may not reuse the shared names. Validated with cmake --list-presets (configure + build) on CMake 3.28. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
Deleting the tracked user presets traded away clone-and-go convenience. Bring it back the conventional way: a committed .example file with the recommended personal setup (my-dev / my-release = shared dev/release + the ccache mixin). Copy it to CMakeUserPresets.json (still gitignored) and adjust freely; the .gitignore comment now points at the cp command. Verified end-to-end: with the example copied into place, cmake --list-presets resolves my-dev/my-release configure + build presets alongside the shared ones with no name collisions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
Layering was mixed: dev/release (workflow names) sat beside windows-vs
(a generator) in one flat list, with near-duplicate ninja-*-vcpkg
plumbing alongside. Now the shape is explicit — hidden vcpkg toolchain
bases (bundled / $VCPKG_ROOT, both exporting compile_commands) grow two
generator families, and build presets pick the config:
ninja / ninja-global : Ninja Multi-Config on every desktop platform
(MSVC on Windows from a VS x64 prompt)
→ build presets debug / release
vs : Visual Studio 2022 (x64), Windows-gated, own
build-vs/ tree (CMake refuses to share a build
dir across generators) → vs-debug / vs-release
Configure presets no longer set CMAKE_BUILD_TYPE: every configure-time
read of it in the tree sits inside if(EMSCRIPTEN) blocks (verified),
and wasm builds go through buildWasm.sh, not these presets. That makes
the ninja family genuinely multi-config — configure once, build Debug
and Release from the same tree.
The dev name is freed for personal presets: CMakeUserPresets.json.example
now defines dev (ninja + ccache) with dev/dev-release build presets.
README documents the new flow (cmake --preset ninja; cmake --build
--preset debug) and run_code_check.py's hint follows the rename.
Validated with cmake --list-presets (configure + build), with and
without the example copied into place.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
The bundled vcpkg submodule is the project's one supported toolchain path — CI, the Android Gradle projects, and the iOS build all use it — so a $VCPKG_ROOT variant is a machine choice, not a project workflow. By the shared-vs-user split this repo now follows, it moves to the personal side: CMakeUserPresets.json.example gains dev-global, which overrides the toolchain via its own cacheVariables rather than an inherits mixin — deliberate, because the EARLIER entry wins in an inherits list, so a toolchain mixin listed after ninja would be silently ignored. run_code_check.py's hint now points at the ninja preset. Shared presets are down to: vcpkg base, ccache mixin, the ninja and vs families, and four build presets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
Dependency acquisition was scattered through a 680-line Engine/CMakeLists
across three mechanisms with no stated policy. Now Engine/cmake/
dependencies.cmake holds every external in three explicit tiers, and each
departure from the default carries its reason in-file:
Tier 1 — vcpkg manifest: the default (glm, Tracy, Bullet, Box2D, fmt,
spdlog, RmlUi, flatbuffers, tinyexr, glad, and the networking
feature's curl + libwebsockets).
Tier 2 — FetchContent, each entry justified: SDL3 (Android Gradle copies
the Java scaffolding out of the source tree, which the vcpkg
port doesn't install; tag pinned to the vendored imgui backend),
Lua (per-platform source build: LUA_USE_POSIX/LUA_USE_IOS, wasm
cross-compile, pinned), sol2 (must resolve against our
lua_static, not vcpkg lua), FastNoiseLite + TinyUSDZ (no vcpkg
port exists), Basis Universal (transcoder-only with wasm
size-trimming defines + host-arch encoder while cross-compiling).
Tier 3 — vendored Engine/external/: raudio (no port; iOS ObjC compile
patch), imgui (compiled into the engine as sources, upgraded by
hand with the SDL3 pin), stb headers.
FFmpeg is called out as the deliberate exception: optional system dep via
pkg-config. Feature wiring (networking sources, TinyUSDZ link/defines,
FFmpeg) stays in Engine/CMakeLists.txt next to its feature blocks.
The root CMakeLists' Android per-ABI triplet synthesis moves to
cmake/AndroidVcpkgTriplet.cmake, included at the same pre-toolchain spot.
The dead `if(!MSVC)` (never-true condition; the -Werror flag never
applied) is annotated as a FIXME rather than fixed — fixing it changes
behavior (Clang-only flag; GCC would reject it) and belongs to its own
change.
Mechanical move, zero behavior change, machine-verified: blocks sliced
verbatim by content-anchored script with command-count parity (16
find_package, 6 FetchContent_Declare, 19 target_link_libraries, ... all
equal before/after), and a configure with -DVCPKG_MANIFEST_INSTALL=OFF
parses the full new include chain, failing only at tier 1's first
find_package exactly as expected without installed deps.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
…tage All 15 build jobs on every platform failed identically inside the vcpkg manifest install at project(): gitlab.freedesktop.org returns 502/504 for the freetype source tarball (freetype → RmlUi, needed on every triplet). Nothing in this branch is involved — the failure precedes any of our CMake being read, and PR #97 (no build-system changes) fails the same way. Empty commit to re-run once upstream recovers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
painfulexistence
marked this pull request as ready for review
August 11, 2026 07:59
…dependency The vcpkg tinygltf port downloads a GitHub release .tar.gz verified by a pinned SHA512. GitHub regenerates those archives lazily, so the bytes (and hash) can drift under a fixed tag — which just broke every cold build at the download step. Upstream vcpkg moved the port to 3.0.0 in March and only maintains hashes there, so the 2.9.7 pin would stay broken. A git clone is content-addressed and immune, and matches how the same author's TinyUSDZ is already acquired here. - tinygltf v2.9.7 via FetchContent (same version the port carried; pure acquisition change, no API movement), consumed as an include dir like FastNoiseLite - nlohmann-json promoted from a transitive install of the tinygltf port to an explicit vcpkg dependency + linked target (public headers use it) - both tinygltf TUs now set TINYGLTF_NO_INCLUDE_JSON and include the same nlohmann as the scene layer — the port used to patch this include; the fetched source is unpatched - drop dead Engine/external/nlohmann/json.hpp (never on any include path) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
Keep the tiered organization and per-dependency justifications, but as sections inside the CMakeLists that own them instead of separate files — one file per directory reads top-to-bottom (repo convention): - Engine/cmake/dependencies.cmake -> a "Third-party dependencies" section in Engine/CMakeLists.txt at the old include site (tier banners intact) - cmake/AndroidVcpkgTriplet.cmake -> the if(ANDROID) triplet-synthesis block back in the root CMakeLists.txt, still between project() and the vcpkg toolchain include (order matters) Verbatim move, machine-checked: content-anchored splice with command-count parity across the file set (find_package/FetchContent_*/add_library/...), moved bodies asserted contiguous, then a configure with -DVCPKG_MANIFEST_INSTALL=OFF parses the full chain and stops at tier 1's first find_package(glm) exactly as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
Drop the rationale prose added in this PR from CMakeLists (tier banners, per-dependency justifications), preset descriptions, launch.json, and the tinygltf TU notes — keep only short section markers and operational one-liners. The reasoning lives in the PR description instead. Comment-only change, machine-checked: with comment lines stripped, every code file is byte-identical before/after; preset/launch JSON is structurally equal ignoring description keys. Presets listing and the manifest-off configure parse behave identically. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
Every CI job cached ${{ github.workspace }}/vcpkg/installed and
vcpkg/packages, but this repo builds in manifest mode, where the
toolchain installs into ${CMAKE_BINARY_DIR}/vcpkg_installed instead
(vcpkg.cmake:421-422 — vcpkg/installed is the classic-mode path). So the
cache saved working state nothing reads back, and every run rebuilt every
port from source.
Switch all seven cache steps to vcpkg's binary cache — prebuilt package
archives keyed by an internal ABI hash, which is the mechanism designed
for reuse across runs. VCPKG_DEFAULT_BINARY_CACHE redirects it into the
workspace so one path works on every runner, instead of the per-OS
defaults (~/.cache/vcpkg/archives vs %LOCALAPPDATA%\vcpkg\archives).
The key namespace moves to vcpkg-bin-* on purpose: reusing the old keys
would let restore-keys extract a stale entry back into vcpkg/installed
and, worse, make the save a no-op against an existing key — the fix
would silently never take effect.
Because the archives are ABI-hashed, leaving the keys coarse is safe: CI
checks out microsoft/vcpkg unpinned, and a port that moved upstream just
misses its archive and rebuilds rather than being wrongly reused.
Covers ci-desktop, e2e, ci-android, ci-ios, ci-web (both jobs), release.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0174mFerMztkv72nqk6Msc9A
Every workflow checked out microsoft/vcpkg master (no ref) instead of the repo's pinned submodule, so CI validated a dependency set that drifts from what developers build against — tinygltf broke on dev machines while CI stayed green because CI was on a different port version entirely. - checkout with submodules: true (vcpkg is the only submodule) and drop the separate master checkout; sparse checkouts add the vcpkg path - release.yml's release-web sparse list also gains cmake/ and triplets/, which the root CMakeLists has required since the helpers refactor Builds on the binary-cache fix that landed just before this: coarse vcpkg-bin-* keys stay correct (archives are ABI-hashed), and with the baseline pinned the cache now stays warm instead of chasing master. First run per platform rebuilds all ports cold; later runs restore them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
The presets shipped in this PR have no CI coverage — every job invokes cmake directly with an explicit toolchain file, so a broken preset would land green. Add a job that runs `cmake --preset vs` + `cmake --build --preset vs-release` on windows-latest, deliberately without ilammy/msvc-dev-cmd: the `vs` preset exists so the .sln path works from a plain shell, and skipping the developer-prompt setup is what puts that claim under test. Scoped to --target AtmosphericEngine. The matrix job already compiles all of Windows with Ninja; what's unproven here is preset resolution and generator wiring, so an MSBuild pass over every example would double the job's cost for coverage that already exists. Shares the matrix job's binary-cache key — same runner OS, same triplet. README: the bootstrap step was Unix-only, and the `ninja` preset needs a developer prompt on Windows, which the quickstart didn't say. Note the .bat and point at `vs` / `vs-debug` for a plain shell. docs/documentation/getting-started.html is left alone on purpose. Its preset names look stale against this repo, but the page walks the reader through cloning Atmospheric-Starter and building *that* — and the starter does define `desktop` and `wasm` presets. The page is correct as written. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0174mFerMztkv72nqk6Msc9A
…kflows CodeQL flagged the workflows without a permissions block (default token is broad). ci-android/ci-ios/e2e already declare contents: read at top level; bring ci-desktop, ci-web, and release in line. release's own jobs keep their explicit contents: write blocks, which override the top level. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
The preset hardcoded the "Visual Studio 17 2022" generator, which errors out on machines that only have a newer VS — windows-latest runners now ship VS 2026, so the new presets-windows CI job failed at project() with "could not find any instance of Visual Studio" right after all 23 vcpkg ports built cleanly. With no generator field, CMake's Windows default is exactly the wanted semantic: the newest installed Visual Studio (2022 on 2022 machines, 2026 on 2026 machines); architecture stays pinned to x64. README/preset text drops the version-specific developer-prompt naming. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013ZCpvozfFySsQahuUxg8m4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Developer-experience overhaul of the build system: CMake presets (including the first-class Windows dev path), third-party dependency acquisition, a tinygltf supply-chain fix, and CI made to validate the same dependency universe developers build against. Distribution (WIN32 subsystem / static triplet / packaging) stays out of scope.
Part 1 — Presets: vcpkg base → generator families
Problems: no MSVC path (the documented
devpreset used ccache + GCC-only flagscl.exerejects); the canonical workflow lived inCMakeUserPresets.json(the machine-local file CMake says to gitignore) while the shared file held only plumbing; workflow and generator names were mixed in one flat list.Final shape — one hidden
vcpkgtoolchain base (bundled submodule — the single supported path) + an opt-inccachemixin grow two generator families; build presets pick the config:ninjadebug/releasevs(Windows-gated)vs-debug/vs-release.slninbuild-vs/; locates MSVC itself; separate tree (CMake won't share a build dir across generators)CMAKE_BUILD_TYPE— every configure-time read sits inif(EMSCRIPTEN)blocks (verified) — so the ninja family is genuinely multi-config: configure once, build Debug and Release.devis now a personal name:CMakeUserPresets.json.exampleshipsdev(=ninja+ccache),dev-release, anddev-global(system vcpkg via$VCPKG_ROOT, moved out of the shared file since it's a machine choice; implemented as acacheVariablesoverride because an inherits-mixin would be silently beaten by precedence).cmake --preset ninja+cmake --build --preset debug;.vscode/launch.jsondebugs the CMake Tools launch target (cppvsdbgon Windows).Part 2 — Dependency acquisition: tiered, justified, inline
Acquisition was scattered through a 680-line
Engine/CMakeLists.txtacross three mechanisms with no stated policy. It now lives as a single "Dependencies" section insideEngine/CMakeLists.txt(no extra files — repo convention is one CMakeLists per directory, read top-to-bottom), grouped by mechanism, with rationale kept in this PR description rather than in-file comments:LUA_USE_POSIX/LUA_USE_IOS, wasm cross-compile, pinned) · sol2 (must bind to ourlua_static, not vcpkg lua) · FastNoiseLite + TinyUSDZ (no vcpkg port exists) · tinygltf (supply-chain — see Part 3) · Basis Universal (transcoder-only with wasm size-trimming defines + host-arch encoder while cross-compiling).Engine/external/: raudio (no port; iOS ObjC compile patch), imgui (compiled into the engine as sources, upgraded by hand with the SDL3 pin), stb.The root's Android per-ABI triplet synthesis stays in the root
CMakeLists.txt(betweenproject()and the vcpkg toolchain include — order matters). The deadif(!MSVC)(never-true condition — that-Werrorflag has never applied) carries a one-line FIXME; fixing it is a behavior change (Clang-only flag) for a separate PR.Part 3 — tinygltf: vcpkg port → FetchContent (hash-drift fix)
Cold builds (first hit on Windows) started failing at the vcpkg download step: the tinygltf 2.9.7 GitHub release
.tar.gzno longer matches the SHA512 pinned in the portfile. GitHub regenerates those archives lazily, so the bytes — and hash — can drift under a fixed tag; upstream vcpkg moved the port to 3.0.0 in March 2026 and only maintains hashes there, so the 2.9.7 pin stays broken forever. A git clone is content-addressed and immune.json_deserializer.hpp,scene_blueprint.hpp) depends on it directly. Now declared, found, and linkedPUBLIC(it appears in public headers).TINYGLTF_NO_INCLUDE_JSONand include the same nlohmann as the scene code (the vcpkg port used to patch that include; the fetched source is unpatched), so one json type flows through the whole build.Engine/external/nlohmann/json.hpp(24.7k lines; never on any include path).Part 4 — CI: pinned vcpkg + caches that actually hit
Two related fixes, landed as two commits:
62e3f0f): every job cachedvcpkg/installed+vcpkg/packages, which manifest mode never reads back (installs go tobuild/vcpkg_installed) — so every run rebuilt every port. All seven cache steps now carry vcpkg's ABI-hashed binary cache (VCPKG_DEFAULT_BINARY_CACHE→ workspace.vcpkg-cache), under a freshvcpkg-bin-*key namespace so stale entries never restore.9a010f5): every workflow checked outmicrosoft/vcpkgmaster rather than the repo's pinned submodule, so CI validated a drifting dependency set — this is why tinygltf broke on dev machines while CI stayed green (CI was on the 3.0.0 port entirely). Checkouts now usesubmodules: true; sparse lists gain thevcpkgpath;release.yml's sparse list also gainscmake/+triplets/, which the root CMakeLists has required since the helpers refactor.Net effect: CI green now means "builds against the pinned baseline developers actually get", and the second run per platform restores prebuilt ports instead of compiling them. The first run per platform is one-time cold (~all ports from source).
Verification
cmake --list-presets(configure+build) on CMake 3.28, with and without the example copied in; host conditions verified.-DVCPKG_MANIFEST_INSTALL=OFFparses the full chain and fails only at the firstfind_package(glm), exactly as expected without installed deps.v2.9.7succeeds; fetchedtiny_gltf.his byte-identical to the release header), and both TUs' define/include sequences were compiled with-fsyntax-onlyagainst that header + nlohmann 3.11.3.microsoft/vcpkgcheckout remains anywhere.cmake --preset ninjaitself yet (the follow-up below).Follow-ups (open for steer)
if(!MSVC)dead condition properly (Clang-gated warning flag) — separate small PR.docs/documentation/getting-started.htmlpreset names (desktop↔ninja)..batvs.sh); flip "Windows (planned)" once CI proves it..vscodetasks/settings/extensions + C++20 fix toc_cpp_properties.json(held pending your call).Separate from PR #97 (Android publishing); branched off
main.🤖 Generated with Claude Code