Skip to content

chore(config): gen-au-build + ctld-tools.exe + MM doc (CTLD-TOOLS-FINALIZE)#48

Merged
davidp57 merged 2 commits into
developfrom
feature/ctld-tools-finalize
Jul 20, 2026
Merged

chore(config): gen-au-build + ctld-tools.exe + MM doc (CTLD-TOOLS-FINALIZE)#48
davidp57 merged 2 commits into
developfrom
feature/ctld-tools-finalize

Conversation

@davidp57

@davidp57 davidp57 commented Jul 20, 2026

Copy link
Copy Markdown
Member

Finalizes the ctld-tools program (per maintainer feedback after lots 1–3).

What

  • gen-au-buildmerge_CTLD.ps1 regenerates src/CTLD_config_defaults.lua from
    CTLD_config.yaml via ctld-tools gen-config on every build. The file is now a git-ignored
    artifact
    (untracked). ci.yml (build + busted) and release.yml gain setup-python +
    poetry; busted generates the defaults before the suite; a conftest.py fixture does so for the
    Python tests. Drift check dropped — dev workflow is just "edit YAML, rebuild".
  • ctld-tools.exe — built (PyInstaller, lupa + datamine bundled) and attached to each Release by
    a separate build-exe job (needs: release), isolated so a packaging failure never blocks the
    CTLD.lua release. Verified locally: validate runs from the packaged exe (lupa + embedded
    dcs_types.json work).
  • Docs — dedicated mission-maker/ctld-tools.md (EN + FR): full user-config.yaml format,
    commands, examples (block + flow), ME loading. Linked in the nav. Developer build docs + ADR 0009
    updated to describe gen-au-build.

Validation

  • Python quality gate green (ruff + mypy + pytest 25) with defaults.lua deleted (conftest regens).
  • merge_CTLD.ps1 tested locally end to end (regenerate → merge → parse OK).

Notes

  • The build now requires Python + poetry (accepted trade-off for the simpler dev workflow).
  • Closes the ctld-tools program. Roadmap: .miz injection, TUI.

Summary by Sourcery

Finalize ctld-tools integration by generating engine defaults at build time, packaging and attaching ctld-tools.exe to releases, and adding dedicated mission-maker documentation.

New Features:

  • Distribute ctld-tools as a packaged ctld-tools.exe attached to each GitHub release for mission makers.
  • Provide dedicated English and French mission-maker documentation describing ctld-tools usage and user-config.yaml workflow.

Build:

  • Make merge_CTLD.ps1 regenerate CTLD_config_defaults.lua from CTLD_config.yaml on every build, treating the Lua as a git-ignored build artifact and requiring Python + poetry.
  • Update build and release workflows to install Python and ctld-tools via poetry so engine defaults are always generated during CTLD.lua builds.

CI:

  • Adjust CI build and busted jobs to install Python/poetry and generate the engine defaults before running tests.
  • Introduce a separate Windows build-exe job in the release workflow that builds ctld-tools.exe and uploads it without blocking the main CTLD.lua release.
  • Advance the published-latest tag only from stable releases in the release workflow.

Documentation:

  • Update developer build docs and ADR 0009 to describe the gen-au-build approach and removal of the drift check.
  • Add mission-maker ctld-tools documentation pages in English and French and link them into the documentation navigation.

Tests:

  • Add a pytest session fixture to regenerate CTLD_config_defaults.lua for tests and remove the drift test that asserted the committed Lua matched a fresh generation.

Chores:

  • Record the CTLD-TOOLS-FINALIZE work item and its scope in the backlog and changelog, documenting the closure of the ctld-tools programme.

…ge (CTLD-TOOLS-FINALIZE)

Finalizes the ctld-tools program:
- gen-au-build: merge_CTLD.ps1 regenerates src/CTLD_config_defaults.lua from the YAML
  via ctld-tools on every build; the file is now a git-ignored artifact (untracked).
  ci.yml (build + busted) and release.yml gain setup-python + poetry; busted generates
  the defaults before the suite; a conftest fixture does so for the Python tests. Drift
  check dropped.
- ctld-tools.exe: built (PyInstaller, lupa + datamine bundled) and attached to each
  Release by a separate build-exe job, isolated so packaging never blocks CTLD.lua.
  Verified locally (validate runs from the packaged exe).
- Docs: dedicated mission-maker/ctld-tools.md (EN+FR) + nav; developer build docs and
  ADR 0009 updated to describe gen-au-build.
@sourcery-ai

sourcery-ai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Reviewer's Guide

Finalizes ctld-tools by making CTLD_config_defaults.lua a generated, git-ignored build artifact produced on every build, wiring Python/poetry setup into CI and release workflows, introducing a dedicated job that builds and uploads ctld-tools.exe independently of the main release, and adding mission-maker and developer documentation plus test fixtures to keep parity tests passing without a committed defaults file.

Sequence diagram for gen-au-build CTLD_config_defaults.lua regeneration

sequenceDiagram
    actor Developer
    participant Merge_CTLD_ps1
    participant CtldTools
    participant SrcDir

    Developer->>Merge_CTLD_ps1: run merge_CTLD.ps1
    Merge_CTLD_ps1->>CtldTools: poetry run ctld-tools gen-config --yaml CTLD_config.yaml --out CTLD_config_defaults.lua
    CtldTools->>SrcDir: write CTLD_config_defaults.lua
    CtldTools-->>Merge_CTLD_ps1: exit 0
    Merge_CTLD_ps1->>SrcDir: read CTLD_config_defaults.lua
    Merge_CTLD_ps1-->>Developer: output merged CTLD.lua build artifact
Loading

File-Level Changes

Change Details Files
Make CTLD_config_defaults.lua a generated, git-ignored build artifact produced via ctld-tools on every build and adjust CI, release, and tests around this workflow.
  • Update merge_CTLD.ps1 to call poetry run ctld-tools gen-config to regenerate CTLD_config_defaults.lua from CTLD_config.yaml and fail fast with a clear message if poetry/Python are missing.
  • Add Python 3.13 setup and poetry installation steps to CI build and busted jobs, plus release workflow, to support gen-config during builds and tests.
  • Generate CTLD_config_defaults.lua in the busted job prior to running the Lua test suite, and add a pytest session fixture that regenerates it for Python tests.
  • Remove CTLD_config_defaults.lua from version control and ignore it in .gitignore, and drop the drift-check test that compared committed Lua to fresh gen-config output.
  • Update ADR 0009, developer build docs (EN/FR), and backlog PRD to describe the new gen-au-build approach and removal of the drift check.
tools/build/merge_CTLD.ps1
.github/workflows/ci.yml
.github/workflows/release.yml
tools/ctld-tools/tests/conftest.py
tools/ctld-tools/tests/test_parity.py
docs/developer/building-and-testing.md
docs/developer/building-and-testing.fr.md
dev/adr/0009-external-yaml-authoring-ctld-tools.md
.backlog/README.md
.backlog/CTLD-TOOLS-FINALIZE/PRD.md
.gitignore
src/CTLD_config_defaults.lua
Introduce a dedicated build-exe job to build ctld-tools.exe with PyInstaller and upload it to releases, decoupled from the main CTLD.lua release job.
  • Split the ctld-tools.exe packaging into a separate build-exe job that depends on the main release job, runs on windows-latest, and has contents: write permissions.
  • Ensure build-exe checks out the repo, sets up Python 3.13, runs poetry-based installation, then builds ctld-tools.exe with PyInstaller including lupa and embedded dcs_types.json.
  • Upload the generated ctld-tools.exe to the GitHub release using gh release upload, removing previous continue-on-error wiring and outcome-based conditions from the main job.
  • Relocate the published-latest tag advancement step into the main release job, clarifying its behavior around stable vs pre-release tags.
.github/workflows/release.yml
Add dedicated mission-maker documentation for ctld-tools (EN/FR) and link it into the site navigation, reflecting the YAML-based user configuration workflow.
  • Create bilingual mission-maker/ctld-tools.md and mission-maker/ctld-tools.fr.md pages that describe ctld-tools usage, user-config.yaml schema, validation and gen-user commands, and Mission Editor loading order.
  • Add the new ctld-tools mission-maker pages to mkdocs.yml navigation under the Mission Maker section.
  • Update the changelog entry to document ctld-tools finalize features, including gen-au-build, exe distribution, and the new documentation.
  • Document the CTLD-TOOLS-FINALIZE lot in the backlog with a PRD describing problem statement, solution, implementation, and testing notes.
docs/mission-maker/ctld-tools.md
docs/mission-maker/ctld-tools.fr.md
mkdocs.yml
CHANGELOG.md
.backlog/CTLD-TOOLS-FINALIZE/PRD.md
.backlog/README.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues, and left some high level feedback:

  • The conftest.py session fixture writes src/CTLD_config_defaults.lua into the repo on every test run, which will leave local working trees dirty; consider generating into a temporary path and pointing the tests at that instead to keep the tree immutable.
  • All workflows are pinned to Python 3.13, which is relatively new and may be unstable on hosted runners; you might want to target a better-supported minor (e.g. 3.11/3.12) unless you rely on 3.13-specific features.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `conftest.py` session fixture writes `src/CTLD_config_defaults.lua` into the repo on every test run, which will leave local working trees dirty; consider generating into a temporary path and pointing the tests at that instead to keep the tree immutable.
- All workflows are pinned to Python 3.13, which is relatively new and may be unstable on hosted runners; you might want to target a better-supported minor (e.g. 3.11/3.12) unless you rely on 3.13-specific features.

## Individual Comments

### Comment 1
<location path=".github/workflows/release.yml" line_range="22-27" />
<code_context>
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.13"
+      - name: Install ctld-tools (the build regenerates CTLD_config_defaults.lua via gen-config)
+        working-directory: tools/ctld-tools
+        shell: bash
+        run: |
+          pip install poetry
+          poetry install --without dev --without build
       - name: Build CTLD.lua
         run: powershell -ExecutionPolicy Bypass -File tools\build\merge_CTLD.ps1
</code_context>
<issue_to_address>
**suggestion (performance):** Consider caching Poetry and the virtualenv to avoid re-installing dependencies on every workflow run.

Given `ctld-tools` likely has non-trivial dependencies, this will add avoidable time and PyPI traffic on every run. You can cache the Poetry environment (e.g. `.venv` or `~/.cache/pypoetry`) with `actions/cache`, and reuse the same pattern in CI so builds stay faster and more consistent across workflows.
</issue_to_address>

### Comment 2
<location path="tools/build/merge_CTLD.ps1" line_range="23-32" />
<code_context>
+$ctldToolsDir = Join-Path $repoRoot "tools\ctld-tools"
+$configYaml   = Join-Path $srcDir   "CTLD_config.yaml"
+$defaultsLua  = Join-Path $srcDir   "CTLD_config_defaults.lua"
+Write-Host "Generating CTLD_config_defaults.lua from CTLD_config.yaml (ctld-tools)..."
+Push-Location $ctldToolsDir
+try {
+    & poetry run ctld-tools gen-config --yaml $configYaml --out $defaultsLua
+    if ($LASTEXITCODE -ne 0) { throw "gen-config returned $LASTEXITCODE" }
+}
+catch {
+    Write-Host "[ERROR] Could not generate CTLD_config_defaults.lua via ctld-tools."
+    Write-Host "        Run 'poetry install' in tools/ctld-tools first (needs Python + poetry)."
+    Pop-Location
+    exit 1
+}
</code_context>
<issue_to_address>
**suggestion:** Use a `try { ... } finally { Pop-Location }` structure to guarantee location restoration.

Currently `Pop-Location` is called in the `catch` and again after the `try/catch`, which only works because the `catch` path exits the script. That coupling to control flow is fragile. Wrapping this in `try { ... } finally { Pop-Location }` ensures the location is always restored, even if additional error branches or exits are introduced later.

Suggested implementation:

```
Write-Host "Generating CTLD_config_defaults.lua from CTLD_config.yaml (ctld-tools)..."
Push-Location $ctldToolsDir
try {
    & poetry run ctld-tools gen-config --yaml $configYaml --out $defaultsLua
    if ($LASTEXITCODE -ne 0) { throw "gen-config returned $LASTEXITCODE" }
}
catch {
    Write-Host "[ERROR] Could not generate CTLD_config_defaults.lua via ctld-tools."
    Write-Host "        Run 'poetry install' in tools/ctld-tools first (needs Python + poetry)."
    exit 1
}
finally {
    Pop-Location
}

```

None required beyond this edit; the try/catch/finally block now guarantees `Pop-Location` is always executed, independent of control flow, and removes the previous coupling between error handling and location restoration.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +22 to +27
- name: Install ctld-tools (the build regenerates CTLD_config_defaults.lua via gen-config)
working-directory: tools/ctld-tools
shell: bash
run: |
pip install poetry
poetry install --without dev --without build

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (performance): Consider caching Poetry and the virtualenv to avoid re-installing dependencies on every workflow run.

Given ctld-tools likely has non-trivial dependencies, this will add avoidable time and PyPI traffic on every run. You can cache the Poetry environment (e.g. .venv or ~/.cache/pypoetry) with actions/cache, and reuse the same pattern in CI so builds stay faster and more consistent across workflows.

Comment on lines +23 to +32
Write-Host "Generating CTLD_config_defaults.lua from CTLD_config.yaml (ctld-tools)..."
Push-Location $ctldToolsDir
try {
& poetry run ctld-tools gen-config --yaml $configYaml --out $defaultsLua
if ($LASTEXITCODE -ne 0) { throw "gen-config returned $LASTEXITCODE" }
}
catch {
Write-Host "[ERROR] Could not generate CTLD_config_defaults.lua via ctld-tools."
Write-Host " Run 'poetry install' in tools/ctld-tools first (needs Python + poetry)."
Pop-Location

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Use a try { ... } finally { Pop-Location } structure to guarantee location restoration.

Currently Pop-Location is called in the catch and again after the try/catch, which only works because the catch path exits the script. That coupling to control flow is fragile. Wrapping this in try { ... } finally { Pop-Location } ensures the location is always restored, even if additional error branches or exits are introduced later.

Suggested implementation:

Write-Host "Generating CTLD_config_defaults.lua from CTLD_config.yaml (ctld-tools)..."
Push-Location $ctldToolsDir
try {
    & poetry run ctld-tools gen-config --yaml $configYaml --out $defaultsLua
    if ($LASTEXITCODE -ne 0) { throw "gen-config returned $LASTEXITCODE" }
}
catch {
    Write-Host "[ERROR] Could not generate CTLD_config_defaults.lua via ctld-tools."
    Write-Host "        Run 'poetry install' in tools/ctld-tools first (needs Python + poetry)."
    exit 1
}
finally {
    Pop-Location
}

None required beyond this edit; the try/catch/finally block now guarantees Pop-Location is always executed, independent of control flow, and removes the previous coupling between error handling and location restoration.

@davidp57
davidp57 merged commit 6982658 into develop Jul 20, 2026
7 checks passed
@davidp57
davidp57 deleted the feature/ctld-tools-finalize branch July 20, 2026 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant