Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .backlog/CTLD-TOOLS-FINALIZE/PRD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
Status: ready

# PRD — CTLD-TOOLS-FINALIZE

> Finalisation du programme `ctld-tools` (voir [ADR 0009](../../dev/adr/0009-external-yaml-authoring-ctld-tools.md)),
> demandée par le maintainer après les lots 1–3.

## Problem Statement

Three loose ends after `CTLD-TOOLS-USERCONFIG`:

1. The engine-config workflow used **committed generated Lua + a drift check**. Editing the config
meant a manual `gen-config` step; forgetting it produced a CI round-trip. The maintainer prefers
the build to regenerate the Lua itself.
2. `ctld-tools.exe` was wired into `release.yml` but **never actually built/tested**, and only as an
isolated best-effort step.
3. The Mission Maker had no **dedicated documentation page** for `ctld-tools` — only a tip.

## Solution

- **gen-au-build**: `merge_CTLD.ps1` regenerates `src/CTLD_config_defaults.lua` from
`CTLD_config.yaml` via `ctld-tools gen-config` on every build. The generated Lua is a git-ignored
artifact. Consumers that need it (build, busted loader, release) generate it; a `conftest.py`
fixture does so for the Python tests. The drift check is dropped.
- **`ctld-tools.exe`**: built and verified (PyInstaller with lupa + the datamine bundled), and
attached to each Release by a **separate `build-exe` job** so a packaging failure never blocks the
`CTLD.lua` release.
- **Docs**: a dedicated bilingual `mission-maker/ctld-tools.md` page, detailed with examples.

## Implementation Decisions

- `merge_CTLD.ps1` calls `poetry run ctld-tools gen-config` before merging; aborts with a clear
message if poetry is missing. The build (and the busted job) therefore require Python + poetry.
- `ci.yml` `build` (Windows) and `busted` (Ubuntu) jobs, and `release.yml`, add `setup-python` +
`poetry install --without dev --without build`; busted also runs `gen-config` before the suite.
- `src/CTLD_config_defaults.lua` removed from git tracking and added to `.gitignore`.
- `release.yml` gains a `build-exe` job (`needs: release`, `windows-latest`): PyInstaller `--onefile
--collect-all lupa --add-data dcs_types.json`, then `gh release upload`.
- ADR 0009 and the developer build docs updated to describe gen-au-build (superseding committed+drift).

## Testing Decisions

- A `conftest.py` session fixture regenerates `CTLD_config_defaults.lua` from the YAML before the
Python tests, so the parity/reference/e2e tests keep working without a committed file. The drift
test (`test_committed_generated_lua_is_up_to_date`) is removed.
- The `.exe` build was validated locally: `scaffold` + `validate` run from the packaged exe (proving
lupa and the embedded `dcs_types.json` work). The `build-exe` CI job re-verifies on each release.
- The `busted` job proves the loader still loads the freshly-generated defaults.

## Out of Scope

- `.miz` injection, TUI (roadmap).
- Any change to the `user-config.yaml` schema or the runtime API.

## Further Notes

- Closes the `ctld-tools` program. Remaining roadmap items: `.miz` injection, interactive TUI.
1 change: 1 addition & 0 deletions .backlog/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ authored **per lot, when the lot is started** (not in batch).
|-----|--------|-------------|--------|
| `FEAT-USERCONFIG-API` | ✅ merged (PR #45) | Replace broken Section 2 of `CTLD_userConfig.lua` with a safe MM API (`ctld.userSetup` callbacks + helpers), relocate `injectAACrates` to bootstrap, fix all parity bugs in userConfig template. | `feature/userconfig-api` |
| `CTLD-TOOLS-CONFIG` | ✅ merged (PR #46) | Lot 2 of `ctld-tools` ([ADR 0009](../dev/adr/0009-external-yaml-authoring-ctld-tools.md)): engine defaults moved out of `CTLD_config.lua` into `src/CTLD_config.yaml` (source of truth, sectioned mm-facing/advanced); isolated poetry package `tools/ctld-tools` (typer/ruamel/lupa, VMCT stack) with `extract` + `gen-config`; committed generated `CTLD_config_defaults.lua` + parity/drift guards + `python-quality` CI. `load()` copies `ctld.__configDefaults`. | `feature/ctld-tools-config` |
| `CTLD-TOOLS-FINALIZE` | ✅ merged (PR #48) | Finalize `ctld-tools`: **gen-au-build** (`merge_CTLD.ps1` regenerates `CTLD_config_defaults.lua` via `ctld-tools`, now a git-ignored artifact; CI/release gain setup-python; drift check dropped) + build & attach **`ctld-tools.exe`** (separate isolated job, verified) + dedicated MM doc page `ctld-tools.md` (EN+FR). | `feature/ctld-tools-finalize` |
| `CTLD-TOOLS-USERCONFIG` | ✅ merged (PR #47) | Lot 3 of `ctld-tools` ([ADR 0009](../dev/adr/0009-external-yaml-authoring-ctld-tools.md)): MM volet — `validate` (user-config.yaml against the reference + datamine, clear report with suggestions) + `gen-user` (compile add/remove/patch ops into `CTLD_userConfig.lua` calling the `ctld.userSetup` helpers, targeting crates/troops **by name**) + `gen-user --scaffold`; embedded `dcs_types.json`; `ctld-tools.exe` attached to Releases (isolated). e2e-tested. | `feature/ctld-tools-userconfig` |
| `DOC-README-CLEANUP` | ✅ merged (PR #21) | Fix README title, restructure Crate Operations sub-sections, relocate AA System Construction, replace Developer Guide with Documentation links. | — |
| `CATCH-UP-PILOT-SCENARIOS` | ✅ merged (PR #24) | Ran the never-executed `pilotPassive`/`pilotActive` scenarios. Audit found the "34 `ia`" was almost all mistagged: 66/66 now pass in a headless sweep (`--no-ai --reset-before-each`), real pilot burden ≈ 2 short menu flights (both PASS). Fixed a pile of test-harness defects + cross-scenario contamination (soft reset `_reset_state.lua`), + one product fix (`refreshMenuSection` flight-state override). Optional remainders: `auto-slow` AI battery (covered by F-176..182), mt16/warehouse, L6 manual (ticket 08). | — |
Expand Down
18 changes: 18 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,15 @@ jobs:
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install ctld-tools (the build regenerates CTLD_config_defaults.lua via gen-config)
working-directory: tools/ctld-tools
shell: bash
run: |
pip install poetry
poetry install --without dev --without build
- name: Build CTLD.lua
run: powershell -ExecutionPolicy Bypass -File tools\build\merge_CTLD.ps1
shell: pwsh
Expand Down Expand Up @@ -99,6 +108,15 @@ jobs:
COVERAGE_FLOOR: '59'
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Generate engine defaults (the busted loader dofile's CTLD_config_defaults.lua)
working-directory: tools/ctld-tools
run: |
pip install poetry
poetry install --without dev --without build
poetry run ctld-tools gen-config --yaml ../../src/CTLD_config.yaml --out ../../src/CTLD_config_defaults.lua
- name: Install Lua + LuaRocks + busted + luacov
run: |
sudo apt-get install -y lua5.1 liblua5.1-dev luarocks
Expand Down
58 changes: 35 additions & 23 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@ jobs:
steps:
- uses: actions/checkout@v7

- uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install ctld-tools (the build regenerates CTLD_config_defaults.lua via gen-config)
working-directory: tools/ctld-tools
shell: bash
run: |
pip install poetry
poetry install --without dev --without build
Comment on lines +22 to +27

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (performance): Consider caching Poetry and the virtualenv to avoid re-installing dependencies on every workflow run.

Given ctld-tools likely has non-trivial dependencies, this will add avoidable time and PyPI traffic on every run. You can cache the Poetry environment (e.g. .venv or ~/.cache/pypoetry) with actions/cache, and reuse the same pattern in CI so builds stay faster and more consistent across workflows.


- name: Build CTLD.lua
run: powershell -ExecutionPolicy Bypass -File tools\build\merge_CTLD.ps1
shell: pwsh
Expand Down Expand Up @@ -51,18 +61,34 @@ jobs:
--title "CTLD v${{ steps.ver.outputs.version }}" \
"${notes[@]}" "${pre[@]}"

# Build and attach the Mission Maker tool ctld-tools.exe. Isolated from the
# deliverable: continue-on-error so a packaging hiccup never blocks the release
# of CTLD.lua. Uploaded to the release that the previous step just created.
- name: Set up Python
uses: actions/setup-python@v6
# A floating "published-latest" tag tracks the last STABLE release. Only a real
# release advances it; a pre-release (rc) leaves production users on the previous
# stable. "published-latest" is not matched by the 'published-v*' trigger, so this
# push does not re-trigger the workflow.
- name: Advance published-latest (stable only)
if: steps.ver.outputs.prerelease != 'true'
run: |
git tag -f published-latest "${{ github.ref_name }}"
git push origin -f published-latest
shell: pwsh

# ─────────────────────────────────────────────────────────────
# Build the Mission Maker tool ctld-tools.exe (PyInstaller, lupa + datamine
# bundled) and attach it to the release. Separate job: if packaging fails, the
# CTLD.lua release (job above) is already published — only the exe is missing.
# ─────────────────────────────────────────────────────────────
build-exe:
name: Build & attach ctld-tools.exe
needs: release
runs-on: windows-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7
- uses: actions/setup-python@v6
with:
python-version: "3.13"
continue-on-error: true

- name: Build ctld-tools.exe
id: exe
continue-on-error: true
working-directory: tools/ctld-tools
shell: bash
run: |
Expand All @@ -72,22 +98,8 @@ jobs:
--collect-all lupa \
--add-data "ctld_tools/data/dcs_types.json;ctld_tools/data" \
ctld_tools/__main__.py

- name: Attach ctld-tools.exe to the release
if: steps.exe.outcome == 'success'
continue-on-error: true
shell: bash
env:
GH_TOKEN: ${{ github.token }}
run: gh release upload "${{ github.ref_name }}" tools/ctld-tools/dist/ctld-tools.exe

# A floating "published-latest" tag tracks the last STABLE release. Only a real
# release advances it; a pre-release (rc) leaves production users on the previous
# stable. "published-latest" is not matched by the 'published-v*' trigger, so this
# push does not re-trigger the workflow.
- name: Advance published-latest (stable only)
if: steps.ver.outputs.prerelease != 'true'
run: |
git tag -f published-latest "${{ github.ref_name }}"
git push origin -f published-latest
shell: pwsh
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,7 @@ tools/dcs-bridge/venv/

# ── Python bytecode cache (tools/ scripts are run directly, never packaged) ──
__pycache__/
*.pyc
*.pyc

# ── Generated engine defaults (gen-au-build via ctld-tools; regenerated by the build) ──
/src/CTLD_config_defaults.lua
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,18 @@ Versioning follows [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Tooling — ctld-tools finalize: gen-au-build, `.exe` distribution, MM docs (CTLD-TOOLS-FINALIZE)

- **gen-au-build**: `merge_CTLD.ps1` now regenerates `src/CTLD_config_defaults.lua` from
`CTLD_config.yaml` via `ctld-tools` on **every build** — it is a git-ignored artifact, no longer
committed. The `build` + `busted` CI jobs and `release.yml` gain `setup-python` + poetry; the drift
check is dropped (the file is always fresh). Dev workflow is now simply "edit the YAML, rebuild".
- **`ctld-tools.exe`** is built (PyInstaller, lupa + datamine bundled) and attached to each Release
by a **separate `build-exe` job**, isolated so a packaging issue never blocks the `CTLD.lua`
release. Verified end to end (the exe runs `validate` with the embedded Lua runtime + type set).
- **Docs**: dedicated `mission-maker/ctld-tools.md` page (EN + FR), with the full `user-config.yaml`
format, commands and examples (block + flow), linked in the site nav.

### Tooling — Mission Maker YAML authoring: `validate` + `gen-user` (CTLD-TOOLS-USERCONFIG)

- **`ctld-tools` gains the MM volet**: `validate` (checks a `user-config.yaml` against the reference
Expand Down
11 changes: 6 additions & 5 deletions dev/adr/0009-external-yaml-authoring-ctld-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,12 @@ Accepted (CTLD-TOOLS-CONFIG, 2026-07-20).
isolated **poetry** sub-project `tools/ctld-tools/`, **typer** CLI, **ruamel.yaml**, **lupa**
(runs `CTLD_config.lua` in-process to read the defaults; `ctld.tr` stubbed to identity),
pytest + ruff + mypy, workflow `python-quality.yml`. A **single** package `ctld_tools`.
- **Generated Lua is committed, not build-generated.** Following the VEAF pattern
(`dcsUnits.lua`, `dcs_types.lua`), `src/CTLD_config_defaults.lua` is committed and merged as-is;
the build stays pure PowerShell (no poetry in the Windows build). A **drift check** in the
quality gate (`gen-config` output == committed file) guards staleness. This supersedes the
"gen-config as a build step" wording.
- **Generated Lua is build-generated (gen-au-build), not committed.** `merge_CTLD.ps1` runs
`ctld-tools gen-config` before merging, so `src/CTLD_config_defaults.lua` is always fresh from the
YAML; it is a **git-ignored build artifact**. The build (and the busted CI job, whose loader loads
it) therefore require Python + poetry. This keeps the dev workflow to "edit the YAML, rebuild" with
no manual regen step. (An earlier iteration committed the file with a drift check; dropped in
favour of gen-au-build per maintainer preference.)
- **Merge order**: the generated defaults module evaluates `ctld.tr(...)` at load time, so it is
merged **after the `CTLD_i18n_*` modules** (not merely after `CTLD_config.lua`).
- **`CTLDConfig:load()`** copies `ctld.__configDefaults` into `self.settings`; the `TEMPLATES`
Expand Down
25 changes: 12 additions & 13 deletions docs/developer/building-and-testing.fr.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,22 +40,21 @@ la sortie existe et n'est pas vide, et téléverse `CTLD.lua` en tant qu'artefac
## Configuration moteur (`ctld-tools`)

Les valeurs par défaut du moteur sont des **données**, pas du code : elles vivent dans
`src/CTLD_config.yaml` (source de vérité unique, sectionnée `mm_facing` / `advanced`). Un module
généré `src/CTLD_config_defaults.lua` définit `ctld.__configDefaults`, que `CTLDConfig:load()`
recopie dans ses settings. **Éditez le YAML, jamais le Lua généré.** Les deux fichiers sont committés.
`src/CTLD_config.yaml` (source de vérité unique, sectionnée `mm_facing` / `advanced`). Au moment du
build, `merge_CTLD.ps1` régénère `src/CTLD_config_defaults.lua` depuis le YAML (via
`ctld-tools gen-config`) ; ce fichier définit `ctld.__configDefaults`, que `CTLDConfig:load()`
recopie dans ses settings. **Éditez le YAML** — le Lua généré est un **artefact de build (ignoré par
git)**, jamais édité à la main ni committé.

Après modification de `src/CTLD_config.yaml`, régénérez le Lua et committez les deux :

```
cd tools/ctld-tools
poetry install
poetry run ctld-tools gen-config --yaml ../../src/CTLD_config.yaml --out ../../src/CTLD_config_defaults.lua
```
Pour changer un défaut : éditez `src/CTLD_config.yaml`, rebuild (`merge_CTLD.ps1` régénère le Lua
automatiquement), committez le YAML. **Le build nécessite désormais Python** : lancez `poetry install`
dans `tools/ctld-tools` une fois (le merge appelle `ctld-tools` ; il s'arrête avec un message clair si
poetry est absent).

`tools/ctld-tools/` est un projet poetry isolé (typer, ruamel.yaml, lupa, pytest + ruff + mypy),
suivant les conventions Python de VMCT. Le job CI `python-quality` applique un **garde de parité**
(régénérer depuis le YAML reproduit les settings d'origine, wrappers `ctld.tr` inclus) et un
**contrôle de dérive** (le Lua généré committé doit être égal à un `gen-config` frais).
suivant les conventions Python de VMCT. Le job CI `python-quality` applique un **garde de parité** :
régénérer depuis le YAML reproduit les settings d'origine, wrappers `ctld.tr` inclus (une référence
figée, détectée avec un traducteur distinctif).

## Exécuter les tests (busted, sans DCS)

Expand Down
24 changes: 11 additions & 13 deletions docs/developer/building-and-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,22 +39,20 @@ output exists and is non-empty, and uploads `CTLD.lua` as a build artifact.
## Engine configuration (`ctld-tools`)

The engine defaults are **data**, not code: they live in `src/CTLD_config.yaml` (the single source
of truth, sectioned `mm_facing` / `advanced`). A generated module `src/CTLD_config_defaults.lua`
defines `ctld.__configDefaults`, which `CTLDConfig:load()` copies into its settings. **Edit the YAML,
never the generated Lua.** Both files are committed.
of truth, sectioned `mm_facing` / `advanced`). At build time `merge_CTLD.ps1` regenerates
`src/CTLD_config_defaults.lua` from the YAML (via `ctld-tools gen-config`); that file defines
`ctld.__configDefaults`, which `CTLDConfig:load()` copies into its settings. **Edit the YAML** — the
generated Lua is a **build artifact (git-ignored)**, never hand-edited or committed.

After changing `src/CTLD_config.yaml`, regenerate the Lua and commit both:

```
cd tools/ctld-tools
poetry install
poetry run ctld-tools gen-config --yaml ../../src/CTLD_config.yaml --out ../../src/CTLD_config_defaults.lua
```
To change a default: edit `src/CTLD_config.yaml`, rebuild (`merge_CTLD.ps1` regenerates the Lua
automatically), commit the YAML. **The build now needs Python**: run `poetry install` in
`tools/ctld-tools` once (the merge calls `ctld-tools`; it aborts with a clear message if poetry is
missing).

`tools/ctld-tools/` is an isolated poetry project (typer, ruamel.yaml, lupa, pytest + ruff + mypy),
following the VMCT Python conventions. The `python-quality` CI job enforces a **parity guard**
(regenerating from the YAML reproduces the original settings, `ctld.tr` wrappers included) and a
**drift check** (the committed generated Lua must equal a fresh `gen-config`).
following the VMCT Python conventions. The `python-quality` CI job enforces a **parity guard**:
regenerating from the YAML reproduces the original settings, `ctld.tr` wrappers included (a frozen
reference caught with a distinctive translator).

## Running tests (busted, no DCS)

Expand Down
Loading