Skip to content

feat(config): Mission Maker YAML authoring — validate + gen-user (CTLD-TOOLS-USERCONFIG)#47

Merged
davidp57 merged 2 commits into
developfrom
feature/ctld-tools-userconfig
Jul 20, 2026
Merged

feat(config): Mission Maker YAML authoring — validate + gen-user (CTLD-TOOLS-USERCONFIG)#47
davidp57 merged 2 commits into
developfrom
feature/ctld-tools-userconfig

Conversation

@davidp57

@davidp57 davidp57 commented Jul 20, 2026

Copy link
Copy Markdown
Member

Implements the CTLD-TOOLS-USERCONFIG lot (lot 3 of ctld-tools — the MM volet). Depends on
FEAT-USERCONFIG-API + CTLD-TOOLS-CONFIG (both merged).

What

  • validate — checks a user-config.yaml against the reference catalogue (crates incl. AA,
    troop groups, array settings) and the embedded DCS type set. Reports errors with suggestions
    (unknown/ambiguous crate name, unknown unit type, weight collision, unknown troop/array).
  • gen-user — compiles add / remove / patch operations into a CTLD_userConfig.lua
    calling the ctld.userSetup helpers. Mission Makers target by NAME — the tool resolves names
    to the unique weight the runtime uses (removeCrate(1000.05) from "Heavy Tank - Abrams"), maps
    namedesc (wrapped ctld.tr), and emits the scalar settings block too.
  • gen-user --scaffold — commented starter user-config.yaml, showing block + flow styles.
  • Embedded dataminedcs_types.json bundled in the package (kept in sync with the vendored
    dcs_types.lua), for offline unit validation. gen_dcs_types.py now emits both.
  • Distributionrelease.yml builds + attaches ctld-tools.exe (PyInstaller), isolated
    (continue-on-error) so a packaging hiccup never blocks the CTLD.lua release.

Validation

  • 26 tests green (ruff + mypy + pytest). Includes an end-to-end test: the generated
    CTLD_userConfig.lua run against the real ctld.userSetup helpers mutates settings
    correctly (crate added present, Abrams removed).
  • No src/ Lua change — this lot is Python + docs + CI. The Lua template stays for power users;
    the YAML flow is documented as recommended (mission-maker/configuration.md, EN + FR).

Notes

  • UX per Mission-Maker feedback: names not weights; block and flow YAML shown, MM's choice.
  • Closes the ctld-tools program; remaining roadmap items: .miz injection, TUI.

Summary by Sourcery

Add Mission Maker-focused YAML authoring support to ctld-tools, including validation and Lua user config generation, backed by an embedded DCS type dataset and exposed via the CLI and release packaging.

New Features:

  • Introduce validate CLI command to check a user-config.yaml against the CTLD reference catalogue and DCS type set, reporting findings and exiting non-zero on errors.
  • Introduce gen-user CLI command to compile a user-config.yaml into CTLD_userConfig.lua, supporting both full generation and scaffold creation for new configs.
  • Provide a commented YAML scaffold file for Mission Makers, illustrating scalar settings, crate/troop operations, and array appends.
  • Add a reference catalogue loader that injects AA crates and resolves crate and troop group names to runtime keys for use by validation and generation.
  • Add an embedded, machine-readable DCS type set JSON and helper to load it for offline unit-type validation.

Enhancements:

  • Extend Lua config loading to optionally inject AA crates so the reference catalogue covers AA systems for name-based resolution.
  • Document the recommended YAML-based authoring flow for Mission Makers in English and French configuration guides, including example commands.
  • Add a module entry point for ctld-tools to support python -m ctld_tools and PyInstaller packaging.

Build:

  • Update the DCS types generation script to emit both the Lua type set and a synced JSON file into the ctld-tools package.
  • Enhance the release workflow to build a PyInstaller-based ctld-tools.exe and attach it to GitHub releases, isolated so failures do not block CTLD.lua releases.

Tests:

  • Add tests for user config generation, including end-to-end execution against real ctld.userSetup helpers to verify settings mutations.
  • Add validation tests to cover crate and troop configuration errors, weight collisions, unknown unit types, array setting misuse, and suggestion messages.
  • Add tests for the reference catalogue to validate crate/troop resolution, AA crate injection, and array setting recognition.
  • Add tests ensuring the YAML scaffold file is valid, compiles cleanly, and produces a minimal Lua user config.
  • Add tests for the embedded DCS type dataset to confirm it loads correctly and stays in sync with the vendored Lua type set.

…D-TOOLS-USERCONFIG)

Implements lot 3 of ctld-tools (the MM volet). `validate` checks a user-config.yaml
against the reference catalogue + embedded DCS type set, reporting errors with
suggestions. `gen-user` compiles add/remove/patch operations into CTLD_userConfig.lua
calling the ctld.userSetup helpers. Mission Makers target crates and troop groups
BY NAME — ctld-tools resolves names to the unique weight the runtime uses (AA crates
included), and flags unknown/ambiguous names.

`gen-user --scaffold` writes a commented starter (block + flow styles). Embedded
dcs_types.json (kept in sync with the vendored Lua set) enables offline unit checks.
release.yml builds + attaches ctld-tools.exe (isolated, non-blocking). Docs EN+FR
present the YAML flow as recommended; the Lua template stays for power users.
@sourcery-ai

sourcery-ai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds Mission Maker-focused YAML authoring to ctld-tools via new validate/gen-user commands, backed by a reference catalogue + embedded DCS type datamine, plus packaging and docs updates so a standalone ctld-tools.exe is attached to releases without impacting CTLD.lua delivery.

Sequence diagram for ctld-tools gen-user command generating CTLD_userConfig.lua

sequenceDiagram
    actor MissionMaker
    participant CLI as gen_user_cmd
    participant GenUser as generate_user_file
    participant Validate as render_user_config
    participant Ref as Reference.from_src
    participant Types as known_dcs_types
    participant FS as FileSystem

    MissionMaker->>CLI: ctld-tools gen-user --yaml --src --out
    CLI->>GenUser: generate_user_file(yaml_path, src, out)
    GenUser->>Validate: render_user_config(load_user_config(yaml_path), Reference.from_src(src))
    GenUser->>Ref: Reference.from_src(src)
    Ref->>FS: load_default_settings(src, inject_aa=True)
    Ref-->>GenUser: Reference
    Validate->>Validate: validate(cfg, ref)
    Validate->>Types: known_dcs_types()
    Types-->>Validate: frozenset[type_names]
    alt has_errors(findings)
        Validate-->>GenUser: raise UserConfigError(findings)
        GenUser-->>CLI: UserConfigError
        CLI->>MissionMaker: print findings
        CLI-->>MissionMaker: exit 1
    else no errors
        Validate-->>GenUser: rendered Lua string
        GenUser->>FS: write_text(out_path, lua)
        CLI->>MissionMaker: echo "gen-user: wrote CTLD_userConfig.lua"
    end
Loading

Flow diagram for DCS type datamine generation and validation usage

flowchart LR
    DCSRepo[Quaggles datamine repo] --> GenScript[gen_dcs_types.py]
    GenScript --> LuaOut[tests/data/dcs_types.lua]
    GenScript --> JsonOut[ctld_tools/data/dcs_types.json]
    JsonOut --> Package[ctld-tools package / ctld-tools.exe]
    Package --> Datamine[known_dcs_types]
    Datamine --> Validate[validate user_config, Reference, types]
    Validate --> CLIValidate[validate_cmd]
    Validate --> CLIGenUser[gen_user_cmd / render_user_config]
Loading

File-Level Changes

Change Details Files
Expose new Mission Maker CLI commands for validating user-config YAML and generating CTLD_userConfig.lua (including scaffold support) with error reporting.
  • Add validate_cmd Typer command that loads user-config.yaml, validates against Reference and DCS types, prints findings and exits non-zero on errors.
  • Add gen_user_cmd Typer command that either writes a scaffold YAML via write_scaffold or compiles a user-config.yaml into CTLD_userConfig.lua via generate_user_file, surfacing validation findings as CLI output.
tools/ctld-tools/ctld_tools/cli.py
Bundle a PyInstaller-built ctld-tools.exe with releases in a way that does not block CTLD.lua publishing.
  • Add Python setup step using actions/setup-python@v6 (3.13) in release workflow with continue-on-error.
  • Install poetry, build ctld-tools onefile executable with PyInstaller including lupa and embedded dcs_types.json, and attach ctld-tools.exe to the GitHub release only if build succeeds; all steps are continue-on-error.
.github/workflows/release.yml
Document the recommended YAML-based Mission Maker workflow and new ctld-tools commands in English and French docs and the ctld-tools README.
  • Add tip sections in EN/FR mission-maker configuration docs that describe scaffold→validate→gen-user flow and note ctld-tools.exe availability per release.
  • Update ctld-tools README to introduce engine config vs Mission Maker command groups and describe validate/gen-user (including scaffold) usage.
docs/mission-maker/configuration.md
docs/mission-maker/configuration.fr.md
tools/ctld-tools/README.md
CHANGELOG.md
Extend Lua defaults loader to support AA crate injection so the reference catalogue includes AA crates for name resolution.
  • Add inject_aa flag to load_default_settings, and when true, stub ctld.utils.log and call CTLDCrateAssemblyManager.injectAACrates on spawnableCrates before returning settings.
  • Ensure existing behavior is preserved when inject_aa is False.
tools/ctld-tools/ctld_tools/luaconfig.py
Generate and embed a machine-readable DCS type name set for offline unit validation and keep it in sync with the vendored Lua datamine.
  • Teach gen_dcs_types.py to emit a compact dcs_types.json into ctld_tools/data alongside the existing tests/data/dcs_types.lua and log both outputs.
  • Add ctld_tools.datamine.known_dcs_types helper that lazily loads dcs_types.json via importlib.resources and exposes it as a frozenset.
tools/dcs-data/gen_dcs_types.py
tools/ctld-tools/ctld_tools/datamine.py
tools/ctld-tools/ctld_tools/data/dcs_types.json
Implement user-config rendering logic that validates configuration and compiles crate/troop/array operations into ctld.userSetup helper calls, resolving names to weights.
  • Introduce UserConfigError to carry validation findings and block generation on errors.
  • Implement scalar block emission into ctld.yamlConfigDatas from settings, crate add/remove/patch mapping with name→desc and weight handling, troop add/remove helpers, and array appends via ctld.addTo.
  • Wire render_user_config to re-run validate against Reference and raise on errors, and generate_user_file to glue YAML loading, Reference.from_src, and CTLD_userConfig.lua writing.
tools/ctld-tools/ctld_tools/genuser.py
Implement validation of user-config.yaml against the reference catalogue and known DCS types, producing structured findings for crates, troops, and array settings.
  • Define Finding dataclass with severity/where/message and string formatting, plus load_user_config YAML loader and has_errors helper.
  • Add crate validation for presence of required fields, unit type membership in known DCS types, unique weight_kg, and crate name/weight resolution via Reference.resolve_crate, emitting helpful error messages and suggestions.
  • Validate troop additions for required names, troop removals via troop_exists/closest_troop, and array setting keys via is_array_setting.
tools/ctld-tools/ctld_tools/validate.py
Introduce a Reference abstraction over CTLD default settings to resolve crate/troop names and recognize appendable array settings.
  • Load settings via load_default_settings with inject_aa=True and index spawnableCrates into crate name→[(weight,section)] mapping and a weight set, plus loadableGroups into a troop name set.
  • Provide crate_weights, resolve_crate (handling numeric weights, unknown names, and ambiguous name collisions with suggestions), troop_exists/closest_troop, and is_array_setting helpers using a fixed ARRAY_SETTINGS list.
tools/ctld-tools/ctld_tools/reference.py
Provide a commented YAML scaffold generator for user-config.yaml that is safe to compile and demonstrates both block and flow styles.
  • Define a SCAFFOLD constant containing a fully commented example user-config with settings, crates, troops, and arrays sections plus usage instructions.
  • Implement write_scaffold to write the scaffold text with UTF-8 encoding and LF newlines.
tools/ctld-tools/ctld_tools/scaffold.py
Add tests covering gen-user behavior (including end-to-end integration), validation rules, reference catalogue resolution, scaffold validity, and datamine synchronization.
  • test_genuser.py checks operation compilation, name resolution, error blocking via UserConfigError, and an end-to-end run where generated CTLD_userConfig.lua is executed with real ctld.userSetup helpers to mutate settings as expected.
  • test_validate.py exercises valid configs, unknown unit errors, weight collisions, crate name suggestions, troop removal errors, and invalid array settings.
  • test_reference.py verifies crate resolution by name/weight (including AA crates), suggestion generation, and troop/array helpers.
  • test_scaffold.py ensures scaffold can be parsed, validates cleanly, and renders a CTLD_userConfig.lua stub without error.
  • test_datamine.py asserts known_dcs_types is nonempty, contains expected types, and matches the type set parsed from tests/data/dcs_types.lua.
tools/ctld-tools/tests/test_genuser.py
tools/ctld-tools/tests/test_validate.py
tools/ctld-tools/tests/test_reference.py
tools/ctld-tools/tests/test_scaffold.py
tools/ctld-tools/tests/test_datamine.py
Expose ctld_tools CLI as a Python module entry point to support python -m ctld_tools and PyInstaller packaging.
  • Add main.py that imports ctld_tools.cli.main and invokes it under main guard.
tools/ctld-tools/ctld_tools/__main__.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • In _scalar_block you concatenate raw Python values into the ctld.yamlConfigDatas YAML block, which will emit booleans as True/False and won’t quote strings; consider reusing _render_value or a dedicated scalar serializer so the block is always valid YAML for the Lua side.
  • The list of ARRAY_SETTINGS in reference.py is hard-coded and can silently drift from the actual appendable settings in CTLD; if possible, derive this from the loaded settings or a single source of truth in the Lua/YAML config instead of duplicating the list here.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `_scalar_block` you concatenate raw Python values into the `ctld.yamlConfigDatas` YAML block, which will emit booleans as `True`/`False` and won’t quote strings; consider reusing `_render_value` or a dedicated scalar serializer so the block is always valid YAML for the Lua side.
- The list of `ARRAY_SETTINGS` in `reference.py` is hard-coded and can silently drift from the actual appendable settings in CTLD; if possible, derive this from the loaded settings or a single source of truth in the Lua/YAML config instead of duplicating the list here.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@davidp57
davidp57 merged commit 0910bd1 into develop Jul 20, 2026
7 checks passed
@davidp57
davidp57 deleted the feature/ctld-tools-userconfig branch July 20, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant