Skip to content

test: validate real builder configs in ci#380

Open
Scr4tch587 wants to merge 1 commit into
mainfrom
kai/validate-scripts-ci
Open

test: validate real builder configs in ci#380
Scr4tch587 wants to merge 1 commit into
mainfrom
kai/validate-scripts-ci

Conversation

@Scr4tch587

Copy link
Copy Markdown
Collaborator

What changed

New `tests/core/runtime/test_real_scripts.py` that validates the real `builders/scripts/` directory (not fixtures):

  • runs `load_all_configs()` over the actual scripts dir — the exact validation the server runs at startup (structure, deps exist, cycle check, granularity, start-date constraints)
  • per-dataset parametrized check that each `builder.py` exists, parses, and defines a top-level `build(dependencies, timestamp)` with the right arity (AST-only, so builder dependencies are never imported and no `.env` is needed)

Why

Until now, nothing in CI validated the real scripts directory — an invalid `config.toml` merged to main would only surface as a server startup failure at the next deploy. With the upcoming dataset-creation flow opening PRs that add dataset directories, CI needs to be the reviewer's guarantee.

Benefit

Green CI on a dataset PR now means: this dataset cannot break server boot. Reviewers can focus on builder logic instead of config mechanics.

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant