Skip to content

Add CLI smoke checks against repo-root test_data/ fixtures in CI #61

@hbelmiro

Description

@hbelmiro

Summary

We want CI to run selected scripts/ CLIs against the committed repo-root test_data/ fixtures (the “golden snapshot”) and require exit code 0. This provides an integration-style sanity check that scripts work end-to-end on representative assets.

Proposed CI location

Add steps to .github/workflows/scripts-tests.yml (for the scripts matrix entry) so these checks run when scripts change.

Candidate CLIs

These CLIs are intended to be “read-only” checks (no file modifications):

  • scripts/check_base_image_tags/check_base_image_tags.py
  • scripts/validate_components/validate_components.py
  • scripts/validate_base_images/validate_base_images.py
  • scripts/validate_metadata/validate_metadata.py
  • scripts/generate_readme (check mode)

Also, evaluate new CLIs that were included after the creation of this issue.

Constraints / design notes

  • The CLIs should be runnable against test_data/ without requiring mutation of fixtures or the repository.
  • If a CLI currently can’t target test_data/ due to hard-coded assumptions (e.g., requiring components/ at repo root, assuming importable packages), options include:
    • removing those repo-layout assumptions and operating from file paths; or
    • introducing a flag like --repo-root PATH to treat test_data/ as the logical root.

Acceptance criteria

  • A set of steps in .github/workflows/scripts-tests.yml runs the chosen CLIs against test_data/ and fails the workflow on non-zero exit.
  • The steps are gated so they don’t run in the .github/scripts matrix entry.
  • The smoke checks are stable across the Python versions used in CI.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions