Skip to content

Derive the planner's test-file inputs from the gate scripts that run them #4473

Description

@Astro-Han

Problem

Whether a scripts/*.test.mjs file runs, and whether a change to it schedules the lane that runs it, are two facts written in three places:

  1. the file itself;
  2. an npm gate script (check:release, check:asf-source) or a node --test step in .github/workflows/ci.yml;
  3. a literal path list in scripts/ci-test-plan.mjs (RELEASE_CONTRACT_FILES, ASF_SOURCE_FILES, and six siblings — 120 hand-written entries across eight sets).

Adding a test requires editing all three. Missing the third means the gate still runs the test, but no change to the test itself selects the lane that runs it, so the test can be edited green.

This has now happened at least three times:

Two guardrails in scripts/ci-test-plan.test.mjs catch the drift after the fact and both have fired — every test a gate script runs reaches a lane that runs that gate and the planner names no path that no longer exists. They convert a silent failure into a loud one, which is a real improvement over nothing, but they exist only to police a duplication that does not need to be there.

Suggested direction

Have the planner derive the test-file entries instead of listing them: read package.json, take every scripts/*.test.mjs a gate script names, and treat it as an input to that gate's lane. Three places become one, and both guardrails lose their reason to exist.

The trade-off worth deciding explicitly: this makes the planner's output depend on npm script text, where today it is a static table. The planner runs before npm ci, so reading package.json is free, but it does widen what a package.json edit can change.

.github/workflows/ci.yml names about eleven more test files directly in node --test steps rather than through an npm script. Those are already covered — the steps that run them are unconditional — but if the derivation is worth doing, it is worth deciding whether those should be derived too, or left alone because an unconditional step needs no selection at all.

Scope note

Split out of #4461, which narrowed several gates and made each one derive its inputs from its own authority. This is the same principle applied to a different list, but it changes planner behaviour for every lane rather than only the ones that PR narrowed, so it was left out deliberately rather than forgotten.

Not in scope

The two guardrails should stay until the derivation lands. They are what makes this issue's failure mode visible at all.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requesthelp wantedExtra attention is needed

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions