Skip to content

Check for regressions between PyPI and reqstool-client@main CI runs #105

Description

@jimisola

Context

`.github/workflows/build.yml` now runs the `reqstool status`/`reqstool validate` steps twice via a matrix (`reqstool-source: [pypi, main]`) — once against the latest released PyPI `reqstool` package, once against `reqstool-client`'s `main` branch (installed via `pip install "reqstool @ git+https://github.com/reqstool/reqstool-client.git@main"\`).

This was added so CI exercises both the stable release and the in-development `main` branch, since `reqstool-client` is intentionally holding off on its next release until the org-wide OpenSpec dogfooding rollout (see `PLAN_dog_fooding.md`) is complete — `main` has accumulated real fixes (e.g. #411) that aren't on PyPI yet.

What to check

  • Periodically compare the two matrix legs' `reqstool status` output for unexpected divergence (beyond the known, intentional gap — `reqstool validate --strict` only runs on the `main` leg since that subcommand isn't published yet).
  • Once `reqstool-client` cuts its next release, confirm the `pypi` leg's output matches the `main` leg's output exactly, then collapse this back down to a single non-matrixed job (drop the `main` leg, restore `reqstool validate --strict` unconditionally).
  • Watch for the `pypi` leg silently masking a regression that only the `main` leg would catch (or vice versa) during the rollout period.

Related: reqstool-client#412 (CLI/MCP consolidation, found during this same rollout).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions