ci: run the generated-model tests and verify regeneration freshness - #39
Merged
damaz91 merged 1 commit intoAug 10, 2026
Merged
Conversation
Observed: tests/spec-constraints.test.js and tests/spec-set-constraints.test.js (the merged Universal-Commerce-Protocol#34/Universal-Commerce-Protocol#37 constraint/injector work) exist and `npm test` is wired in package.json, but no workflow runs them. The model-generation pipeline (generate_models.sh + scripts/*.mjs) is likewise never exercised in CI. So both directions of a regression ship green: - an injector regresses but the committed src/spec_generated.ts is not regenerated: `npm test` still reads the old committed models and passes; - the committed models go stale versus the pinned UCP spec: nothing compares them to a fresh regeneration. Expected: CI runs the committed tests, and asserts the committed models equal a fresh regeneration from the UCP spec release the 0.4.x line targets (README compatibility table: 0.4.x -> 2026-04-08). This adds .github/workflows/tests.yml with two jobs, following the repo's own workflow idiom (actions/checkout@v5, actions/setup-node@v6, node-version lts/*, npm ci; push/pull_request to main like linter.yml): - test: npm ci + npm test, wiring the existing Universal-Commerce-Protocol#34/Universal-Commerce-Protocol#37 tests into CI. - model-drift: checkout the pinned ucp@release/2026-04-08, regenerate via generate_models.sh, normalize with the repo's own pinned pre-commit prettier hook (single source of truth, no second prettier version), then git diff --exit-code src/spec_generated.ts with an actionable ::error::. The drift job is a check, not a generator: it never auto-commits. The committed artifact stays the release path, aligning with the committed-artifact direction maintainers preferred over the auto-commit approach in the closed Universal-Commerce-Protocol#10. This parallels the merged python-sdk#62, which added the same test-plus-regeneration CI to the Python SDK.
vishkaty
force-pushed
the
ci-run-tests-and-drift
branch
from
August 8, 2026 21:25
a9ff669 to
8f316e3
Compare
damaz91
added
status:needs-triage
damaz91
approved these changes
Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Observed
js-sdk has generated-model tests (
tests/spec-constraints.test.js,spec-set-constraints.test.js— the #34/#37 injector work) and a wirednpm test, but no workflow runs them or the model generation. The existing workflows are conventional-commits, governance, linter (pre-commit only), pr-review-listener, and release (build+publish on release). So:.int().gte(0)fromPriceSchema.amountmakesnpm testred, but no workflow runs it.npm teststays green — only regenerate-and-compare catches it.This is the same CI-theater gap closed for python-sdk in #62.
Change
A
tests.ymlwith two jobs, matching the repo's own idiom:test—npm ci+npm test(wires the existing tests into CI).model-drift— checks out the pinned spec (ucp@release/2026-04-08, which the 0.4.x line targets per the README), regenerates, normalizes via the repo's own pinned pre-commit prettier hook, and fails if the committedsrc/spec_generated.tsdiffers. No auto-commit — the committed-artifact flow stays the release path (this is a drift check, aligning with the direction in the closed feat: add auto-generate models workflow #10, not a generator).The generator toolchain is already deterministic via
package-lock.json(npm ci), so no extra pin is needed.Verification (locally, by hand)
npm testRaw quicktype output is unformatted (1707-line diff), so prettier normalization is mandatory and reproduces the committed file byte-for-byte. actionlint clean; the workflow matches the repo's existing action-tag convention and adds
permissions: contents: read.