Repository navigation
Run the test suite against TypeScript sources behind an uncheatable coverage gate #52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
49932c2
Run the test suite against TypeScript sources behind an uncheatable c…
unbraind b4b1112
Harden the coverage gate: TZ pinning, test-tree typecheck, dynamic so…
unbraind fd545f9
Report a runner failure before the presence check, and harden two gat…
unbraind d9a9654
Close three more gate escapes found in review: stale lcov, bad entrie…
unbraind b0db23e
Propagate the runner's exit status before reading the coverage report…
unbraind 8267e4a
Strip block comments when checking whether an ignored module is type-…
unbraind 794654b
Resolve the effective tsconfig via the compiler instead of parsing it…
unbraind 7c3ba6c
Fail closed when the effective tsconfig cannot be resolved
unbraind File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| id: pm-linear-v618 | ||
| title: Enforce a real coverage gate by running tests against TypeScript sources | ||
| description: "The package measured no coverage at all: tests imported the compiled output from dist, so the only obtainable numbers pointed at generated JavaScript offsets an author cannot act on, and no threshold was wired into any gate. The suite now runs against the TypeScript source directly. Node executes TypeScript natively from the version floor this package already declares, so tests import the source module and the coverage report lands on real source lines. The gate is scripts/coverage-gate.ts: it enforces per-dimension thresholds and reconciles the reported file list against the configured source list. That second check is the point. Node omits files that never load from its report entirely rather than reporting them at zero, so without it a package can satisfy a threshold while an entire module goes unexercised." | ||
| type: Feature | ||
| status: closed | ||
| priority: 1 | ||
| tags[3]: code-quality,coverage,testing | ||
| created_at: "2026-07-28T23:44:31.418Z" | ||
| updated_at: "2026-07-28T23:44:40.001Z" | ||
| deadline: "2026-08-04T23:44:31.418Z" | ||
| closed_at: "2026-07-28T23:44:40.000Z" | ||
| author: pi-agent | ||
| estimated_minutes: 240 | ||
| acceptance_criteria: "Tests import the TypeScript source rather than dist so coverage reports on editable lines; The gate fails the build when any threshold regresses; The gate fails when a configured source file never loads, proven by a negative control; Thresholds are recorded in package.json as a ratchet at the measured baseline; The gate runs in the release check and in CI on Node 22 and 26" | ||
| resolution: "Tests moved onto the TypeScript source and scripts/coverage-gate.ts wired into release:check and CI." | ||
| expected_result: "A coverage gate runs in the release check and in CI on both supported Node lines, reports against the TypeScript source, and fails when the source is untested or a threshold regresses." | ||
| actual_result: "npm run check passes. npm run coverage passes with 63 tests, index.ts present in the report, at 70.53 line, 80.42 branch and 80.43 function coverage. A negative control adding an untested __probe.ts source made the gate exit 1 and name that file; removing it returned exit 0. Thresholds pinned at 70/80/80. CI legs unobserved until the PR runs." | ||
| comments[1]{created_at,author,text}: | ||
| "2026-07-28T23:44:35.248Z",pi-agent,"Negative control run: with a throwaway __probe.ts source added to coverageGate.sources, the gate exited 1 and named __probe.ts as never loaded; after removing the probe it exited 0 again. Presence check is not vacuous." | ||
| learnings[1]{created_at,author,text}: | ||
| "2026-07-28T23:44:35.677Z",pi-agent,"Node omits never-loaded files from its coverage report entirely rather than reporting them at zero, so a threshold flag alone is satisfiable by narrowing what the suite imports. Any coverage gate must reconcile the report against the configured source list." | ||
| close_reason: Coverage gate landed and locally verified; the Node 22 and 26 CI legs are unobserved until the pull request runs. | ||
| body: "Baseline measured 2026-07-29 across the single source file index.ts: 70.53 line, 80.42 branch, 80.43 function. Thresholds pinned at the measured floor (70/80/80) so the number can only ratchet up." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| {"ts":"2026-07-28T23:44:31.418Z","author":"pi-agent","author_source":"asserted","agent_harness":"pi","agent_provenance":{"model":null},"op":"create","patch":[{"op":"replace","path":"/body","value":"Baseline measured 2026-07-29 across the single source file index.ts: 70.53 line, 80.42 branch, 80.43 function. Thresholds pinned at the measured floor (70/80/80) so the number can only ratchet up."},{"op":"add","path":"/metadata/id","value":"pm-linear-v618"},{"op":"add","path":"/metadata/title","value":"Enforce a real coverage gate by running tests against TypeScript sources"},{"op":"add","path":"/metadata/description","value":"The package measured no coverage at all: tests imported the compiled output from dist, so the only obtainable numbers pointed at generated JavaScript offsets an author cannot act on, and no threshold was wired into any gate. The suite now runs against the TypeScript source directly. Node executes TypeScript natively from the version floor this package already declares, so tests import the source module and the coverage report lands on real source lines. The gate is scripts/coverage-gate.ts: it enforces per-dimension thresholds and reconciles the reported file list against the configured source list. That second check is the point. Node omits files that never load from its report entirely rather than reporting them at zero, so without it a package can satisfy a threshold while an entire module goes unexercised."},{"op":"add","path":"/metadata/type","value":"Feature"},{"op":"add","path":"/metadata/status","value":"open"},{"op":"add","path":"/metadata/priority","value":1},{"op":"add","path":"/metadata/tags","value":["code-quality","coverage","testing"]},{"op":"add","path":"/metadata/created_at","value":"2026-07-28T23:44:31.418Z"},{"op":"add","path":"/metadata/updated_at","value":"2026-07-28T23:44:31.418Z"},{"op":"add","path":"/metadata/deadline","value":"2026-08-04T23:44:31.418Z"},{"op":"add","path":"/metadata/assignee","value":"pi-agent"},{"op":"add","path":"/metadata/author","value":"pi-agent"},{"op":"add","path":"/metadata/estimated_minutes","value":240},{"op":"add","path":"/metadata/acceptance_criteria","value":"Tests import the TypeScript source rather than dist so coverage reports on editable lines; The gate fails the build when any threshold regresses; The gate fails when a configured source file never loads, proven by a negative control; Thresholds are recorded in package.json as a ratchet at the measured baseline; The gate runs in the release check and in CI on Node 22 and 26"},{"op":"add","path":"/metadata/expected_result","value":"A coverage gate runs in the release check and in CI on both supported Node lines, reports against the TypeScript source, and fails when the source is untested or a threshold regresses."}],"before_hash":"3cc22dff72be7b14824654a7a64ea62b04799939b2fee54c1b5f52ca60bf6df0","after_hash":"b568294529a3482ab96c1a05fc1ff6cdc67d169329dd60c5915ed2e18097c0dc","message":""} | ||
| {"ts":"2026-07-28T23:44:35.248Z","author":"pi-agent","author_source":"asserted","agent_harness":"pi","agent_provenance":{"model":null},"op":"comment_add","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-07-28T23:44:35.248Z"},{"op":"add","path":"/metadata/comments","value":[{"created_at":"2026-07-28T23:44:35.248Z","author":"pi-agent","text":"Negative control run: with a throwaway __probe.ts source added to coverageGate.sources, the gate exited 1 and named __probe.ts as never loaded; after removing the probe it exited 0 again. Presence check is not vacuous."}]}],"before_hash":"b568294529a3482ab96c1a05fc1ff6cdc67d169329dd60c5915ed2e18097c0dc","after_hash":"3d60f1f5a5512f3fd310ad27af56ea57968ca8f4f25cf43dab06fb7a6b9a7d97"} | ||
| {"ts":"2026-07-28T23:44:35.677Z","author":"pi-agent","author_source":"asserted","agent_harness":"pi","agent_provenance":{"model":null},"op":"learning_add","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-07-28T23:44:35.677Z"},{"op":"add","path":"/metadata/learnings","value":[{"created_at":"2026-07-28T23:44:35.677Z","author":"pi-agent","text":"Node omits never-loaded files from its coverage report entirely rather than reporting them at zero, so a threshold flag alone is satisfiable by narrowing what the suite imports. Any coverage gate must reconcile the report against the configured source list."}]}],"before_hash":"3d60f1f5a5512f3fd310ad27af56ea57968ca8f4f25cf43dab06fb7a6b9a7d97","after_hash":"f6cef1ff20960102859df670da0c05689557ed09354b1decd9c9ded8f5292832"} | ||
| {"ts":"2026-07-28T23:44:40.001Z","author":"pi-agent","author_source":"asserted","agent_harness":"pi","agent_provenance":{"model":null},"op":"close","patch":[{"op":"remove","path":"/metadata/assignee"},{"op":"replace","path":"/metadata/updated_at","value":"2026-07-28T23:44:40.001Z"},{"op":"replace","path":"/metadata/status","value":"closed"},{"op":"add","path":"/metadata/closed_at","value":"2026-07-28T23:44:40.000Z"},{"op":"add","path":"/metadata/resolution","value":"Tests moved onto the TypeScript source and scripts/coverage-gate.ts wired into release:check and CI."},{"op":"add","path":"/metadata/actual_result","value":"npm run check passes. npm run coverage passes with 63 tests, index.ts present in the report, at 70.53 line, 80.42 branch and 80.43 function coverage. A negative control adding an untested __probe.ts source made the gate exit 1 and name that file; removing it returned exit 0. Thresholds pinned at 70/80/80. CI legs unobserved until the PR runs."},{"op":"add","path":"/metadata/close_reason","value":"Coverage gate landed and locally verified; the Node 22 and 26 CI legs are unobserved until the pull request runs."}],"before_hash":"f6cef1ff20960102859df670da0c05689557ed09354b1decd9c9ded8f5292832","after_hash":"f7dd19c6de6ed2b4bbc42bcf63068f130407dbc1f8a7416d012956ff46aeb5c8"} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,3 +13,5 @@ dist-test/ | |
| .env.* | ||
| !.env.example | ||
| .agents/pm/extensions/ | ||
|
|
||
| coverage/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.