Repository navigation
Did your last release change what your plugin can do? (2-minute check) #7
Description
Hi — I maintain dsh-vet, the static audit tool for DSH plugins. It just learned to compare two releases of the same plugin and report exactly what risk-relevant behavior changed — new endpoints, new capabilities, new install scripts, changed confidence — while ignoring line moves and reformatting.
I ran it on your last two releases (0.9.5 → 0.9.6) so you don't have to install anything. Both scans used the same scanner, full rule profile, complete coverage.
Zero risk-relevant changes. All 7 audited behaviors identical, grade C → C. If that matches your intent for the release, the diff just saved you the re-review — and it would have caught a new endpoint or install script if there'd been one.
Three questions:
- Does that match what you intended to change in that release?
- Would a per-PR "what changed since the base revision" summary (one Action input) be something you'd read before merging?
- Anything you'd want such a comparison to surface that static analysis can't see today?
Answers in any form are useful; I'll record anonymized takeaways in the pilot record. This is the last feedback loop before deciding whether to build CI gating on top (opt-in "fail on new risk"), so negative answers are as valuable as positive ones.
— Roger (dsh-vet)
Activity
pan17 commentedon Sep 11, 2026
Thanks for running this comparison.
Yes, the result matches the intent of the 0.9.6 release. The release only changed
/historyfiltering and presentation, adding/history alland related tests. It was not intended to add or remove any risk-relevant capability, endpoint, or install behavior, so the C → C result with zero added, removed, or changed behaviors looks correct.A per-PR “what changed since the base revision” summary would be useful. I would especially value:
One limitation I would like to see surfaced, where possible, is behavior that depends on the composed DSH profile or host version—for example, runtime service contracts, registered tools/commands, or
ctx.injectbehavior. Those may not be fully visible from the plugin package alone. A warning that the result only covers package-level static behavior would still be useful.Thanks again—the comparison was helpful and saved a manual re-review for this release.
rogerdigital commentedon Sep 11, 2026
Thanks for the detailed reply — this is exactly the feedback loop we needed, and glad it saved the re-review.
On the scope limitation you raised: agreed, and it's now stated explicitly on every
dsh-vet diffoutput and every PR comparison section (not only inside the scan context's coverage limitations):That change is merged and ships in the next release. The other four properties you listed (no-changes vs. incomparable distinction, prominent added/changed callouts, comparability info, observations at unchanged grades) are how the diff already reads in 0.4.0 — good to know the design landed where you'd want it.
If you cut another release, we'd genuinely value hearing whether you run the comparison again — repeated real use is the bar we set ourselves before building any CI gating on top (opt-in "fail on new risk"), so a "used it a second time" (or "didn't, because X") matters as much as a positive first impression.
— Roger