Repository navigation
fix(drift-audit): two silent failures that hid real drift for two months - #1185
Merged
Merged
Conversation
The audit has run green every Monday since 2026-07-06 while filing exactly
nothing and while one of its checks could never have passed. Both are fixed
here; the drift it was hiding is fixed in the four companion PRs.
1. Issue emission failed silently.
emit_issue() called _gh(), which discards stderr and returns None on
failure, then printed "opened new issue on <repo>" WITHOUT checking. main()
returned 0 unconditionally in emit mode ("the issues ARE the deliverable").
DRIFT_AUDIT_TOKEN has never been set, so the job ran on GITHUB_TOKEN, which
cannot open an issue on a sibling repo -- and the displayxr-publish-bot App
has no `issues` permission at all, so wiring it would not have helped
either. Seven weekly runs, five claimed issues each, zero that exist: no
drift issue has EVER been filed on any repo, open or closed.
Now: writes go through _gh_full() and are checked; failure prints the real
gh error; the run raises a ::warning::; and the findings still land as ONE
aggregate issue on this repo, which a workflow token can always write to.
The exit code now tracks whether the findings reached a human rather than
whether drift exists.
2. The vendored-spec comparison could never clear.
strip_license_header() treated lines starting with '#', '*' and '//' as
comments. In a markdown doc those are content: it ate the canonical file's
'# Title' and '**Status:**' paragraph while stopping dead at the vendored
copy's '> Vendored copy' provenance banner. So the two hashed unequal no
matter how faithfully the copy was synced -- confirmed by hashing a
byte-perfect re-sync, which still compared unequal. A permanently
un-actionable finding.
Replaced with strip_doc_banner(), which drops leading blockquote/HTML-
comment provenance and leaves markdown content alone. Verified: the current
copy still differs (real drift), a faithful re-sync now matches.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ua8CevgSHNqoyiy7ziZjHk
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You asked how we let the drift happen. We didn't — the detector caught it every week and its reporting was broken, with a green check on top.
drift-audithas run every Monday since 2026-07-06, all seven runs green. Today's log says:No such issue has ever existed on any of those repos — open or closed.
Failure 1 — emission failed silently
emit_issue()called_gh(), which discards stderr and returnsNoneon failure, then printedopened new issue on <repo>without checking.main()returned 0 unconditionally in emit mode ("the issues ARE the deliverable — don't also fail the job").DRIFT_AUDIT_TOKENhas never been set, so the job runs onGITHUB_TOKEN, which cannot open an issue on a sibling repo. Wiring the publish-bot App instead wouldn't have helped — it has noissuespermission at all:Now: writes go through
_gh_full()and are checked; failure prints the realgherror; the run raises a::warning::; and findings still land as one aggregate issue on this repo, which a workflow's own token can always write to. The exit code now tracks whether findings reached a human, not whether drift exists.Verified against a deliberately unwritable repo:
Failure 2 — the vendored-spec check could never clear
strip_license_header()treated#,*and//as comment markers. In markdown those are content: it ate the canonical file's# Titleand**Status:**paragraph while stopping dead at the vendored copy's> **Vendored copy.**banner. The two could never hash equal however faithfully the copy was synced.Confirmed by hashing a byte-perfect re-sync — still unequal. A permanently un-actionable finding, which is the kind that teaches people to ignore an audit.
Replaced with
strip_doc_banner(). Verified both directions: the current copy still differs (real drift), a faithful re-sync now matches.Companion PRs — the drift itself
displayxr-commonv2.5.0 → v2.7.0displayxr-commonv2.6.1 → v2.7.0SupportsFileDialog)Still needs a human decision
The aggregate-issue fallback works with no new secrets, but it's a fallback. To restore per-repo issues, either set
DRIFT_AUDIT_TOKENto an org-scoped PAT withissues: write, or grantdisplayxr-publish-bottheissuespermission. Both are org-admin actions I can't take.