Skip to content

ums: pull_requests[].head.sha tracks the PR, not the run - #1388

Merged
d-morrison merged 4 commits into
mainfrom
ums/pr1384-pull-requests-head-sha
Aug 10, 2026
Merged

ums: pull_requests[].head.sha tracks the PR, not the run#1388
d-morrison merged 4 commits into
mainfrom
ums/pr1384-pull-requests-head-sha

Conversation

@d-morrison

@d-morrison d-morrison commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Closes #1387.

UMS pass for lessons from PR #1384's review lifecycle (merged as edfab8d8).

Every figure below was derived by command at 19fc91b6, the head this PR
pushes, with the deriving command published beside it.

Corrections to this body

Dogfooding the rule this PR adds to
address-every-comment.md: the body
is edited and the correction recorded, so nothing is silently overwritten
and any earlier comment citing the old numbers still resolves.

Figure At ede6b0a9 At 5bed1d61 At 19fc91b6 (current)
Added lines 162 184 228
fully-clean.cases.md 512 -> 585 512 -> 607 512 -> 651
Column-1 # hits 3 4 4
Relative links checked 1336 1336 1337

Causes: 5bed1d61 recorded a counterexample that arrived mid-review;
19fc91b6 addresses the round-1 review finding.

What landed, and where

Lesson Disposition Home
1. A dispatch-triggered run's pull_requests[].head.sha is a third misleading SHA surface Recorded fully-clean.md (rule) + fully-clean.cases.md (two case records)
2. A PR-body staleness finding is not cleared by a correction comment Recorded, narrowed to the anti-pattern address-every-comment.md
3. A self-review commit landed after the body was written Judged redundant, skipped ---
4a. Review cost data Folded into lesson 2 as affordability evidence address-every-comment.md
4b. find-near-duplicates.py defaults to the skills/*/SKILL.md corpus Already documented skills/find-overlap/SKILL.md lines 68 and 109

Lesson 1 --- placement rationale

fully-clean.md is auto-loaded (CLAUDE.md:443); review-verdict-pitfalls.md
and every .cases.md are not. So the operative rule goes in the auto-loaded
file and the measurements go in the case record.

It slots in as the third entry in an existing sequence, after "a review
comment's header SHA can be stale" and "that remedy assumes the run checked out
the PR head". The increment: the first two mislead toward staleness, which
invites a needless re-trigger; this field misleads toward currency, which is
the confident direction.

Lesson 3 --- why it was skipped

ardi.md already says to "treat any round that
changes the diff as expiring every figure the body already states, not only the
figure that round was about". A self-review commit is a round that changes the
diff, so the existing rule covers the trigger completely and prescribes an
identical remedy. Adding a sharpening would be the duplicate
challenge-redundant-content.md
warns about.

Lesson 4b --- why nothing was written

--corpus defaults to skills/*/SKILL.md, confirmed via --help.
skills/find-overlap/SKILL.md:68 already states it in prose, and line 109 shows
the --corpus 'memories/*.md' override. The session's error was not reading
that, which is not a corpus gap.

The brief's central claim did not reproduce, and I did not assert it

The brief stated that run 31354330266 had head_sha = a0ef37c2 while
pull_requests[0].head.sha read 7fe25776. My unauthenticated REST read
returned pull_requests: [] --- empty. Per the brief's own instruction not to
trust it, I did not record that claim.

Instead I derived a stronger, currently-reproducible finding that explains
both the brief's observation and the present emptiness: the field is a live
pointer resolved at read time, so it reports the PR's current head whatever
commit the run checked out, and the array empties once the PR closes.

#=> examined 60; non-empty 14; field==current head 14; run head_sha differs 8

The MCP actions_get get_workflow_run omits pull_requests entirely --- the
field the whole lesson is about --- so the measurement went through raw REST.

Round 1 review: finding Addressed

The review at 5bed1d61 returned Needs more work with one finding, and it
was correct. The case record asserted that run 31357711790's head_sha and
pull_requests[0].head.sha agree because the run was dispatched with --ref at
the PR branch, and that it therefore could not illustrate the two diverging.
Both halves were false, and my own commit falsified them. Verified live
rather than from the reviewer's quote:

#=> workflow_dispatch ums/pr1384-pull-requests-head-sha ede6b0a9 [(1388, '5bed1d61')]

--ref pins head_sha and does nothing to the other field, which re-resolves
on every read. Pushing 5bed1d61 --- the commit carrying the sentence --- is
what separated them.

Fixed on the merits in 19fc91b6, not by pasting the suggested wording. The
reviewer named one sentence; sweeping the passage found the code block's #=>
output was stale too, so both were corrected. Three things folded in:

  1. The run is now a second case record for the entry's own thesis. On ums: update_trigger needs a fresh run_once_at to re-arm a fired one-shot #1384
    the field pointed the confident direction and made a superseded verdict look
    current. Here it points the useful direction and correctly says the PR moved
    past the commit whose check went red. Same field, opposite usefulness,
    neither reading wrong --- which supports the thesis better than ums: update_trigger needs a fresh run_once_at to re-arm a fired one-shot #1384 alone.
  2. The general lesson, one artifact further in than PR-body staleness: a
    claim that two live fields agree has a short shelf life, and where the
    sentence is written into a commit, that commit is itself the falsifying
    event. Carried as a Do/Don't pair.
  3. The corrected reading is dated, since the field tracks the PR head and
    will differ on any later run --- applying the new rule to the sentence
    stating it.

fully-clean.md needed no change: its Do bullet already says to read the
field as a fact about the PR's current head and nothing else, which correctly
applied prevents the --ref misconception.

The red review / require-review on run 31357711790 is a cancelled run at
a superseded head
, not a failure with findings (completed / cancelled,
confirmed via the API). Already triaged in a PR comment; not re-dispatched.

Verification

All at 19fc91b6, after committing, with the three-dot range. Every
line-scoped scan was re-run after the commit, since a diff-editing pass expires
them.

Diff scope (git diff --stat origin/main...HEAD):

 shared/workflow/address-every-comment.md |  47 +++++++++++
 shared/workflow/fully-clean.cases.md     | 139 +++++++++++++++++++++++++++++++
 shared/workflow/fully-clean.md           |  42 ++++++++++
 3 files changed, 228 insertions(+)

Per-file line counts, git show origin/main:<f> against git show HEAD:<f>:

File Before After
shared/workflow/fully-clean.md 754 796
shared/workflow/fully-clean.cases.md 512 651
shared/workflow/address-every-comment.md 1172 1219

Added-line scan over git diff -U0 origin/main...HEAD:

added lines examined: 228 across 3 file(s)
banned punctuation hits: 0
column-1 bare '#' hits: 4

All 4 column-1 hits are #=> output lines inside fenced code blocks, which is
why markdownlint-cli2 reports clean below rather than MD018.

Checks:

Check Result
markdownlint-cli2 Linting: 499 files / Summary: 0 issues in 0 files
check-new-line-breaks.py @ pinned 209bfb76 No lines missing semantic breaks.
scripts/check-memory-file-size.py No memory file exceeds 1200 lines.
scripts/check-links.py Checked 1337 relative links across 473 markdown files / no broken links
scripts/validate-skills.py rc=0
scripts/check-vendored-drift.py rc=0

The line-break check ran against a gha clone pinned at
209bfb76bd412aa10d4d56a809d791a6dc8f10df, read out of validate.yml:127, so
it matches what CI runs rather than a shallow main.

Dupe-check --- normalized (re.sub(r"[`*_\s]+", " ", s).lower()) on
both needle and haystack, across 318 markdown files. Every needle is taken
from the diff's own added lines, so each is a known positive that demonstrates
the instrument fires. All six return 0 hits outside this PR's 3 files:
pull_requests[].head.sha, live pointer to the pull request,
the array empties when the PR closes,
the only surface separating the candidates,
a correction comment does not clear it,
the next reviewer re-reads the body.

Test suite: 12 suites run, 2 failed --- test_ai_session (4 cases) and
test_compare_shell_forms (1 case). Both reproduce with identical counts on a
clean origin/main, extracted via git archive origin/main | tar -x to a
scratch directory rather than mutating this worktree. Already tracked in #1382,
so no duplicate was filed. Neither suite exercises any of the three markdown
files this PR touches.

Advisory context budget: check-context-closure.py reports over budget both
before and after --- 1,192,175 bytes at origin/main against 1,196,891 at
ede6b0a9. CLAUDE.md is untouched at 82,438 characters (55.0% of the
harness's 150,000-char cap); both later commits touch only the
non-auto-loaded .cases.md, so they add nothing to the loaded budget.

claude added 2 commits August 10, 2026 04:53
Records a third way a review's commit attribution misleads, alongside the
stale caption and the workflow_dispatch head_sha already in fully-clean.md.
Unlike those two it points the confident direction: the field is a live
pointer to the PR, so it reports the current head whatever the run read.

Also names the near-miss for a PR-body staleness finding: a correction
comment leaves the body's false text standing, so the next reviewer
re-reads it and the finding survives.
@d-morrison
d-morrison marked this pull request as ready for review August 10, 2026 05:09
@d-morrison
d-morrison requested a balanced review from Copilot August 10, 2026 05:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

The case record stated two limits, one of which was that the 60-run sample
could not separate 'dispatch runs never populate pull_requests' from 'the
array empties once the PR closes'.

This PR's own review dispatch (run 31357711790) is a workflow_dispatch on an
open PR's branch returning a non-empty array, which falsifies the first
hypothesis outright and leaves closure as the explanation.

Copy link
Copy Markdown
Collaborator Author

Triage of the red review / require-review (check run 93360986277) -- no action needed, and deliberately not re-dispatching.

It belongs to run 31357711790, whose conclusion is cancelled rather than failure. That is a review which never reached a verdict, not one that found problems, so there is nothing in it to address. Per review-verdict-pitfalls, a cancelled review is the one case where retrying is the cause rather than the remedy: the reusable workflow's concurrency group is keyed on the PR number with cancel-in-progress, so dispatching again now would kill the run currently in flight and reproduce the same red check.

A fresh review is already running -- run 31357891930, review / claude-review in_progress since 05:13:48Z -- so the correct move is to let it finish.

Worth recording that this PR's own subject showed up in its diagnosis. The cancelled run's head_sha is ede6b0a9, while its pull_requests[0].head.sha reads 5bed1d61:

field value what it tracks
head_sha ede6b0a9 the commit the run was dispatched at
pull_requests[0].head.sha 5bed1d61 the PR's head now

Those disagree for exactly the reason this PR documents: the second field is a live pointer to the pull request rather than a record of what the run read. Here it happens to be the more useful of the two, since it correctly says the PR has moved past the commit whose check went red -- which is what makes the red check stale rather than actionable. On #1384 the same disagreement pointed the other way and made a superseded verdict look current. The field is not wrong in either case; it simply answers a different question than the one a reader asks of it, which is the entry's whole point.

Other checks at the current head are green: validate and new-line-breaks / check-new-line-breaks both success, twice each.

Continuing to watch until this merges or closes.


Generated by Claude Code

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Review finding on #1388: the case record asserted that run 31357711790's
head_sha and pull_requests[0].head.sha agree because the run was dispatched
with --ref at the PR branch, and that it therefore could not illustrate the
two diverging. Both halves were false.

--ref pins head_sha and does nothing to the other field, which re-resolves on
every read. Pushing 5bed1d6 -- the commit carrying the sentence -- is what
separated them. Live: head_sha ede6b0a, field 5bed1d6.

Also folds in the two things the finding hands over: the run is a second case
record showing the field's usefulness inverting while its behaviour stays
constant, and the general lesson that a claim two live fields agree has a
short shelf life when the commit stating it is itself the falsifying event.

The corrected reading is dated, since the field tracks the PR head and will
differ on any later run.

Copy link
Copy Markdown
Collaborator Author

ARD round 1 --- 1 finding, Addressed

Pushed 19fc91b6. Review dispatched at that head.

Addressed: the --ref agreement claim in fully-clean.cases.md

The finding is correct, and my own commit is what falsified the sentence.
Verified against the live API rather than the quoted values:

curl -sS "https://api.github.com/repos/Morrison-Lab/ai-config/actions/runs/31357711790" \
  | python3 -c "import json,sys; r=json.load(sys.stdin); print(r['event'], r['head_branch'], r['head_sha'][:8], [(p['number'], p['head']['sha'][:8]) for p in r['pull_requests']])"
#=> workflow_dispatch ums/pr1384-pull-requests-head-sha ede6b0a9 [(1388, '5bed1d61')]

Both halves of the sentence were wrong:

  1. It asserted a causal guarantee that does not exist. --ref pins
    head_sha and does nothing to pull_requests[].head.sha, which re-resolves
    on every read. What was true is narrower --- the two agreed at dispatch
    time, before any further push
    .
  2. "cannot also serve as an example of the two diverging" is false. That run
    is exactly such an example, and pushing 5bed1d61 --- the commit
    carrying the sentence --- is what made it one.

Fixed on the merits rather than by pasting the suggested wording. Two notes on
scope:

  • The reviewer named one sentence; the passage had two stale things. The
    code block's #=> output still read [(1388, 'ede6b0a9')]. Derived the site
    list rather than inheriting the enumeration, per
    address-every-comment,
    and corrected both.
  • fully-clean.md needed no change. Its Do bullet already says to read
    the field as a fact about the PR's current head and nothing else, which
    correctly applied prevents the --ref misconception. Adding a line would be
    the duplicate challenge-redundant-content warns about.

Folded in

The run is a second case record for the entry's own thesis. On #1384 the
field pointed the confident direction and made a superseded verdict look
current. Here it points the useful direction and correctly says the PR moved
past the commit whose check went red. Same field, opposite usefulness, neither
reading wrong --- which supports the thesis better than #1384 alone did.

The general lesson, carried as a Do/Don't pair: a claim that two live
fields agree is a state claim with a short shelf life, and where that sentence
is written into a commit, the commit is itself the event that can falsify it.
That is the PR-body staleness this PR already documents, one artifact further
in, where the falsifying push and the claim are the same commit.

The corrected reading is dated. The field tracks the PR head, so the block
now says it was read at 2026-08-10 when the head was 5bed1d61, and that the
second SHA is expected to differ on any later run. Applying the new rule to the
sentence stating it, so this correction does not go stale the same way.

Not re-triaged

The red review / require-review on run 31357711790 is completed /
cancelled at a superseded head, not a failure with findings. Confirmed via
the API, already triaged, not re-dispatched.

Re-derived at 19fc91b6

Every figure in the PR body was re-derived after committing, with the
three-dot range, and the body carries a Corrections to this body table
showing the movement across all three heads rather than silently overwriting.

Figure ede6b0a9 5bed1d61 19fc91b6
Added lines 162 184 228
fully-clean.cases.md 512 -> 585 512 -> 607 512 -> 651
Relative links 1336 1336 1337

markdownlint-cli2 0 issues, check-new-line-breaks clean at pinned
209bfb76, memory size gate clean, 0 broken links, 0 banned punctuation in 228
added lines.


Generated by Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


Code review

This round contains exactly one commit since the last review round (5bed1d6119fc91b6, "ums: correct a false agreement claim my own commit falsified"), touching only shared/workflow/fully-clean.cases.md (+50/-6). It's a direct, deliberate fix for the finding raised in my previous review round (comment):

a stale "the field agrees with head_sha" claim, falsified by the very commit that states it — shared/workflow/fully-clean.cases.md, lines 604-607.

The fix fully resolves that finding, and I re-verified it carefully rather than just reading the prose:

  • The false sentence ("the run was dispatched with --ref... so its head_sha and the field agree ... this run cannot also serve as an example of the two diverging") is now removed entirely — a corpus-wide search of the final file text for that wording (and the surrounding phrases) turns up zero hits.
  • The replacement text is honest about both halves being wrong, dates the one true observation ("read at 2026-08-10, when ums: pull_requests[].head.sha tracks the PR, not the run #1388's head was 5bed1d61"), and correctly separates the stable field (head_sha, pinned by --ref at dispatch) from the dynamic one (pull_requests[].head.sha, which re-resolves on every read).
  • I re-ran the exact curl | python3 one-liner quoted in the new text against the live run (31357711790). It behaves precisely as the new prose claims: head_sha is still frozen at ede6b0a9, while pull_requests[0].head.sha has moved on again — now to 19fc91b6 (this review's own head), confirming the text's prediction that "the second SHA is expected to differ on any later run."
  • I independently checked the new claim that require-review "went red" at the commit the field has moved past: check-runs for commit ede6b0a9 show review / require-review = completed/failure and review / claude-review = completed/cancelled — exactly matching the text's explanation of why the field pointing at a newer commit is the correct (non-misleading) reading in this second case.
  • The new [address-every-comment.md](address-every-comment.md) cross-reference resolves (verified target exists at this ref), and the git/curl commands and PR/run numbers cited are all real and match live state.
  • No ASCII-punctuation violations, no column-1 # issues outside fenced code blocks, and CI's new-line-breaks / check-new-line-breaks reports success on the current head.

No new issues were introduced by this fix. The one open finding from the prior round is closed.

Verdict

Ready for merge — the sole finding from the previous review round (stale, self-falsifying "field agrees with head_sha" claim) has been fixed in commit 19fc91b6 and independently re-verified against live GitHub state; no new issues found.

@github-actions

Copy link
Copy Markdown
Contributor

💰 Cost: $6.2852 (review) — run

@d-morrison
d-morrison merged commit f5aed92 into main Aug 10, 2026
8 checks passed
@d-morrison
d-morrison deleted the ums/pr1384-pull-requests-head-sha branch August 10, 2026 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ums: a run's pull_requests[].head.sha tracks the PR, not the run, so it can name a commit the run never read

3 participants