Skip to content

fix(review): resolve escaped finding paths and drop findings on an excluded conventions file - #124

Merged
aliasunder merged 15 commits into
mainfrom
fix/resolve-escaped-finding-paths
Oct 1, 2026
Merged

aliasunder merged 15 commits into
mainfrom
fix/resolve-escaped-finding-paths

Conversation

@aliasunder

@aliasunder aliasunder commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

Summary

When a file path contains a double quote, the prompt writes the path attribute as path="docs/a"b.md". The system prompt tells the model to copy that attribute into a finding's file, and a model that does so writes docs/a"b.md. The unknown-file filter compared that spelling against the raw paths, which hold ", and dropped the finding as "file not in prompt context". Reviewers lost valid findings on related files, related docs, priority docs, and the conventions file whenever the path contained a quote.

The filter now keeps a finding whose file matches a known path only after each " is read as ", and rewrites file to that decoded spelling. A file that matches as written is never rewritten, so a path that literally contains " stays as the model wrote it.

A PR that changes the conventions file can also remove it from the review diff through diff_exclude_paths or linguist rules. The excluded-files note tells the model that file's content was not provided, and the system prompt says not to report findings on excluded paths, yet findings on the conventions file were still kept. The filter now accepts the conventions path only when diff exclusion did not remove it, so those findings are dropped like findings on any other excluded file. The conventions section is still sent unchanged as the review's rubric.

A finding on any diff-excluded file, the conventions file included, is now reported apart from unknown-file drops. It was logged as dropping finding: file not in prompt context and counted as an unknown file, which pointed an operator at context assembly although the path is listed in the excluded-files note. It now logs dropping finding: file excluded from the review diff and has its own count. Which findings are dropped is unchanged.

Changes

  • src/review/prompt.ts: adds unescapeAttributeValue beside escapeAttributeValue, decoding only the one entity the encoder writes. Each function's comment names the other so they change together.
  • src/review/filter-unknown-file-findings.ts: resolves each finding as known, unescaped, excluded, or unknown, with the written spelling checked first. Takes the diff-excluded paths, normalized like the known paths, and returns findings on them as droppedAsExcludedFile apart from droppedAsUnknownFile. The known check runs first, so the set of dropped findings does not change. Returns the rewrites as unescapedFileRewrites. The JSDoc states the new contract.
  • src/orchestrate.ts:
    • Logs each rewrite at debug level (resolved escaped finding file to a prompt path) with the phase, both spellings, and the line. The dropped count and the job summary still count only dropped findings.
    • The per-phase filter summary log is renamed from non-finding filter applied to model output to per-phase finding filters applied to model output, since it also reports unknown-file drops and cross-phase duplicates.
    • The pipeline builds one closure over the prompt paths and the diff-excluded paths and passes it to the per-phase filter step, which no longer forwards path lists. Each excluded-file drop logs dropping finding: file excluded from the review diff at warn with the same fields as the unknown-file warning (phase, file, line, category). The per-phase filter summary log and the job summary carry droppedAsExcludedFile.
    • The conventions path joins the filter's accepted paths only when it is not among the diff-excluded paths, compared after posix.normalize on both sides like the neighbouring priority-doc check. The conventions section of the prompt is unchanged.
  • src/review/review-summary.ts: the findings-pipeline table gains a Dropped as excluded file row after Dropped as unknown file.
  • action.yml and README findings_count: the removed-findings list names findings on files diff exclusion removed.
  • README.md:
    • Unknown-file filter: findings on excluded files are reported as excluded-file drops, and the drop bullet names both warnings and both job-summary rows. How it works step 6 names both drop kinds.
    • Unknown-file filter: lists what the model was given (diff-header paths, changed files, import-traced related files, mention-matched and priority docs, the conventions file), states that excluded files are listed by path at the end of the diff with their content withheld, so findings on them are dropped (an excluded conventions file is still sent as the review's conventions, but findings on it are dropped too), says findings on renamed-from and deleted paths post as standalone comments, states what normalization ignores and that matching is case-sensitive, and adds the decoding rule with its log line.
    • The job-summary bullet and the Non-finding filter section say the job summary shows each filter step's drop count, and use the renamed log message.
    • Both filter sections use one order phrase: each filter runs before cross-phase and cross-run deduplication, the severity threshold, and the cap.
    • How it works: the prompt-tag step, the within-run overlap rule (including which finding a same-severity tie keeps), and the review body are described in plain terms.
    • The beyond-diff bullet says those findings are on files the model was given.
    • The conventions_budget_tokens row splits its reporting conditions into two sentences.
  • Tests: new filter cases for the rewrite, a decode that keeps the rest of the written spelling, raw-wins, and neither-spelling-known. A seam test renders a path holding & and " through buildUserPrompt and feeds the attribute back as file. Each matching-rule filter test seeds a finding the rule keeps and one it drops, and new cases cover a leading or trailing / on either side of the match. One orchestrate test asserts the posted path, the debug log, the drop warning, and the filter counts. A second shows a finding filed on the escaped spelling is deduplicated against a prior anchor on the decoded path. Two more run the real prompt builder on a PR that changes the conventions file: when the file is diff-excluded, a finding on it is dropped with the warning, nothing posts, and the prompt still carries the conventions section; the same holds when conventions_file is spelled ./AGENTS.md; when it is not excluded, the finding posts inline. The excluded-file tests assert the excluded-file warning and no unknown-file warning; the plain excluded-file test also asserts the filter summary log and both job-summary rows. New filter cases cover the excluded bucket beside kept and unknown findings, normalized excluded paths, an excluded path reached through " (not rewritten), and a path both known and excluded. A job-summary test pins the whole findings-pipeline section with the new row.

Testing

  • npm test: 829 passed (26 files). npm run lint and npm run build are clean.
  • Mutation checks, each restored afterwards:
    • An identity decoder failed 6 tests (4 filter, 2 orchestrate), each because the escaped finding was dropped as unknown.
    • Keeping a decoded match without rewriting its file failed the orchestrate dedup test, because the finding was posted again instead of matching the prior anchor.
    • Changing the encoder to " failed the seam test and the two prompt tests that pin the escaped attribute.
    • An encoder that also escapes & as & failed the seam test, whose path holds both & and ".
    • A decoder that also decodes & failed the "decodes only "" test.
    • Checking the decoded spelling before the written one failed the raw-wins test only.
    • A filter that keeps every finding, and one that drops every finding, each failed every matching-rule test individually.
    • Removing the diff-exclusion check on the conventions path failed the excluded-conventions test only, because the finding on the conventions file was kept and posted.
    • Comparing the configured conventions path without normalizing it failed the ./AGENTS.md test only.
    • Routing excluded-path findings back to the unknown bucket failed 7 tests (4 filter, 3 orchestrate excluded-file tests), each because the drop landed in droppedAsUnknownFile with the unknown-file warning.

🤖 Generated with Claude Code

aliasunder and others added 3 commits September 30, 2026 18:07
The prompt writes a double quote in a path attribute as ", and the
model copies that attribute into a finding's file. The unknown-file filter
compared that spelling against the raw paths and dropped the finding. It
now keeps a finding whose file matches only after decoding ", rewrites
file to the decoded path, and logs each rewrite at debug level. A path that
matches as written is never rewritten.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d-diff tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ship-Check: pr-review · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/review/prompt.ts
@umm-actually

umm-actually Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

umm-actually re-reviewed at d8f0e40

1 new finding(s) posted (8 tracked finding(s) across all runs).


umm-actually · deepseek/deepseek-v4.1-flash

aliasunder and others added 2 commits September 30, 2026 18:21
…llers

- Name the normalization the filter applies, state that kept findings keep
  the model's spelling, and give the real reason the written spelling is
  checked first.
- State why only a double quote is escaped in path attributes, and link the
  file-anchoring instruction to the decode in the filter.
- Destructure the dropped findings under their own name so the returned
  count no longer shares it, and state why drops warn while rewrites log at
  debug.
- Make the prompt-path comment true for renamed paths and note which
  normalization the filter applies; restate the deadline-coverage comment.
- Drop a length check that the following .some() already covers.
- Rewrite the README escaped-path bullet to lead with the scenario.

Ship-Check: code-quality · claude-opus-5-5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each matching-rule test now seeds a finding the rule keeps and one it
drops, so a filter that keeps or drops everything fails that test on its
own.

- The "decodes only &quot;" test carries an "&amp;" in its path, so a
  decoder that decodes other entities fails it.
- The rendered-attribute test uses a path with "&" and '"', so an
  encoder that starts escaping "&" without a matching decoder fails it.
- The attribute regex is scoped to the one test that reads it.

Ship-Check: test-audit · claude-opus-5-5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/orchestrate.ts
… docs

Git C-quotes a path that contains a double quote, and parse-diff keeps
the backslash escape, so a diff header does not print such a path
unchanged. The README now says the header prints the diff's own
spelling, which the filter matches as written.

conventionsRenderInFull promised exactly one full copy, but a truncated
conventions section with no budget left for the changed-file block
sends none. It now promises no more than one.

Ship-Check: bug-check · claude-opus-5-5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/review/__tests__/filter-unknown-file-findings.test.ts Outdated
… for what it reports

Ship-Check: triage · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread README.md Outdated
Comment thread src/review/__tests__/filter-unknown-file-findings.test.ts
aliasunder and others added 4 commits September 30, 2026 18:39
…xception

The within-run overlap step also drops one of two same-severity findings,
keeping the earlier line. The conventions file is read even when
diff_exclude_paths or linguist rules exclude it, so findings on it are kept.

Ship-Check: code-quality · claude-opus-5-5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ship-Check: triage · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each matching-rule filter test now also seeds a finding the rule drops, so
a filter that keeps everything fails it. New cases cover a leading or
trailing slash on either side of the match, and an orchestrate test shows
a finding filed on an escaped path is deduplicated against a prior anchor
on the decoded path.

The README's unknown-file filter section now says excluded files are
listed by path at the end of the diff with their content withheld, and
that the conventions file is sent even when excluded.

Ship-Check: pr-monitor · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The conventions path joins the unknown-file filter's accepted paths only
when diff exclusion did not remove the file. The conventions section is
still sent as the rubric. README states the new behaviour and uses one
order phrase for both filters.

Ship-Check: triage · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aliasunder aliasunder changed the title fix(review): keep findings whose file copies an escaped path attribute fix(review): resolve escaped finding paths and drop findings on an excluded conventions file Sep 30, 2026
Ship-Check: pr-monitor · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/orchestrate.ts
aliasunder and others added 2 commits September 30, 2026 19:03
…rops

A finding on a diff-excluded file, the conventions file included, was
dropped with the unknown-file warning and counted as an unknown file.
That pointed an operator at context assembly, though the path is listed
in the excluded-files note. The filter now returns those findings in their
own bucket, logged as `dropping finding: file excluded from the review
diff` and counted in a `Dropped as excluded file` job-summary row and the
per-phase filter log. Which findings are dropped is unchanged.

Ship-Check: pr-monitor · claude-opus-5-5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/__tests__/orchestrate.test.ts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread README.md
@aliasunder
aliasunder merged commit dceb80c into main Oct 1, 2026
9 checks passed
@aliasunder
aliasunder deleted the fix/resolve-escaped-finding-paths branch October 1, 2026 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant