Autocollapse generated files#19
Conversation
Adds a batched `git check-attr --stdin` call covering the four Linguist-style attributes we care about (linguist-generated, linguist-vendored, linguist-documentation, binary) and surfaces the results under DiffResponse.attrs. Graceful empty result when git is unavailable (pure-jj repo with no .git/).
Ports the name-based rules from generated.rb plus the full regex lists from vendor.yml and documentation.yml (pinned to linguist @ 537297cdae3ab05f8d5dd1c03627a5bd73707b19) into pure functions that run over file paths. Adds a small merge helper: .gitattributes decisions from M1 win over path rules on a per-attribute basis, so an explicit `linguist-generated=unset` still lets documentation detection run.
Adds a batched git cat-file --batch reader that pulls the blob content for each file in the diff, then runs the content-based subset of generated.rb (source maps, protobuf variants, Haxe, Cython, gRPC, Sorbet RBI, ~30 more) to flag files that aren't caught by path rules. Content lookups only run for files where `generated` isn't already decided — check-attr wins, path rules win, content fills in. Also adds the sqlx-query path rule that was missing from M2.
Seeds the initial collapsed state for any file flagged by the new attrs field (matching how viewed files already collapse), and drops those files from the ProgressBar denominator so the count reflects only the files the reviewer is expected to look at. Documentation files are not auto-collapsed — they're usually worth reading, so only generated/vendored/binary collapse.
Pass the file attrs down to FileCard and render a small pill for each attribute that's set, right before the +/- stats in the file header. Documentation files only ever get badged; the other three also auto-collapse (per the previous commit).
|
Thank you, I'm very interested in this. I see linguist is in ruby, that's why we can't use it directly. The change is a lot of lines but it's mostly just data. I'm still concerned about the ratio of length to value here — this more than doubles the amount of server code, I think. I wonder if it might be better to build in a way to let the user specify auto-collapse patterns, that way we wouldn't have to build in general-purpose detection. Maybe it could go in a special JSON config file outside the repo. A map of project directories to arrays of glob patterns? |
|
Yeah, I should definitely have split apart the git attributes change from the default rules; I can break this up into a couple of PRs, although probably not in the next day or so. I do agree that it's a lot of additional code, although it is also likely to be very stable -- I don't think linguist changes much at all. I see there is what seems to be a full node port at https://www.npmjs.com/package/linguist-js if you wanted to defer to a dependency, but I can't vouch for the quality. Or we could just not land the autodetection. |
2 participants
Closes #18.
Hi, this PR ports linguist's logic for detecting generated files from ruby, and adds detection of the
linguist-generatedgit attribute (among others, likelinguist-vendor). It then uses those to auto-collapse generated files in frontend, matching the behaviour with github's code review tool. Generated files also won't be counted in the the "N of M files viewed" tally.It looks like this:
Responsible disclosure, I used Claude Code (Opus 4.7) to generate this PR after chatting with the model for a while about the approach. I've reviewed the code myself and think it's fine, and happy to finesse it more. The auto-detection code is... extensive... but is mostly just a bunch of regexes run against file names and content, as well as some basic heuristics to try to detect e.g. minified JS files.