Fix sourcehunt preprocessor losing cloned tree to GC - #134
Merged
Conversation
Preprocessor._clone_or_use_local stashed the cloner SourceAnalyzer on self._analyzer, which Preprocessor.run() then rebinds to a fresh analyzer configured for the target repo. That dropped the only strong reference to the cloner, so its tempfile.TemporaryDirectory finalizer ran and rmtree'd the freshly cloned tree before the file walk hit it — producing zero file_targets for any git-URL target. Store the cloner on self._cloner so its lifetime is bound to the Preprocessor. Regression test walks a real local git repo through the git-URL code path and asserts file_targets is populated. Also in this change: - SourceAnalyzer.clone() accepts a full 40-char commit SHA in `branch`. Shallow `git clone --depth 1 --branch <ref>` rejects SHAs, so SHA refs fall through to a blobless full clone + `git checkout <sha>`. - Log the raw machine-fd sourcehunt request payload (routing redacted) so bridge-side misconfigurations surface in kubectl logs.
jorge-garcia-le
approved these changes
Aug 6, 2026
Sourcehunt only ever passes branch names (CLI --branch, TargetConfig, CampaignTargetConfig all default to 'main'; the one eval that pins to a commit does its own clone + checkout and hands sourcehunt --local-path, bypassing this method). The SHA-detection code path had no caller.
Full 40-char SHAs are now valid --branch values on sourcehunt requests (cwpro bridge sends them). git clone --depth 1 --branch <ref> rejects SHAs, so a SHA branch triggers a blobless full clone followed by git checkout <sha>. Names keep the fast shallow-clone path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Preprocessor._clone_or_use_localstashed the clonerSourceAnalyzeronself._analyzer, whichPreprocessor.run()then rebinds to a fresh analyzer. The old cloner refcount hit zero, itstempfile.TemporaryDirectoryfinalizer ran, and the freshly cloned tree wasrmtree'd before the file walk hit it — so any git-URL sourcehunt target produced zerofile_targets. Fix binds the cloner toself._clonerso its lifetime tracks thePreprocessor.SourceAnalyzer.clone()accepts a full 40-char commit SHA inbranch(cwpro bridge sends these).git clone --depth 1 --branch <ref>rejects SHAs, so a SHA branch triggers a blobless full clone +git checkout <sha>; names keep the fast shallow-clone path.kubectl logs.Test plan
python -m pytest tests/test_sourcehunt_preprocessor.py::TestPreprocessorClonePath -v—test_clone_survives_analyzer_rebindwalks a real local git repo through the git-URL code path and assertsfile_targetsis populated.f6e2ca4388a531c4a5996fc8043c38f1cf57897fand confirm the vulnerable-commit tree is scanned.