Tell the user when a scraper runs but outputs nothing - #27
Merged
benrfairless merged 3 commits intoAug 24, 2026
Merged
Conversation
Previously a scraper that wrote nothing to stdout or stderr finished silently, leaving no way to tell a successful quiet run from one that did nothing at all. Track which streams the run log lines came from and, when none of them were the scraper's own stdout or stderr, print a confirmation that the run succeeded. Lines on morph's internalout stream don't count as scraper output. The streaming upload moves from execute into a new MorphCLI.run to keep the method within the RuboCop metrics backlog; the module length allowance in .rubocop_todo.yml grows to cover the new lines. Resolves #7 Assisted-by: OpenCode:anthropic.claude-fable-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
benrfairless
marked this pull request as ready for review
August 24, 2026 03:42
benrfairless
enabled auto-merge
August 24, 2026 03:42
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
…es-n Brings the gzip upload and the explicit data.sqlite handling from main into the no-output warning branch. The streaming upload keeps its extraction into MorphCLI.run and drops the local timeout variable in favour of main's inline env_config.fetch(:timeout, 600). Assisted-by: OpenCode:claude-opus-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
The assertions used req.body.include?("data.sqlite") which worked
when the upload was an uncompressed tar (the filename appeared as
plain text in the tar header). After gzip compression was added the
string no longer appears in the raw multipart body.
Replace with tar_entry_names(), which locates the gzip magic bytes
in the multipart body, decompresses the tar, and returns the list of
entry names to check against.
Assisted-by: Claude Code:claude-sonnet-4-6
ianheggie-oaf
approved these changes
Aug 24, 2026
ianheggie-oaf
left a comment
ianheggie-oaf
left a comment
Member
There was a problem hiding this comment.
Approved - LGTM - Worked through sentry's suggestion as per comment
benrfairless
deleted the
feature/7-warn-when-scraper-produces-no-output
branch
August 24, 2026 05:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When a scraper run finishes without writing anything to its own stdout or stderr,
morphnow printsScraper didn't output anything, but it ran successfully.so a quiet successful run is distinguishable from a run that did nothing. Lines on morph'sinternaloutstream (e.g. "Injecting configuration...") don't count as scraper output, and the message is skipped entirely on failed runs because the error middleware raises before it's reached. To keepMorphCLI.executewithin the RuboCop metrics backlog, the streaming upload moved into a newMorphCLI.runhelper andMorphCLI.lognow returns the name of the stream it wrote; theMetrics/ModuleLengthallowance in.rubocop_todo.ymlgot a single targeted bump (114 to 124) for the genuinely new code rather than a regeneration.Motivation and Context
Resolves #7. A scraper that produced no output previously finished in total silence after the
Uploading...line, giving the user no indication the run had actually happened and succeeded.How Has This Been Tested?
Three new specs cover the behaviour, written first and verified failing before the fix: the message appears for a run with only
internaloutoutput, and is suppressed when the scraper writes to stdout or to stderr.bundle exec rspecpasses (32 examples, 0 failures, coverage 97.7% against the 90% minimum) andbundle exec rubocopis clean.Screenshots (if appropriate):
Types of Changes
Checklist:
AI disclosure
This change was written with AI assistance: OpenCode with model anthropic.claude-fable-5 (see the
Assisted-bycommit trailer). The change was made against the repository's own AGENTS.md and the org contributing guide.