Compress uploaded code with gzip - #26
Merged
Merged
Conversation
Wrap the tar upload in a Zlib::GzipWriter so scraper code is gzip-compressed before being posted to the server, cutting upload time and bandwidth. The gzip stream is finished (not closed) so the tempfile handle stays open and rewound for the upload, matching the existing tar trailer handling. The multipart content type becomes application/gzip. Note: morph.io's ApiController#run_remote currently unpacks the upload with Archive::Tar::Minitar.unpack on a plain tar, so it needs a matching change to decompress before this can be released. Resolves #1 Assisted-by: OpenCode:anthropic.claude-fable-5 Signed-off-by: Ben Fairless <ben@oaf.org.au>
benrfairless
marked this pull request as ready for review
August 24, 2026 03:32
benrfairless
enabled auto-merge
August 24, 2026 03:33
This was referenced Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The
morphcommand now gzip-compresses the tar of scraper code before uploading it.MorphCLI.create_tarwraps the tempfile in aZlib::GzipWriteraround the existing Minitar output, producing a.tar.gzinstead of a plain.tar. The gzip stream is finished (not closed) so the returned tempfile handle stays open and rewound for the upload, preserving the method's existing contract, and the multipart content type is nowapplication/gzip.Motivation and Context
Scraper code was uploaded as an uncompressed tar, wasting bandwidth and upload time. Text-heavy scraper directories compress very well, so this is a cheap win, especially on slow connections.
Resolves #1
Heads up: this needs a matching server-side change (openaustralia/morph#1519) before release. morph.io's
ApiController#run_remotecurrently unpacks the upload withArchive::Tar::Minitar.unpackon a plain tar, so the server needs to decompress (or detect gzip) first. A CLI release before that lands would breakmorphagainst production.How Has This Been Tested?
Test-first: added specs asserting the archive is gzip-compressed (magic bytes, size reduction on compressible input, and readable via
Zlib::GzipReader+Minitar::Input), confirmed they failed against the old implementation, then implemented the change and confirmed the full suite passes (bundle exec rspec, 30 examples, 0 failures, 97.6% line coverage).bundle exec rubocopis clean. Not yet run end-to-end against a server, since morph.io doesn't accept gzipped uploads yet (see above).Screenshots (if appropriate):
Types of Changes
Checklist:
AI assistance
Written with AI assistance (
Assisted-by: OpenCode:anthropic.claude-fable-5), reviewed by a human before submission.