Repository navigation
Bound the bot's --feeds-url download - #138
Merged
Merged
Conversation
downloadFeeds called fetch(url) with no deadline and no size limit, so a slow or huge response held the bot until the job timeout. The refresh side already had a 30 second deadline and an 8 MiB cap in refresh/fetch.mjs, but refresh/ is not in the npm package, so the bot could not use it. The bounded reader moves unchanged to lib/fetch-bounded.mjs, and refresh/fetch.mjs re-exports it, so refresh behavior is byte-identical. downloadFeeds now reads through fetchBoundedBody. An adapter keeps the legacy injected response shapes (text(), string body, data, plain object) working, and each of them is measured against the cap. A bound violation is a download failure like any other, so vendored fallback and its warning are unchanged. Fixes #134. Implemented by an Opus subagent; reviewed by Claude.
readBoundedBody and readErrorDetail added chunk.byteLength, which a string chunk does not have, so the count became NaN. A body that streamed strings then skipped the 8 MiB cap, and an error body that streamed strings was read to the end. Both now count the bytes of Buffer.from(chunk), the same buffer they already kept. For a Uint8Array chunk that buffer has the chunk's byteLength, so refresh, which only sees Uint8Array chunks, counts exactly as before. A text() result that is a Buffer or Uint8Array is now decoded as UTF-8. A Buffer already worked through String() and on main; a Uint8Array stringified to a comma list and failed as invalid JSON. The instanceof Response pass-through is removed. The duck-typed path gives a real Response the same status, ok, headers, and body, so the branch could not change any result. New tests cover byte and string text() results, duck-typed and string-streamed bodies above and below the cap, a string-streamed error body read only to the detail cap, and legacy error shapes with and without detail. Implemented by an Opus subagent; reviewed by an Opus adversarial review.
A late-pass review found that dropping the ok !== false check from the adapter passed the whole suite, so a 302 Response or a legacy ok: false shape with a feed body would have been accepted. Both now have a test, and both fail with the check removed. Implemented by Claude; reviewed by an Opus late-pass review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #134.
Summary
refresh/fetch.mjstolib/fetch-bounded.mjs, which is in the npm package.refresh/fetch.mjsre-exports it.downloadFeedsreads--feeds-urlresponses throughfetchBoundedBody. An adapter keeps the legacy injected shapes working, and the cap applies to each of them.feeds.allow_vendored_fallbackthe bot falls back with the same warning; without it, the bot stops.What a bot user can notice
URL returned HTTP 500becomes--feeds-url fetch failed for URL: HTTP 500 - <detail>.Accept-Language: enand an abort signal.Tests
main.npm testpasses. Refresh test output is unchanged.🤖 Generated with Claude Code