Repository navigation
Conversation
The pinned wreq-util stops at Firefox 151, so define firefox_152 to firefox_158 locally on top of its Firefox 151 profile. Each overrides what differs on the wire: the cipher suites (154 dropped ECDHE_ECDSA_AES_256_CBC_SHA), the supported groups (156 dropped ffdhe2048/ffdhe3072), Accept-Language q=0.9 and HEADERS priority weight 42. JA3, JA4, the Akamai fingerprint, the ClientHello extensions and the HTTP/2 request match the official macOS release build of each version. Reproduce Firefox's HTTP/1.1 request for these profiles as well: Host first, header names spelled and ordered as on a navigation, and Connection: keep-alive. Headers the caller adds take Firefox's position on both protocols, and headers the profile does not know keep the caller's order and spelling. TE is left out for http1_only clients. header_order keeps precedence, and bundled profiles are unchanged on the wire. Add unit tests for the profile table and the header order, and local wire tests for the HTTP/1.1 request. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Wire-fingerprint correctness requires human verification of the pinned vendored wreq implementation and Firefox capture comparisons, both absent from the review snapshot.
No open issues 🎉
What changed in this PR
Adds Firefox 152–158 emulation to wreq-rb so callers can replay requests using newer Firefox profiles beyond those bundled with wreq-util.
Changes:
- Defines local profiles with release-specific TLS settings, updated headers, and HTTP/2 priority.
- Integrates Firefox header spelling and ordering into client, per-request, and batch requests while preserving explicit
header_orderprecedence. - Adds Rust and Ruby coverage and documents protocol limitations and profile-mixing behavior.
| File | Description |
|---|---|
test/emulation_test.rb |
Tests new profile selection, User-Agent values, platform overrides, and unknown releases. |
test/client_test.rb |
Tests HTTP/1.1 header spelling, ordering, overrides, and batch behavior. |
README.md |
Documents new profiles, header behavior, and known limitations. |
ext/wreq_rb/src/lib.rs |
Registers the Firefox profile module. |
ext/wreq_rb/src/firefox.rs |
Defines Firefox 152–158 profiles and profile-level unit tests. |
ext/wreq_rb/src/client.rs |
Integrates local profiles and tracks header ordering across request paths. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Farmed cookies for the Bing engine work only with Firefox 158 (https://github.com/serpapi/SerpApi/pull/10812). So it would also be useful to support replaying requests through wreq.
Fable 5.2 High:
Summary
Adds
firefox_152…firefox_158. The pinnedwreq-util(0.2.0) stops at Firefox 151, so these profiles are defined locally inext/wreq_rb/src/firefox.rson top of its Firefox 151 profile. A name thatwreq-utilknows still resolves to its profile, and a unit test fails once it ships one of these releases, so the local entry can be dropped then.What each profile sets
ffdhe2048/ffdhe3072ECDHE_ECDSA_AES_256_CBC_SHA)ffdhe2048/ffdhe3072ffdheAll seven also correct three things the bundled Firefox 151 profile gets wrong:
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHAis no longer offered,Accept-Languageisen-US,en;q=0.9, and the HTTP/2 HEADERS priority weight is 42.HTTP/1.1 request
These profiles also reproduce Firefox's HTTP/1.1 request, on every
emulation_os:Hostfirst, header names spelled and ordered as Firefox sends them on a navigation, andConnection: keep-alive(the HTTP/2 client strips it).Cookie,Referer,Origin,Content-Type, cache validators, …) take Firefox's position, over HTTP/2 as well.TE: trailersis left out forhttp1_onlyclients. It cannot be dropped when the protocol is negotiated.header_orderkeeps precedence over the profile's order.Bundled profiles are unchanged on the wire.
Verification
http1_only.Known gaps
None of these is visible in JA3, JA4 or the Akamai fingerprint; all come from the TLS / HTTP stack and also apply to the bundled Firefox profiles:
:path: /is HPACK-indexed.http1_onlynarrows ALPN tohttp/1.1, which Firefox never offers, so over TLS it is either an extraTEheader or a different ALPN.