Skip to content

Add Wreq::Client#preconnect - #11

Open
zyc9012 wants to merge 4 commits into
mainfrom
client-preconnect
Open

zyc9012 wants to merge 4 commits into
mainfrom
client-preconnect

Conversation

@zyc9012

@zyc9012 zyc9012 commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Adds preconnect, which opens a connection to an origin ahead of time without sending a request. It does the DNS lookup, TCP connect, proxy tunnel and TLS handshake, then leaves the connection idle in the client's pool. The next request to that origin reuses it and skips connection setup.

client = Wreq::Client.new

client.preconnect("https://api.example.com")                 # => nil, raises Wreq::Error on failure
client.preconnect("https://api.example.com", proxy: proxy)   # per-request options supported

client.get("https://api.example.com/data")                   # reuses the warm connection

Behaviour

  • Matching the later request: preconnect builds its pool key the same way a real request does, so proxy: and emulation: (plus emulation_os: etc.) must match the later request for the connection to be reused. Other request options are ignored.
  • Timeout: timeout: limits how long the preconnect can take; otherwise the client's timeout and connect timeout apply.
  • Existing connections: if an idle connection to the origin already exists, it's reused and no new one is opened.
  • Proxies: works with both client-level and per-request proxies. HTTPS goes through a CONNECT tunnel with TLS inside it.
  • Threads and cancellation: the GVL is released during the call. cancel and Ruby thread interrupts abort it.
  • Connection lifetime: the connection is subject to the pool's 90s idle timeout and may also be closed by the server. If it's closed, the request opens a fresh connection. preconnect does nothing with pool_max_idle_per_host: 0.

Implementation

  • Vendored wreq (patches/0002-add-preconnect.patch):
    • Adds Client::preconnect(uri) and RequestBuilder::preconnect().
    • HttpClient's request setup is split into a shared prepare(), so preconnect and real requests compute the pool key the same way.
    • preconnect asks the pool for a connection and drops it straight away, which puts it back as idle. HTTP/1 connections are already awaited until ready before being handed out, so the dropped connection is reusable.
    • The middleware stack can't run without a real response, so preconnect skips it. It repeats the two checks that stack would apply before connecting: the scheme / https_only check and the total timeout.
  • Ruby binding: Client#preconnect(url, opts = {}) uses the same option handling and cancellation pattern as the request methods.
  • extconf.rb: patch detection now handles stacked patches. Before, each patch was reverse-checked on its own, so once 0002 touched src/client.rs (also changed by 0001), 0001 looked unapplied and the build aborted. Now it finds the newest patch that reverse-applies cleanly and treats it and every earlier patch as applied.

Testing

  • New test/preconnect_test.rb, mostly against a local keep-alive server. It checks that:
    • preconnect opens a connection without sending a request;
    • the next request reuses it;
    • repeated preconnects reuse the idle connection;
    • a per-request proxy is reused;
    • refused connections and bad schemes raise;
    • the TLS handshake is attempted and timeout: is enforced;
    • there is also a live HTTPS smoke test against httpbun.com.
  • The full suite (rake test) passes on a clean checkout of the branch, with both patches applied from scratch by extconf.rb.
  • Checked by hand: HTTPS through a local CONNECT proxy. With both a per-request and a client-level proxy, preconnect opened one tunnel and the next get reused it.
  • Rough latency against https://www.google.com/generate_204, 8 runs: median cold get about 650 ms, versus about 170 ms for a get after preconnect.

Not covered

  • No count option. Preconnecting several origins, or several connections to one origin, means calling preconnect from separate threads.
  • No automated test for HTTPS through a proxy or for SOCKS proxies. SOCKS takes the same connection path but hasn't been tested.

zyc9012 and others added 4 commits November 5, 2026 13:48
Detect already-applied patches by finding the newest patch that reverse-applies
cleanly, rather than reverse-checking each patch independently. The old check
broke once a later patch touched the same file as an earlier one, causing
extconf to re-apply the earlier patch and abort.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add Client::preconnect and RequestBuilder::preconnect, which establish a
pooled connection (TCP, proxy tunnel, TLS) without sending a request. The
connection descriptor is derived the same way as for a real request, so
per-request proxy and emulation options yield a reusable pool key. The
scheme/https_only check and total timeout normally applied by the middleware
stack are replicated, since preconnect bypasses it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Expose preconnect(url, opts = {}) to warm up a connection to an origin
without sending a request, so a later request can skip DNS, TCP, proxy
and TLS setup. Request options such as proxy, emulation and timeout are
honored so the warmed connection matches the later request. The call
releases the GVL and is interruptible via cancel and thread interrupts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant