Repository navigation
Conversation
trusche
left a comment
There was a problem hiding this comment.
Thanks @strzibny, another fully agentic review from me here. Sorry, you're my guinea pig for this. Please dismiss if not accurate, but let me know so I can refine (and pick another victim).
Claude Opus 5.5, effort: high. AI review: I read the full diff and the vendored wreq source. I built the extension, ran test/client_test.rb, and probed the edge cases with local servers and a local proxy.
Request changes. The feature works for the normal case. The README explanation of ports is correct, and I checked it against wreq's code. But the PR says this option is for use after a DNS security check. In three cases the client does not use the pinned IP. It does a normal DNS lookup and gives no error:
- when a proxy is used, including the system proxy from
HTTP_PROXY, which wreq turns on by default - when the URL host has different upper and lower case than the key
- after a redirect to another host
At a minimum, the README must describe these cases. I also found one small test bug. Details are in the inline comments. Three comments are blocking and the rest are optional cleanup.
QA: test/client_test.rb passes: 16 runs, 0 failures. CI is green. When you run test_resolve_accepts_multiple_addresses alone, it fails with NameError. I ran the proxy, case, redirect and input checks locally with local servers. None of the checks used a live search.
| builder = builder.local_address(addr); | ||
| } | ||
|
|
||
| if let Some(resolve_hash) = hash_get_hash(&opts, "resolve")? { |
There was a problem hiding this comment.
If the client uses a proxy, it ignores resolve: and gives no error. The PR description says this option is for use after a DNS security check, so this matters.
I tested this with a local HTTP proxy. In all 3 cases (3 of 3), the proxy received GET http://wreq-resolve.invalid/ HTTP/1.1. The proxy got the host name, not the pinned IP, so the proxy does its own DNS lookup:
- client option
proxy: "http://127.0.0.1:<port>" - per-request
proxy:(client.rs:695) HTTP_PROXYset in the environment, and nono_proxy: true. wreq uses the system proxy by default (vendor/wreq/src/client.rs:286and:484).
SOCKS5h has the same behavior (vendor/wreq/src/conn/proxy/socks.rs:190). The tests do not find this because every new test sets no_proxy: true (test/client_test.rb:160).
Please add this to the README. If you use resolve: for security, you must also set no_proxy: true. Another option is to raise an error when you get resolve: and proxy: together.
| let mut overrides: Vec<(String, Vec<SocketAddr>)> = Vec::new(); | ||
| hash.foreach(|k: Value, v: Value| { | ||
| let ruby = unsafe { Ruby::get_unchecked() }; | ||
| let domain: String = if k.is_kind_of(ruby.class_symbol()) { |
There was a problem hiding this comment.
Blocking (for the security use case). wreq compares the key with the host in the URL exactly. It does not ignore upper and lower case (vendor/wreq/src/dns/resolve.rs:144). If the host does not match, the client uses normal DNS and gives no error:
| key | URL | result |
|---|---|---|
wreq-resolve.invalid |
http://wreq-resolve.invalid/ |
200 (control) |
wreq-resolve.invalid |
http://WREQ-Resolve.invalid/ |
normal DNS |
wreq-resolve.invalid |
http://wreq-resolve.invalid./ |
normal DNS |
a.test:443 (curl style) |
(any) | accepted, can never match |
For an SSRF check, the attacker controls the URL. The attacker can write the host in mixed case, and then the client does not use the pinned IP. If you only make the keys lowercase, the problem stays, because the URL host keeps its case.
The minimum fix: write in the README that the key must equal the URL host exactly. A better fix: make the lookup ignore case.
| end | ||
|
|
||
| def free_port | ||
| server = TCPServer.new("127.0.0.1", 0) |
There was a problem hiding this comment.
Blocking (small). free_port uses TCPServer before require "socket" runs on line 175. When you run this test alone, it fails:
$ bundle exec ruby -Ilib -Itest test/client_test.rb -n test_resolve_accepts_multiple_addresses
NameError: uninitialized constant ClientTest::TCPServer
The full file passes only because an earlier test loads socket. Put require "socket" once at the top of the file, as test/response_test.rb:5 does. Then remove the require lines in the helpers.
| on port 8080. | ||
|
|
||
| An explicit port in the request URL always wins over a port in the override. | ||
| Domains without an entry resolve normally. |
There was a problem hiding this comment.
Please add one sentence here: a redirect to a different host uses normal DNS. I tested a 302 to other.invalid, and the client did a DNS lookup for it. This behavior is expected. But a user who uses resolve: for security must know that redirects can go to hosts that nobody checked.
| def test_resolve_leaves_other_domains_alone | ||
| client = resolving_client("wreq-resolve.invalid" => "127.0.0.1") | ||
| assert_equal 200, client.get("https://httpbun.com/get").status | ||
| end |
There was a problem hiding this comment.
This test calls httpbun.com on the internet. It still passes when resolve: does nothing, and it fails when the computer is offline. Please use the local server and a host that has no entry, for example http://localhost:#{port}/. Then the test shows that resolve: does not change other hosts.
|
|
||
| # Spins up a local HTTP/1.1 server, yields its port, and returns the Host | ||
| # header of the request it received. | ||
| def with_local_http_server |
There was a problem hiding this comment.
This helper is almost the same as capture_wire_headers (line 231): it accepts one connection, reads the headers and sends a 200. Please keep one helper that returns the headers, and use it for both.
| k.funcall("to_s", ())? | ||
| } else { | ||
| TryConvert::try_convert(k)? | ||
| }; |
There was a problem hiding this comment.
The code that changes a symbol or a string key into a String now exists four times: here, and at lines 815, 832 and 904. Please add one helper (for example key_to_string) and call it in all four places. A follow-up PR is fine for this.
| let mut addrs: Vec<SocketAddr> = Vec::new(); | ||
| if v.is_kind_of(ruby.class_array()) { | ||
| for elem in RArray::try_convert(v)?.into_iter() { | ||
| let addr_str: String = TryConvert::try_convert(elem)?; | ||
| addrs.push(parse_socket_addr(&addr_str)?); | ||
| } | ||
| } else { | ||
| let addr_str: String = TryConvert::try_convert(v)?; | ||
| addrs.push(parse_socket_addr(&addr_str)?); | ||
| } |
There was a problem hiding this comment.
The array branch and the single-string branch both convert to String and call parse_socket_addr. First make a Vec<String> (one item, or the items of the array). Then parse in one loop.
| /// | ||
| /// An IPv6 address with a port must be bracketed: "::1:8080" is a valid IPv6 | ||
| /// address in its own right and parses as one, not as "::1" on port 8080. | ||
| fn parse_socket_addr(s: &str) -> Result<SocketAddr, magnus::Error> { |
There was a problem hiding this comment.
The name parse_socket_addr sounds like a general helper, but the error message on line 862 is only about resolve. Maybe rename it to something like parse_resolve_address. Also add the domain to the error message (invalid resolve address for 'example.com': 'not-an-ip'). When the hash has many entries, the user can then find the bad one.
| ## Overriding DNS | ||
|
|
||
| Pass `resolve` to point specific domains at IP addresses of your choice, | ||
| bypassing DNS for them (the equivalent of curl's `--resolve`). Useful for |
There was a problem hiding this comment.
"the equivalent of curl's --resolve" is not exactly true. curl's option applies to one host and port pair, and this option applies to every port of the domain. Please change it to "similar to".
Look up resolve: entries through a custom resolver that ignores case and a trailing dot. wreq's built-in overrides compare the URL host byte for byte, so a differently spelled host silently fell back to real DNS. Reject keys that can never match (host:port, URLs, IP addresses, non-ASCII) or that collide after normalization, and name the domain in address errors. Document how proxies and redirects interact with resolve:. Make the resolve tests local and runnable on their own, merge the duplicated test server helpers, and share the symbol/string key conversion.
|
@trusche I think it was warranted, can you recheck again? |
|
Thanks @trusche! I checked every comment and all of them held up, so there was nothing to dismiss. Fixes are in 9737a7b. The three blocking ones:
The optional ones are all done too:
CI is green on Ruby 2.7 and 3.4. |
Support overriding DNS resolution that wreq itself already supports.
Example:
This can be helpful when DNS is already resolved as part of a security check.