compiler: validate $ref URLs in fetchFile to prevent SSRF

[evilgensec]

Summary

fetchFile() in compiler/reader.go called http.Get() with any URL derived from a $ref value in a user-supplied OpenAPI specification, with no validation of the URL scheme or destination host. An attacker who can supply a crafted spec to a gnostic-based CI pipeline or tool can cause gnostic to make HTTP requests to:

• Cloud instance metadata endpoints (e.g. http://169.254.169.254/latest/meta-data/iam/security-credentials/)
• Internal network services (RFC 1918 ranges)
• Loopback addresses
• Non-HTTP URI schemes

Root Cause

readBytesForFile() detects whether a $ref value is a URL by checking url.Parse().Scheme != "" and, if true, passes the raw string directly to fetchFile(). fetchFile() then calls http.Get(fileurl) with no further validation.

Fix

This commit adds validateFetchURL() which is called inside fetchFile() before the HTTP request is issued. It enforces:

1. Scheme allowlist — only http and https are permitted; any other scheme (e.g. file://, ftp://) is rejected with an error.
2. Private/reserved IP blocklist — if the URL host is a literal IP address, it is checked against:
   • 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 (RFC 1918)
   • 127.0.0.0/8 (loopback)
   • 169.254.0.0/16 (link-local / cloud IMDS)
   • 100.64.0.0/10 (shared address space, RFC 6598)
   • IPv6 equivalents (::1/128, fc00::/7, fe80::/10)

Example — before patch

# malicious_spec.yaml
paths:
  /foo:
    $ref: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"

$ gnostic --pb-out=. malicious_spec.yaml
# Issues HTTP GET to cloud metadata — IAM credentials returned and cached

Example — after patch

$ gnostic --pb-out=. malicious_spec.yaml
Error: URL "http://169.254.169.254/latest/meta-data/iam/security-credentials/" targets a
private or reserved IP address and cannot be fetched

Notes

• Hostname-based private network access (where a public DNS name resolves to a private IP) is not addressed by this patch; that requires post-resolution checking and is a follow-up hardening step.
• The fix does not affect local file resolution (readBytesForFile handles the non-URL path separately and is unaffected).

Fixes SSRF vulnerability reported via Google VRP (Issue 495622065).

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[evilgensec]

Please note that the vulnerability was reported through Google VRP and they redirected the issue to be forwarded here. Fixes SSRF vulnerability reported via Google VRP (Issue 495622065). @timburks @precision @rspier @morphar