Allow Gemfury's path-style S3 downloads for Gemfury jobs

[v-robaiken]

Fixes #308

Problem

#288 lets jobs with a *.fury.io credential reach Gemfury's bucket through its Transfer Acceleration hosts. Some Gemfury accounts are redirected to the same bucket as a path-style URL on the shared S3 endpoint instead:

302 https://pypi.fury.io/***/<account>/-/ver_xxx/<pkg>.whl
GET https://s3.amazonaws.com/gemfury/gems/<id>/<file>?X-Amz-...
* egress not allowlisted s3.amazonaws.com

Allowing s3.amazonaws.com as a host is not an option. With path-style addressing, every bucket in S3 lives on that host, so the job could send data to any bucket.

Fix

Add an optional derived_paths list to registry_redirect_derivations. Each entry is an exact host plus a literal path prefix. It is allowed only when:

• the job holds a credential for the matching registry, as with derived;
• the host matches exactly, with no subdomains, regional endpoints or globs;
• the method is GET or HEAD, since these are pre-signed downloads;
• both the escaped and decoded paths start with the prefix, so %2F and %2e tricks can't move the request into another bucket;
• the path has no . or .. segments and no backslashes.

At startup, prefixes must be a literal, normalized /segment/ ending in a slash, so /gemfury can't also match /gemfury-attacker.

Gemfury now uses this rule:

  - credential_host: .fury.io
    derived: [...]
    derived_paths:
      - host: s3.amazonaws.com
        path_prefix: /gemfury/

The existing host-only matching is unchanged; this adds one more way for a request to be allowed. I've updated the allowlist skill doc with when to use derived_paths.

Tests

• Allowed: Gemfury path-style downloads with a port, a trailing dot, an encoded +, and HEAD.
• Blocked:
  • the bare host;
  • other buckets and look-alike buckets (gemfuryx, gemfury-attacker, GEMFURY);
  • dot segments, including encoded %2e%2e;
  • encoded slashes (%2F) and backslashes (%5C);
  • subdomains, regional endpoints and suffix look-alikes of the host;
  • PUT, POST and DELETE;
  • jobs without a Gemfury credential.
• Startup validation rejects bad hosts and prefixes.
• I checked that the tests catch problems by removing the escaped-path check and the dot-segment check in turn; each removal makes the tests fail.

[Copilot]

🟡 Changes recommended

Startup validation currently accepts host and path-prefix configurations that can never match requests.

2 open findings

• Reject non-normalized hostnames during validation · New
• Reject blacklist prefixes that mismatch escaped URL paths · New

What changed in this PR

Adds credential-derived, path-scoped egress rules for secure Gemfury downloads through shared S3 endpoints.

Changes:

• Adds exact host/path-prefix matching for GET and HEAD requests.
• Configures /gemfury/ access on s3.amazonaws.com.
• Adds security, validation, and regression tests.

File	Description
internal/​handlers/​egress_dynamic_hosts.go	Implements credential-derived path rules.
internal/​handlers/​egress_dynamic_hosts_test.go	Tests matching and security boundaries.
internal/​handlers/​egress_allowlist.go	Integrates path-scoped request authorization.
internal/​handlers/​egress_allowlist_defaults.yaml	Adds the Gemfury S3 path rule.
internal/​handlers/​egress_allowlist_defaults.go	Validates path derivation configuration.
.github/​skills/​add-egress-allowlist-domain/​SKILL.md	Documents path-scoped derivations.

🧠 Review effort: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.