Skip to content

perf(pm): shrink binary 10MB→6.4MB — system git, single TLS provider, drop hickory/h2 - #3182

Closed
elrrrrrrr wants to merge 3 commits into
nextfrom
perf/pm-binary-size
Closed

elrrrrrrr wants to merge 3 commits into
nextfrom
perf/pm-binary-size

Conversation

@elrrrrrrr

@elrrrrrrr elrrrrrrr commented Jun 18, 2026 •

Copy link
Copy Markdown
Contributor

Draft / investigation. All three trims are implemented, tested, and benched. Opening as draft for review of the tradeoffs (notably: git is now a runtime requirement for git deps).

Result

build (macOS arm64, release profile) size
baseline (next) 10 MB
+ single TLS provider + drop hickory 8.6 MB
+ system git (drop gix + h2) 6.4 MB

−3.6 MB / −36%. The new pm-binary-size GHA job reports Linux + macOS sizes and asserts the trimmed dep set on both.

1. Git deps via system git (biggest win, ~2.2 MB)

Git dependencies were cloned by bundled gix (a full Rust git impl). Switched to shelling out to the user's git — exactly what npm/pnpm/yarn do. gix dragged in a ~20-crate family, plus gix-transport's default-featured reqwest (the sole non-dev puller of h2/HTTP-2), plus bstr→regex-automata. All gone.

  • resolver/git.rs reimplemented on std::process::Command("git"): shallow clone --depth 1 --branch <ref> for branch/tag/HEAD; full clone + checkout for a SHA. rev-parse HEAD pins <cache>/<name>/<sha>/; working tree copied (exec bits + symlinks preserved, .git skipped, path-traversal guard kept). Same public API + cache layout — no façade/pm changes.
  • Tradeoff: git is now required for git:/github: deps; a missing binary errors with a clear message naming it (npm is the same). Registry installs need no git.
  • Verified: e2e git-deps resolves all three spec forms (github:owner/repo, git+https#tag, owner/repo#tag) to the correct pinned commit.

2. One TLS provider — ring (~0.6 MB)

macOS was compiling two rustls crypto providers: aws-lc-rs (forced by ruborist) and ring (pulled by pm's rustls-tls). Dropped the macOS aws-lc-rs override → all native targets use ring.

The aws-lc-rs special-case existed for "~40% faster M-series cold resolves" — that didn't reproduce. Same-machine A/B (ant-design-sized cold resolve, cache cleared, 5 interleaved runs):

registry ring (median/min) aws-lc-rs (median/min) aws-lc vs ring
registry.npmmirror.com 1.47s / 1.32s 1.87s / 1.83s +27% slower
registry.antgroup-inc.cn 2.59s / 2.26s 2.84s / 2.19s +10% median / tied min (noisy)

cold resolve is network-bound, so aws-lc's crypto edge is invisible while its per-client load_native_certs + install_default adds fixed overhead. So this is a size win and a small perf win on China registries.

3. Drop unused hickory-dns (~0.5 MB)

Both clients already set .dns_resolver(shared_resolver()) (custom getaddrinfo + 300s cache), so reqwest's hickory-dns feature was compiled but never used — pure dead weight, zero behavior change, helps every platform.

Research — lighter reqwest / tokio?

Not viable. reqwest is heavy but load-bearing for the resolver's async concurrent-fetch model, and reqwest 0.12 requires tokio; both are deeply embedded. Sync HTTP (ureq) would abandon the concurrency and still need rustls. (Note: with gix gone, reqwest is now pulled only by pm/ruborist, so it could in principle be swapped — but the cost/risk still isn't worth it.) The realistic levers were the dependency trims in this PR.

Verification

  • Local macOS arm64 release → 6.4 MB; cargo tree -e no-dev confirms gix/gix-transport/h2/aws-lc-sys/hickory-* all absent, ring sole provider.
  • A/B cold-resolve bench (above) on both registries.
  • e2e git-deps (3 spec forms) resolve via system git; cargo clippy --all-targets -D warnings clean; cargo test -p utoo-pm 272 passed + 13 ruborist git tests.
  • pm-binary-size GHA workflow (ubuntu + macos) reports size + asserts the trimmed set.

🤖 Generated with Claude Code

elrrrrrrr and others added 2 commits June 19, 2026 01:32
Shrinks the release binary by removing redundant/dead dependency weight,
validated by an A/B cold-resolve bench (see PR body for numbers):

- TLS: drop the macOS-only `aws-lc-rs` override; unify every native target to
  rustls' default `ring` provider. The macOS binary was compiling BOTH crypto
  providers (aws-lc-rs forced by ruborist + ring pulled by pm's `rustls-tls`),
  ~610K of dead duplication. A same-machine A/B (cold resolve, ant-design-sized
  tree) showed ring is *faster* than aws-lc-rs on registry.npmmirror.com
  (-27%) and tied on registry.antgroup-inc.cn — the old "aws-lc ~40% faster"
  rationale did not reproduce on these registries.
- DNS: drop reqwest's `hickory-dns` feature (~497K). Both clients already set
  `.dns_resolver(shared_resolver())` (custom getaddrinfo + cache), so hickory
  was compiled but never used at runtime — pure dead weight, zero behavior
  change, helps every platform.
- HTTP/2: drop `http2` from our reqwest features (every client forces
  `.http1_only()`). NOTE: h2 still links in transitively via gix-transport's
  default-featured reqwest; fully removing it requires the native-git lever
  (see PR body).

Local (macOS arm64, release profile): 10 MB -> 8.6 MB.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Builds the release `utoo` binary on ubuntu + macos and reports its size to the
job summary, plus asserts the trimmed dependency set (no aws-lc-sys / hickory,
ring present). Gives size-focused PRs a shipped-artifact size signal on both
platforms instead of relying on a one-arch local build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes dependencies and unifies the TLS provider across native targets. It removes the hickory-dns and http2 features from reqwest and eliminates the macOS-specific TLS configuration that used aws-lc-rs, aligning macOS with other native targets to use the default ring provider. This allows for the removal of several unused dependencies, including hickory-resolver, moka, and rustls. The reviewer suggested a further optimization in crates/pm/Cargo.toml to replace "rustls-tls" with "rustls-tls-native-roots" to avoid statically embedding the webpki-roots certificate database, which will further reduce the binary size.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread crates/pm/Cargo.toml
"hickory-dns",
"http2",
"json",
"rustls-tls",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To further optimize the binary size, consider replacing "rustls-tls" with "rustls-tls-native-roots" in crates/pm/Cargo.toml.\n\nUsing "rustls-tls" enables reqwest's default WebPKI roots feature, which statically embeds the webpki-roots certificate database into the binary, adding around 150–250 KB of unnecessary weight. Since utoo-ruborist already uses and requires "rustls-tls-native-roots" on native targets, the final binary will already rely on the system's native certificate store. Unifying this to "rustls-tls-native-roots" here allows Cargo to completely drop the webpki-roots dependency from the dependency graph, saving additional binary size.

Suggested change
"rustls-tls",
"rustls-tls-native-roots",

Git dependencies (`git:`/`github:`) were cloned by the bundled `gix` (a full
Rust git implementation). Switch to shelling out to the user's `git` — the same
thing npm/pnpm/yarn do — and drop gix entirely.

This is the single biggest binary win: gix pulls a ~20-crate family, plus
`gix-transport`'s default-featured reqwest (which is what kept `h2`/HTTP-2 in
the binary), plus `bstr`'s regex-automata. Removing it drops all of that.

- `resolver/git.rs` reimplemented on `std::process::Command("git")`:
  shallow `clone --depth 1 --branch <ref>` for branch/tag/HEAD, full clone +
  `checkout` for a SHA (or a ref `--branch` can't name); `rev-parse HEAD` pins
  the `<cache>/<name>/<sha>/` dir; the working tree is copied (exec bits +
  symlinks preserved, `.git` skipped, path-traversal guard kept). Same public
  API + cache layout — no façade/pm changes.
- `git` is now required for git deps; a missing binary errors with a clear
  message naming it (npm behaves the same).
- `native-git` feature drops `gix`, keeping only `tempfile`.
- CI size-assert extended to gix/gix-transport/h2 (production tree).

Local (macOS arm64, release): 8.6 MB -> **6.4 MB** (10 MB baseline -> 6.4 MB,
-36% overall). Verified: e2e git-deps resolves all three spec forms
(`github:owner/repo`, `git+https#tag`, `owner/repo#tag`) via system git.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@elrrrrrrr
elrrrrrrr force-pushed the perf/pm-binary-size branch from 5b83498 to eb0bd81 Compare June 19, 2026 02:41
@elrrrrrrr elrrrrrrr changed the title perf(pm): trim binary — single TLS provider + drop unused hickory (size investigation) perf(pm): shrink binary 10MB→6.4MB — system git, single TLS provider, drop hickory/h2 Jun 19, 2026
@elrrrrrrr

Copy link
Copy Markdown
Contributor Author

Superseded by #3183 (codex), which takes the same approach (system git, ring TLS, drop hickory/http2) but is a superset: it also trims tokio/full → concrete features and reqwest stream (slightly smaller, 6.32 MB vs 6.4 MB), hardens the git fallback (SHA-256 + --), and folds the size-assert into pm-ci.yml. #3183's CI is green; this branch accidentally bumped the next.js submodule pointer. Posted my ring-vs-aws-lc A/B bench data to #3183 as the empirical justification for dropping aws-lc. Closing in favor of #3183.

@elrrrrrrr elrrrrrrr closed this Jun 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant