perf: remove hot-path allocation, syscalls, and redundant fsyncs - #3
Open
ctxswitch wants to merge 1 commit into
Open
perf: remove hot-path allocation, syscalls, and redundant fsyncs#3ctxswitch wants to merge 1 commit into
ctxswitch wants to merge 1 commit into
Conversation
ChannelId::as_key heap-allocated a String on every metadata key build, several times per request; it now encodes into a stack buffer via Ulid::array_to_str, which is what Display already calls, so keys stay byte-identical. The three key builders in rocksdb.rs size their Vec exactly instead of allocating twice and reallocating. bind_reference takes a Durability and the proxy passes BestEffort, removing an fsync from every upstream 304; remove_artifact drops its fsync to match evict, which performs the same delete. The rest are independent and behaviour-preserving: classify runs once per request rather than twice, the Method round-trip and the ring clone are gone, prefetch-only header parsing moved inside the prefetch branch, Identity-encoded files no longer fstat for a length known by construction, the recent-use filter derives four probes from one hash, the space ledger snapshots under a single lock, route_prefix is built only by the three handlers that read it, and already-materialized metadata streams without holding a foreground permit.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second of six stacked PRs. Based on
cleanup/phase-1-bugs(#2) — review that onefirst. No structural change here; every item is strictly faster and same-or-smaller.
Widest blast radius:
ChannelId::as_keyheap-allocated aStringon every metadatakey build, several times per request. It now encodes into a
[u8; ULID_LEN]stackbuffer via
Ulid::array_to_str— the functionDisplayitself calls, so keys staybyte-identical — and the
.expectgoes with it.artifact_key/reference_key/eviction_keynow size oneVecexactly instead ofto_vec()+extend_from_slice(two allocations plus a realloc each).bind_referencetakes aDurability; the proxy passesBestEffort, removing anfsync from every upstream 304.
remove_artifactdrops its fsync to matchevict,which performs the same delete without one.
The remainder are independent:
classify()runs once per request instead of twice,the
reqwest::Method::from_bytesround-trip and the ring.cloned()are gone,prefetch-only header parsing moved inside the prefetch branch (both copies),
Identity-encoded files skip anfstatfor a length known by construction, therecent-use filter derives four probes from one hash by Kirsch-Mitzenmacher instead
of building a
DefaultHasherper probe, the space ledger snapshots under a singlelock rather than three that could disagree,
route_prefixis built only by thethree of fifteen handlers that read it, already-materialized metadata streams via
Body::fromwithout holding a foreground permit,observe_http_requesttakes aStatusCode,server_request_identityreturns borrowed strings,channel_fencereturns the expiry from the record it already read, and
Referencegainedis_valid/into_stringso two call sites stop allocating to validate-and-discard.Verification
make ciclean, includingclippy --all-targets -- -D warnings.On-disk format guards pass unchanged, confirming
as_keyis byte-identical:fresh_store_uses_only_channel_keys_and_record_schema_oneandregistered_channel_survives_reopen_without_legacy_policy_fields.Crash-recovery coverage for the durability change passes unchanged:
tests/integration/maintenance.rsand the channel deletion/resume tests.New
src/cache/recent_use_test.rscovers mark/rotate lifetime, identity keying,and a false-positive bound on the derived probes — confirmed to fail (25 false
positives against a bound of 5) when the probes are forced to collapse onto one
slot.
Log output for the single-
classifychange was checked against a temporarycapturing subscriber and is unchanged on both data and control routes.
make ciDocumentation and examples are accurate
New or changed behavior has test coverage
Operational impact
Two deliberate behaviour changes:
GETon the Bazel action cache with an unparseable hash now returns 404 insteadof 400, matching
get_http_cache. The rejected key could only miss anyway. Notest asserted the 400.
reference→artifact binding that a 304 had refreshed; the next request re-validates
upstream and rebinds. The artifact bodies these references point at were already
published best-effort, so this aligns the metadata write with the data write
rather than weakening a guarantee that held end-to-end.
No storage format, configuration, route, or security boundary changes. Metadata keys
are byte-identical, so no migration.
Review notes
Start with
src/channel/identity.rsand the three key builders insrc/storage/metadata/rocksdb.rs— if those are right, the on-disk format isuntouched, which the two format guard tests independently confirm.
Two places where the implementation deviates from what was planned, both
deliberate:
delete_referencedid not get adurabilityparameter. Its only caller isreached solely from
DELETE /references/{r}, which is durable. The parameterwould have exactly one call site, always passing
Durable— dead flexibility thatremoves no fsync.
bind_referenceis where the win is and it is threaded.server_request_identityreturns(&str, Cow<'_, str>), not&'static str.The proxy arm's operation is a borrowed path segment and the fallback is
MatchedPath::as_str(); neither is'static. Tying both to the request lifetimeis allocation-free all the same.
Noted for later phases, not addressed here:
classifyandserver_request_identitystill each collect a
Vec<&str>of path segments per request, needed for theirslice patterns — the largest remaining per-request allocation on those paths. After
the snapshot change,
SpaceLedger::free_observed(),committed_since(), andreserved()are referenced only by tests.