Repository navigation
Description
The copying minor traverses the whole young object graph twice: once in CopiedMinorEligibility::evaluate's preflight, to prove that nothing reachable is pinned, and again in the collector that copies. On json_pipeline the first traversal is 21.8% of the build_out phase and 14.5% of total wall, and it produces no collection result at all.
Measured (pinned quiet mini, current main v0.5.1363, canonical 500k fixture)
build_out at 500k is 1,956 ms, of which 1,872 ms is GC pause across exactly two collections. The minor is 1,123.5 ms, but its own phase_us.copying_nursery timer only accounts for 666.2 ms. The 457 ms residual is eligibility, and the symbolicated profile agrees to within 1%:
sample window = the build_out phase (marker-triggered, 2072 ms window vs a 2070 ms phase), 1504 leaf samples
542 gc_collect_minor_copying_fast_path_with_eligibility (the copy + RS work)
294 CopyingNurseryPreflight::drain <- mutable_root_preflight_reason
40 CopyingNurseryPreflight::drain <- dirty_slot_preflight_reason
---
334 = 22.2% of build_out, ~460 ms
A probe that skips both preflight walks (PERRY_PROBE_SKIP_PREFLIGHT=1, an unmerged local arm), 5 interleaved rounds, output SHA identical on every row:
| build_out 200k | build_out 500k | total 200k | total 500k | |
|---|---|---|---|---|
| main | 734 ms | 1,963 ms | 1,158 ms | 3,024 ms |
| preflight skipped | 602 ms (−18.0%) | 1,536 ms (−21.8%) | 1,025 ms (−11.5%) | 2,586 ms (−14.5%) |
The subject was verifiably live and the decision verifiably unchanged (#7024/#7025): the minor still ran, still reported eligible=true, and still promoted 280,996,784 bytes — byte-identical to main — while the second traversal disappeared from the layout counters:
| minor-cycle counter | main | preflight skipped |
|---|---|---|
pause_us |
1,121.3 ms | 684.7 ms |
layout_scans.pointer_slots_read |
22,041,026 | 13,827,513 |
layout_scans.unknown_layout_slots_read |
16,500,006 | 10,500,003 |
copying_nursery.promoted_bytes |
280,996,784 | 280,996,784 |
sweep.freed_bytes |
17,848 | 17,848 |
What the walk is actually deciding
mutable_root_preflight_reason and dirty_slot_preflight_reason do a transitive reachability walk (CopyingNurseryPreflight::drain → scan_object_fields → check_ptr_with_reason, with a PtrHashSet seen set sized to the live young graph) to answer two booleans:
- is any transitively reachable
Eden/FromSurvivorobjectGC_FLAG_PINNED? →PinnedYoungRoot/PinnedYoungDirtySlot - was a non-arena candidate seen while the malloc registry is unavailable and was not empty at start? →
MallocRegistryUnavailable
(2) is already an O(1) fact — CopyingPointerSet::malloc_registry_empty_at_start / malloc_registry_available.
(1) is O(live young graph) today, but its producers are countable. In production, GC_FLAG_PINNED is set at exactly three sites:
thread.rs:1490— the cross-threadspawnpromise (allocated in malloc space by construction, per the comment there)thread.rs:1617pin_promise—Atomics.waitAsyncstring/format.rs:90— theSMALL_INT_CACHEentries (longlived space, andcheck_ptr_with_reasononly trips onEden/FromSurvivor, so these can never cause the fallback)
The conservative scan does not use this bit — it pins into the separate CONS_PINNED set (barrier.rs:653), and it only runs at all when conservative_stack_scan_decision() == Scan, which the same traces show is skip_disabled for every automatic arena_bytes cycle. copying.rs:630 preserves the bit across a copy but never creates one.
Proposed shape
Route the three setters through one gc::pin_object(header) helper that also arms a process-wide monotone latch, and skip both preflight walks when the latch is clear and the malloc-registry question is already answered. Monotone means no unpin bookkeeping: a process that has ever pinned pays the walk forever, which is strictly no worse than today.
What this needs before it can merge — this is a safety check, not a cost
Removing a guard on the moving collector is the highest-severity change class in this tree, and an evacuated pinned object is a use-after-free for whoever holds the raw pointer (the cross-thread promise queue holds exactly that: a raw usize with no scanner). So:
- A scanning gate in
lintthat fails on anyGC_FLAG_PINNEDset outsidegc::pin_object— the fix(runtime): JSX and raw-JSON shared a class id; add a scanning gate (#7587) #7589 class-id-collision gate is the precedent. Without it the latch's completeness is a comment, and the next pin site added silently makes the collector unsound. - A sabotage test: plant a pinned Eden object, assert the copying minor still falls back with
PinnedYoungRoot, and assert it goes green only because the latch was armed — i.e. delete the latch arming and watch the test go red (the perf(gc): break the survivor-promotion handoff livelock (#7592) #7594 lesson: a test asserting!preflight_reason(..)cannot fail on an empty heap). - gc-ratchet, both arms on one host.
layout_scans.*andmalloc_validation_lookupschange materially on every copying minor (the table above), so this needs a deliberate, documented counter shift rather than a silent one — same treatment as perf(gc): live-proportional collection budgets at both generations (#7592) #7596's12_large_live_set.heap_total_bytes.
A cheaper alternative that avoids the pin proof entirely, if (1)–(3) prove too costly: hand the preflight's seen set and worklist to the copy phase as a pre-computed reachable set, so the graph is still walked once. Bigger refactor, no new invariant.
Found while re-deriving the next json_pipeline lever after #7624 and #7633 (#7592). The second-largest item on the same profile — the full collection's shadow BTreeSet valid-pointer census, 245.5 ms of a 748.3 ms full — is fixed separately and needs no new proof.
Activity
docs: changelog fragment for #7645
perf(gc): skip the copying minor's eligibility preflight when its ans…
test(gc): plant every pinned-fallback test through pin_object, and co…
docs: changelog fragment for #7645
chore(gc-ratchet): re-pin 01_nursery_churn.heap_used_bytes for #7645
perf(gc): skip the copying minor's eligibility preflight when its ans…
test(gc): plant every pinned-fallback test through pin_object, and co…
docs: changelog fragment for #7645
chore(gc-ratchet): re-pin 01_nursery_churn.heap_used_bytes for #7645
perf(gc): stop traversing the young graph twice — skip the copying mi…
docs(gc-handoff): triage #7803 on a corpus that links again (#7989)
lint: three more gates for #10925, #10936, #10938 and #10941
lint: three more gates for #10925, #10936, #10938 and #10941
lint: three more gates for #10925, #10936, #10938 and #10941
lint: three more gates for #10925, #10936, #10938 and #10941