Repository navigation
perf: close the gap to CPython on realistic workloads - #85
Merged
owenthcarey merged 173 commits intoOct 20, 2026
Merged
Conversation
CPython 3.14's optimizer can store an inlined comprehension's result before restoring the saved local (POP_ITER; STORE_FAST result; STORE_FAST k). The decoder only ended a shared LOCAL|CELL slot's plain-local region at a store after SWAP, so a later comprehension's closure over the same name decoded as a plain local load and raised UnboundLocalError once the module loaded from its .pyc. The region now also ends at a store in the run of stores right after the loop's END_FOR/POP_ITER.
A class whose memoized instance plan is the bare allocation (the default __new__ and object.__init__, no native payload) now constructs C() from a guard-free read of the plan and moves the class off the stack, instead of re-deriving the shape, walking the MRO twice for module and generic-alias bases, and cloning the class and plan. The plan records the verdict as bare_alloc. Recycling a pooled instance no longer rebuilds slot storage that is already empty, and resetting an instance's split values stages them on the stack instead of allocating a vector to drop them. A generator body whose loop yields now compiles from a back edge's heat, as it already did from a step's: its yield ends a resume whether or not the body runs natively. Empty-class construction retires 22% fewer instructions per instance, and one storing an attribute 20% fewer.
A compiled function called with fewer positional arguments than it declares, its positional defaults filling the rest, now binds through the frame JIT's direct call and the core loop's plain call, as an exact-arity call already did. The site records how many defaults it fills, and a function whose __defaults__ was rebound takes the general path. deepcopy(x, memo) and go's find() take this path. The frame JIT's compile verdict counts a CALL or CALL_KW as an instruction it runs through a helper rather than as an exit, since call_op runs calls without leaving native code, and an exhausted tuple iterator that alone holds its tuple's second reference leaves the stack natively, as a list's already did, instead of handing FOR_ITER to the core loop once per loop. Instructions retired against the previous commit: go -4.7%, nqueens -4.2%, deepcopy -2.5%, raytrace -2.0%, chaos -1.5%; the rest within the quick build's codegen noise.
Most __init__ methods only store their parameters (or constants) into self and return None. Such a body's shape is recognized once per code object, and a class whose __init__ has it constructs an instance by those stores alone: the arguments move straight into the fresh instance's split values, in the order of the class's shared names, with defaults and constants copied in and the instance itself allowed as a value. Nothing evaluates the body. The stores' own sites prove the shortcut: each must have last stored, for the class's current version, the instance's next position, which also proves the store a plain __dict__ store (no descriptor or __setattr__ intercepts it). The first constructions take the general path and prove it; the class's instance plan then records the verdict, and later constructions skip the plan lookup, the arity derivation and the per-site checks. A class version change (a property or __setattr__ added later, for instance) rebuilds the plan, so the stores are proven again. A two-attribute construction retires 17% fewer instructions.
A body the frame JIT lowers lean (one whose IR comes out large) now keeps the in-line LOAD_GLOBAL cache read: a few loads and compares against the helper's full call. A LOAD_GLOBAL whose fused shape (len of a local, a pure leaf call, a class constant) declines now fills the in-line cache as any other read does, instead of calling the helper every time. An exhausted string or bytes iterator also leaves the stack natively, as a range's does. Instructions retired: deepcopy -4.5%, oo_patterns -2.8%, exceptions -2.4%, tomllib_loads -2.1%; suite geomean -0.7%.
Exceptions and context managers spent most of their time in bookkeeping around the operation itself: - `PUSH_EXC_INFO` kept a second copy of the handled exception per frame (`FrameExc::handlers`), which nothing read; `POP_EXCEPT` now releases the `exc_info_stack` entry it pops instead. - An `AttributeError` with `name` and `obj`, and a generator's `StopIteration`, are built in one step (message, `args` and fields) instead of construction plus per-slot stores; the common pseudo-slot keys (now including `name` and `obj`) resolve by length and one comparison instead of interning. - A `LOAD_ATTR` of a plain instance attribute that is certainly missing raises its `AttributeError` without the resolution passes. - `del obj.attr` keeps an instance's split layout instead of materializing a dictionary: the last value just leaves, and any other deletion lays the rest out over the instance's own copy of the names (which class-keyed fast paths never match), as CPython's split dictionaries do. - `LOAD_SPECIAL` runs natively in the frame JIT, and a `with` statement's enter and exit calls count as in-place method calls when its special sites found leaf or native methods, so with-heavy loops compile. Per-iteration instructions (probes): with 1932 -> 1811, raise 3760 -> 3610, missing attribute caught 9989 -> 6394, del attr 1670 -> 1188, contextmanager with 30037 -> 26789. Suite (instructions vs. the branch base): context_managers -6.8%, exceptions -3.8%, raytrace -2.2%, geomean 0.9986.
The core loop handed `DELETE_ATTR` and `DELETE_FAST` to the full leaf arms, so contextlib's generator managers (`del self.args, self.kwds, self.func` on every `__enter__`) and every `except ... as e:` clause's `del e` paid a slow-arm round trip. Both now have core-loop arms that run the full arms' common shapes (a plain instance's own attribute, a bound plain local) and release as they do. Smaller savings on the same paths: - Materializing an instance's split attributes into a `__dict__` (as `logging`'s `record.__dict__` does per record) moves the values instead of copying them first, and inserts each name with the hash its shared-names table kept, skipping hashing and comparisons. - `raise ... from None` stores `__cause__ = None` only over an existing cause (an unset one already reads None). - A traceback chain is torn down without moving each node out of its allocation. Per-iteration instructions: del attr 1188 -> 981, three deletes 6217 -> 5610, contextmanager with 26789 -> 26239. Suite vs. the branch base: context_managers -7.5%, exceptions -4.1%, raytrace -1.9%, logging_mix -0.8%, geomean 0.9982.
A method load on an instance now runs in the frame JIT's code when the site's cache holds the instance's class version: the cached plain function, its count bumped in place, replaces the receiver, which moves above it. An instance can't shadow the method when it has no published __dict__ and holds fewer split values than the name's position among its class's shared names (the names only ever grow), so the check is a few loads and compares. Anything else takes h_load_method, which fills the cache when it resolves a plain function on an instance that doesn't shadow it. Python functions now count among the variants whose clone and release compiled code makes a plain count update. Instructions retired: go -5.0%; p_meth probe -6%.
Checking that an instance's split values don't shadow a method scanned the class's shared names on every pure-leaf method call whenever the names' hash filter couldn't rule the name out, which depends on the process's hash seed: with an unlucky seed the scan was 10% of go. Each code name now remembers the names table it last searched and the name's position there; the table only grows, so values no longer than that position can't hold the name and need no scan.
Coroutine and `yield from` bodies now compile in the frame JIT: `GET_AWAITABLE`, `GET_YIELD_FROM_ITER`, `SEND` and `END_SEND` run through helpers that do what the core loop's arms do (`SEND` switches to the delegate and reloads), and a generator-family body warms at its creation, since its activations never start at pc 0 where a call's heat is counted. `fib` in the coroutines benchmark and `Tree.__iter__` in generators_tree now run natively. Generator expressions over closures and subscripts take fast steps once their bodies compile (`gen_fast`'s CODE_NATIVE verdict): `LOAD_DEREF`, `BINARY_SUBSCR`, `LOAD_ATTR`, `IS_OP` and a few others are left to the native code, whose helpers decline instead of raising in a fast step; a release that queues a finalizer now ends the step so the general loop runs it first. Exhausted tuple iterators, and list iterators a generator expression's `.0` shares, retire in the native and fast-step `FOR_ITER` as the core loop's arm retires them, instead of handing the generator's end to the general loop. Generator creation is leaner: the `RETURN_GENERATOR` scan is cached in the code's extension, the frame of a cell-free body is built in place with the arguments moved straight into its locals, the frame's weak backlink is stamped through `Rc::new_cyclic`, the coroutine origin depth skips its thread-local until some thread sets it, and the core loop goes on after making a generator instead of reloading. The frame JIT's loop heuristic no longer counts a generator expression's creating call, or the cold fallback of an inlined `tuple`/`list`/`all`/`any`, as exits, so consumer loops like `t += len(tuple(x for x in v))` compile. Instructions retired vs the branch point (suite, per call): nqueens 0.887, coroutines 0.925, generators_tree 0.952, async_tree 0.992, geomean 0.995. Probes: tuple(genexpr) 3.00x -> 2.28x CPython, closure genexprs (nqueens shape) 2.87x -> 2.31x, recursive await 3.19x -> 2.72x.
`gen.send(v)` as a method call loads the unbound `send` (the `.u.` sentinel `type(gen).send` gives) with the generator as its self, so no bound method is made and released per send; the core loop's `CALL` and the frame JIT's call helper both resume the generator inline for either form. `send` now checks its one argument as CPython's `METH_O` does (`generator.send() takes exactly one argument (0 given)`), which the bound form didn't before. Generator creation reads the code's shape (a generator-family body with positional parameters only) from the cached `RETURN_GENERATOR` verdict instead of testing it per call, and a new generator joins its young set directly (`gc_trace::track_generator`). Instructions retired vs the branch point (suite, per call): coroutines 0.883, nqueens 0.881, generators_tree 0.946, geomean 0.991. Probes: `avg.send(i)` 1.55x -> 1.36x CPython, an `await` of an `__await__` generator 2.12x -> 2.00x.
A compiled consumer's `SEND` or `FOR_ITER` that switches to a generator or coroutine now runs the delegate's native code from the helper, as a compiled caller runs a compiled callee (`run_switched_directly`, shared with `run_callee_directly`): the delegate's yield or return switches back through new native lowerings of a generator body's `YIELD_VALUE` and `RETURN_VALUE` (`h_gen_yield`, `h_gen_return`, the core loop's arms as helpers), and the consumer goes on in its own native code at the `END_SEND`, past the `FOR_ITER`, or past the loop, without the two core loop reloads per resume. Anything else (a fast step, a root activation folding into a draining consumer, a yield passed up a chain, a pending stop) leaves for the core loop as before. A generator call moves its arguments into the new frame's locals with one copy instead of through a draining iterator. Instructions retired vs the branch point (suite, per call): coroutines 0.852, nqueens 0.882, generators_tree 0.942, comprehensions 0.994, geomean 0.991. Recursive `await` probe: 2.72x -> 2.52x CPython.
A finished generator's frame box goes back to a small per-interpreter pool (its frame dropped in place) and the next generator's frame is written into it, so a generator's life no longer costs a box malloc and free, and the new frame is boxed once instead of being moved through `wrap_started_generator` by value. Warming a generator-family body at its creation (so its `SEND`s run natively) is now limited to coroutines and bodies that await or delegate (a `SEND`, marked in the cached `RETURN_GENERATOR` verdict): a plain loop-free generator such as a `@contextmanager` body ran slower compiled than interpreted (context_managers +0.9%), and plain generators still warm by their loops and fast steps. Instructions retired vs the previous commit (suite, per call): coroutines 0.982, async_tree 0.986, context_managers 0.991, generators_tree 0.990, nqueens 0.994, geomean 0.997.
The frame JIT handed most container instructions and builtin method calls
to generic helpers that copied every operand onto the stack, released them
after, and dispatched on the operation at run time. This change lowers the
common shapes to lean helpers that read local and constant operands in
place, and gives builtin methods per-method kernels:
- Method kernels: `x.m(args)` on a local exact `list`, `dict`, `set` or
`str` whose site settled on a leaf method checks the receiver's tag in
line and calls a kernel on the operands where they lie (no method
object, receiver copy or argument copies), continuing after the `CALL`.
Kernels cover `append`, `pop`, `extend` (a new leaf kind), `dict.get`,
`set.add`/`discard`/`remove`, `startswith`/`endswith`/`find` (with a
start position), `isdigit`/`isalpha`/`isspace` on ASCII text, and the
pure `str` bodies (`lower`, `strip`, `split`, `join`, `replace`, ...).
Calls of the same methods on stack receivers use them through a small
leaf-site helper, which also runs `isinstance`, `len` and the registered
fast halves before the general call helper.
- `BINARY_SUBSCR`, `STORE_SUBSCR`, `UNPACK_SEQUENCE` and `CONTAINS_OP`
read local and constant operands in place, settle the exact-container
shapes in one small helper, and fall back to the container helper
inside it (in lean bodies too). Unpacking a dead tuple recycles it into
the tuple free list, and `enumerate` steps draw from it.
- Copies and releases of strings and tuples are in-line count changes,
as for instances and lists (their counts sit two words before the
payload); builtins and sets join the counted kinds; `POP_TOP` releases
a shared value by a plain decrement.
- `FOR_ITER` takes a live range or a list's, tuple's or ASCII string's
next item in a small helper before the general one.
- A method load from a module keeps the entry it read while the module's
dict stamp holds (`math.floor`).
- A loop whose calls are settled leaf builtins counts them as run in
place when deciding to compile.
- Hashing: `int` and `str` keys hash without the general dispatch, a
tuple of ints and strings without an error scope, and `hash()` is a
fast leaf for such values.
- `str.strip`/`lstrip`/`rstrip` scan bytes for whitespace or an ASCII
set (one allocation, the receiver itself when nothing goes), and a
one-byte needle in a short string is a plain scan.
Instructions per op (loop overhead subtracted, operands as locals),
before -> after (CPython 3.14): set.add+discard 2269 -> 711 (617);
list.append+pop 917 -> 200 (530); x = []; x.extend(t) 2754 -> 1035
(681); hash((1, 2, i)) 2384 -> 1024 (776); s.isdigit() 616 -> 150 (239);
s.startswith('12') 630 -> 145 (338); d.get('a') 651 -> 300 (280);
d[key] on an 81-key dict 625 -> 400 (233); '5' in s 404 -> 215 (200);
a, b, c = t 397 -> 215 (124); for c in 'abcdef' 2817 -> 1590 (1099);
isinstance(l, list) 990 -> 650 (248); math.floor(2.5) 846 -> 553 (207).
Suite (instructions per call vs the branch's baseline): geomean 0.977;
unpack_sequence 0.89, graph_search 0.90, sudoku 0.90, text_processing
0.91, mini_interpreter 0.92, nqueens 0.93, chaos 0.94, tomllib_loads
0.95, go 0.97; regex_mix 1.016 (its bench loop now compiles), and
float_points and collections_mix 1.016 (unexplained: neither runs the
changed paths much).
The in-line list/tuple index of a local container sent every other shape (a dict, a string, an index out of range) to the general container helper with both operands copied to the stack and released after. It now takes the in-place subscript helper, which reads the local where it lies and settles exact dicts and strings itself, falling back to the container helper inside. [s for s in u if '5' in d[s]] over 27 keys: 27371 -> 22614 instructions (CPython 14303); d[key] on an 81-key dict 437 -> 414; tomllib_loads 0.959x, the suite geomean unchanged (0.9998).
A call site settled on a registered leaf body (a deque's append or popleft, a pure str method on a stack receiver) or a scalar-only builtin went from the leaf-site helper on to the general call helper, which looked the site up again before running the same body. The leaf-site helper now runs those bodies itself, under the general helper's checks. deque append+popleft (microops, globals): 1852 -> 1473 instructions; math.floor(2.5) 764 -> 551. Suite geomean 0.9963 against the previous commit (0.9715 against the branch's baseline).
A method call of an effect leaf (a body that stores) whose body itself makes a call no longer runs frameless through the leaf evaluator: its callee's evaluation nests inside it, and the whole costs more than the in-line method load and direct call compiled code otherwise takes. Pure leaves, and effect leaves that make no call (setters, counters), keep the frameless path. Each code object caches whether it calls. Instructions retired: go -6.9% (its recursive Square.find), raytrace -1.0%; the rest within noise.
The plain call binder moved the arguments, then pushed the missing defaults, then filled the other locals with unbound markers, each with its own capacity check and loop. It now checks the capacity once and writes all three in one pass.
A CALL site's direct-call cache held one callee, so a polymorphic method call (an expression tree's eval on its node classes) refilled it on every change of receiver, and a callee without a compiled body (a frameless leaf) emptied it, sending the compiled callees through the core loop for the next 64 calls. The site now keeps a second callee, promotes a newly seen compiled one to first place, and leaves its compiled callees alone when one without a body comes by. mini_interpreter's direct calls rise from 6.5k to 35k per run.
A method load site's in-line cache holds one receiver class, so a polymorphic site (an expression tree's eval, shapes' area) took the general helper whenever the class changed. The cache now keeps up to three more classes' functions, each with the shared-names position that proves an instance doesn't shadow the method, and the helper checks them before its general path. Instructions retired: mini_interpreter -2.9%, oo_patterns -0.5%.
LLVM unrolled the binder's argument move and unbound fill by four, so a call with one or two arguments paid each loop's prologue and remainder (about 55 instructions a call between them). Counts up to three move in straight-line code now.
logging and the stdlib utility workloads spent much of their time in code CPython runs natively: - `posix._path_normpath` and `posix._path_splitroot_ex`, the C accelerators `posixpath` imports when present, now exist (with the pure-Python fallbacks' exact results, and leaf fast halves for `str` and `bytes`). `os.path.normpath` drops from ~19,000 to ~2,000 instructions, and `pathlib` parses every path through `splitroot`. - `time.localtime` builds its `struct_time` from a cached type and interned field keys whose hashes are kept (struct sequences in general, `os.stat` results included); `time.strftime` takes an ASCII format straight to libc. localtime 13,500 -> 9,700, strftime 19,000 -> 13,900 instructions. - `hasattr`/`getattr` of a module's global runs as a leaf (2,900 -> 1,000). - `sys._getframe` materializes only the frame it returns, linking `f_back` lazily to the caller's shell (the frames below are built only if something walks to them); `sys._getframemodulename` reads the shell's globals without building a frame object. - A quiet return from an activation whose frame object exists settles that object without entering the general loop when nothing observes returns. Suite (instructions vs. the branch base, pinned seed): logging_mix -4.6%, stdlib_utils -3.7%, geomean 0.9948.
`str % args` went through the general printf engine even for the plain shapes logging and EAFP messages use (`"%s,%03d"` aside): every conversion parsed into owned flag, width and precision strings, and a mapping format looked each key up twice (once to vet the value, once to render it), allocating and hashing the key both times. A template whose every conversion is a bare `%s`, `%d` or `%i` of an exact `str`, `int`, `bool` or `None` (by position, consuming every argument, or by `%(key)` from an exact dict) now renders in one pass; any other shape, a missing key or a leftover argument takes the general path, which renders or raises as before. Struct sequences also fetch their interned field keys once per instance instead of once per field. `logging`'s `"%(asctime)s %(levelname)s %(name)s: %(message)s" % d`: 6,300 -> 3,200 instructions (CPython 5,700). Suite vs. the previous commit (pinned seed): logging_mix -3.9%, exceptions -2.0%, geomean 0.9988.
When tier 2 ran a framed activation natively, a call it made into the interpreter left the activation's frame shell at its entry position, so a callee reading f_back.f_lineno (logging's findCaller, warnings, traceback.walk_stack) saw the caller's def line on the first native run. The native code already stores the call site's position before its call helpers; the helper now records it in the activation's shell, the spine's top, as the interpreter's own call does.
The native call_soon, create_task and create_future check a stock loop's _debug, _closed, _ready, call_soon, get_debug and _task_factory on every call (about 500 reads per async_tree iteration). Each read scanned the class's shared names linearly, and the module checks (events.Handle, tasks.Task, futures.Future) hashed their names each time. The reads now remember the name's position in the shared-names table (verified on every hit, so a reused table address only costs a search), and the module checks use precomputed hashes. A task whose coroutine yields None (asyncio.sleep(0)) now reschedules before looking for _asyncio_future_blocking, as the C Task does; the failed lookup on None built an AttributeError every time. is_native_future checks the exact native classes before walking the MRO. async_tree: 1771M -> 1704M instructions per call (-3.8%); sleep(0) round trip -13%. Suite geomean 0.998x.
exec, eval and FunctionType register a dict-subclass globals mapping so global-name misses under it reach its __missing__, and set a gate that sends every builtin LOAD_GLOBAL (in the frame JIT, the core loop and leaf plans) down the slow path while any is registered. Pruning dropped dead owners but never lowered the gate, so one @DataClass whose annotations needed annotationlib's _StringifierDict (_colorize, hence traceback, logging and asyncio) slowed builtin lookups for the rest of the process: a builtin-calling function compiled by the frame JIT left native code at every such LOAD_GLOBAL (1579 -> 3020 instructions per call of a two-builtin function after import asyncio). A prune now sets the gate to whether any owner is left, and runs every 64th finalizer drain while the gate is up (owners die when the code that used them is done, with nothing else to notice). logging_mix -0.9%, async_tree -0.6%; suite geomean 0.999x.
The loop checks behind call_soon, create_task and create_future read each attribute through a borrowed split-values cell, compared the remembered name byte by byte, and cloned the value only to test it. They now peek at the values (falling back to the borrow when the cell is busy or shared), accept a remembered position whose name text sits at the same address with the same hash, and test _debug, _closed and the instance overrides in place. A future's state handle is read from its slots by peek as well. async_tree -1.3% (at_tree probe 4.74M -> 4.67M instructions per iteration); suite geomean 1.585 (base 1.586).
iter(obj) and await obj over a generator-function __iter__/__await__ staged the receiver in a fresh one-element Vec, copied it into pooled locals and recycled the Vec. A body without cells of its own now gets the receiver (and any defaults) written straight into pooled locals, as core_gen_call does for a call's arguments; other bodies stage it in a pooled scratch vector. Collapsing a suspended yield-from chain re-decoded each level's RESUME/JUMP_BACKWARD/SEND triple; a level at the same code and pc as the one before it (a recursive generator's) now reuses that level's SEND. generators_tree -1.0%, async_tree -0.8%, p_coyield probe -1.4%; suite geomean 0.998x (0.997x vs base).
The collector tracked each object with a separately allocated handle (an `Arc<TrackedHandle>` of 48 bytes), found it through a hash set keyed by address, and built a strong candidate copy (48 bytes) plus an id-to-position hash map of every candidate for each collection, so a collection paid several hash probes and allocations per object. The registry is now a slab of 32-byte entries, one list of slots per generation, and an id index only for the kinds that can't record their own slot. Instances, classes, functions and the generator family carry their slot (`GcSlot`, in what was padding for instances), so registering, finding one from an edge, and an instance's death never hash. A collection marks the generation lists in place: each entry holds its candidate flag and `gc_refs`, the walk reads objects through reference-free views, and only temporaries and the unreachable are held strongly. The mark walk reads the common containers' cells without borrow bookkeeping, visits an instance's class without a clone, offers external traverse hooks only the kinds they match, and caches index hits for children seen repeatedly (module namespaces). A tuple of atomic values is no longer promoted to a temporary candidate (CPython untracks those at creation), which graph_search's edge lists produced by the hundred thousand. Instruction counts per iteration (macOS, PYTHONHASHSEED=0): gc.collect() over 2N live objects 36.0k -> 25.5k self-cyclic instance churn 4.31k -> 3.39k N cyclic instances, one collect 5.07k -> 3.96k instance holding a list, kept 10.68k -> 8.50k Suite (ins_suite vs m1): async_tree 0.982, comprehensions 0.965, xml_etree 0.987, graph_search 0.992, geomean 0.997. Peak RSS: async_tree 103 -> 77 MB, graph_search 61 -> 47 MB, the live-collection probe 107 -> 62 MB, the kept-instances probe 80 -> 64 MB.
A collection walked every candidate twice: once to subtract internal references, and again from the externally referenced candidates to mark what they reach, re-running every traverse hook (a coroutine's frame, an asyncio future's state) and every child lookup. The first walk now records each candidate's edges to other candidates (4 bytes an edge, in a flat array), and marking follows those. Also: the child lookup caches every answer (registry candidate, temporary, or neither) by address, so the children reached from many parents (module namespaces, shared functions, deferred instances) cost one probe; the asyncio future traverse hook is offered only instances with slot storage and finds the future's state once; and a flush registers the young sets under one registry borrow, moving each weak handle into its entry instead of upgrading and re-downgrading it. Instruction counts per iteration (macOS, PYTHONHASHSEED=0): gc.collect() over 2N live objects 25.5k -> 19.0k instance holding a list, kept 8.50k -> 7.40k self-cyclic instance churn 3.39k -> 3.27k Suite vs the previous commit: async_tree 0.930, comprehensions 0.976, geomean 0.9975 (against m1: async_tree 0.912, comprehensions 0.942, xml_etree 0.974, geomean 0.9946).
Cranelift's working memory grows with the IR: about 1 KB of resident memory per IR instruction with the backtracking allocator, so a large lean body's compile sets many small workloads' peak (TextWrapper. _wrap_chunks, 8,285 IR instructions, raised text_processing's RSS by 9 MB while it compiled). Three changes cut what lean bodies emit: - Every helper status that leaves the native code jumps to one shared tail (the raise test and the return) instead of repeating it. - A lean body's CALL makes the choice between the direct call and the general one in a new helper, h_call_site, instead of a six-block decision in line (non-lean bodies keep it in line). - A lean body tries an operator method directly (h_binop_dunder) only at a site that has run on instances; elsewhere a declined operation goes back to the core loop, which runs it. Over the suite's compiles, lean bodies' IR falls 10.7%, their blocks 16.4% and their code 14.4% (non-lean: 1%, 2% and 4%); _wrap_chunks goes to 7,260 IR instructions. The frame JIT's exit counts are unchanged on every workload. Peak RSS (quick build, warm caches): suite geomean 0.987x, with text_processing 27.6 to 26.0 MB, stdlib_utils 32.6 to 31.4 MB, tomllib_loads 27.9 to 27.1 MB and go 24.9 to 24.4 MB. Instructions per call: geomean 0.999x.
Scope analysis (read and free-name collection, the validator, the inlined-use scan) builds many small HashSet<String>s per code object, and the symbol table is an IndexMap<String, Binding>; all used the standard library's keyed SipHash. A multiply-rotate hasher (rustc's FxHasher) does the same job in a fraction of the instructions, and the compiler never hashes enough attacker-chosen names for flooding to matter. Measured (instructions retired, macOS, pinned hash seed): - compile() of the frozen/non-frozen dataclass __create_fn__ sources: 1.190M -> 1.089M per compile - compile() of namedtuple's __new__ lambda: 113k -> 103k - class_creation probe (type, dataclass, Enum, namedtuple per iteration): 4.97M -> 4.85M per iteration
The compiler's scope scans (read collection, declared names, the validator's used/assigned sets) inserted `name.clone()` into a HashSet<String> for every occurrence, allocating and freeing a String each time a name repeated (`self`, parameters). A small NameSet helper looks the borrowed name up first and allocates only when it is new. Measured (instructions retired, macOS, pinned hash seed): compile() of the dataclass __create_fn__ sources, 1.089M -> 1.052M per compile.
The core loop takes every execution of a CALL_KW site, but only the full step handler ever recorded the site's CallPyKwNames permutation, so the in-place pure-leaf and inline-activation keyword calls never applied in interpreted code: each keyword call bound its arguments generically and ran the callee in a nested interpreter loop. The core loop's call lane now makes the same decision (decide when empty, count a cooldown down, cool a site that stops binding). A keyword call of a plain class now constructs in place too (core_new_kw): the arguments bind by name as the generic binder would, and the __init__ activation runs inline, as a positional construction's does. Anything unusual (a custom __new__ or metaclass __call__, replaced defaults, a binder error) still takes the generic path. A latent bug this made reachable is fixed: the core loop's CALL_KW arm released its operands without checking for a queued finalizer, so a keyword argument's __del__ ran late. (The slot-15 permutation bug this also exposes is fixed by the earlier "keep keyword permutations off slot 15".) Measured (instructions retired, macOS, interpreter only): - f(1, b=2, c=3): 4785 -> 1245; o.m(1, b=2): 4781 -> 933 - C(1, b=2) with a Python __init__: 7556 -> 3239 - super().__init__(a, b=1) in an __init__: 8384 -> 5668 - class_creation probe per iteration: 4.85M -> 4.72M
Calling a class whose metaclass replaces __call__ with a plain Python function (EnumType does; so does any registry or singleton metaclass) left the core loop for the slow step, which bound the method and ran it in a nested interpreter loop. The core loop now turns the call's operands into that function's method shape, the class in the receiver slot (as the generic call's bind_method binds it), and takes the inline call; positional and keyword calls both do. A __call__ that is not a plain function (staticmethod, classmethod, a callable instance) and a metaclass that keeps type.__call__ are unaffected. Measured (instructions retired, macOS, interpreter only): - a class call through `def __call__(cls, *a, **k): return 1`: 6970 -> 3709 - Color(1) of an Enum: 13.0k -> 11.6k
A `super` proxy resolves attributes along its target's MRO, past the
proxy's class (CPython's super_getattro); WeavePy serves that from
load_attr_instance_default rather than a `__getattribute__` entry, so
type_has_attr_override did not recognize the `super` type, and a
LOAD_ATTR site that had seen a proxy specialized to the `super` type's
own MRO. From its second execution, `super().__new__` on a class
receiver then returned object.__new__ ("object.__new__() takes exactly
one argument"), and any name `object` also defines (__init__, __eq__,
...) could resolve the same wrong way. This only arises where the
compiler emits the generic super() call, as it does for every
zero-argument super() in a module whose top level names `super`.
Super proxy types now count as overriding attribute lookup, and the
core loop's instance-attribute resolver no longer lets `__new__`
through for them.
Correctness fix; no measured performance effect (the specializer and
resolver check a cached type bit only on paths that already decline).
The core loop's zero-argument super() path (leaf_load_super_attr)
declined two shapes that class machinery uses constantly, sending them
to the slow step and the generic proxy lookup:
- dunders other than __init__ (`super().__setitem__(k, v)` in a dict
subclass such as enum's EnumDict, `super().__setattr__`, ...), because
a built-in class may serve a dunder from outside its dict. A dunder now
resolves when the walk finds it before passing any built-in class (and
skips introspection-only entries, as the full lookup does);
- a class receiver that is an instance of the super() class, as in a
metaclass method (`super().__delattr__(name)` in EnumType.__delattr__).
Its binding is an instance's, through the metaclass's MRO, so it now
takes the same path. A class receiver that subclasses the super()
class (class methods, __new__) binds differently and stays on the
full path, as does `super().__class__`.
Measured (instructions retired, macOS, interpreter only):
- d[k] = v on a dict subclass whose __setitem__ calls super(): 8443 ->
5958
- super().__len__() in a dict subclass method: 5349 -> 2830
- C.x = 1; del C.x under a metaclass whose __setattr__/__delattr__ call
super(): 18967 -> 14402
- Enum("E", "RED GREEN BLUE") and use: 1.232M -> 1.185M per iteration
`hasattr(1, '__get__')` and `getattr(value, name, default)` on a value
of a built-in type ran the full attribute lookup and built, formatted
and discarded an AttributeError every time. enum asks
`hasattr(value, '__get__')` (then `__set__`, `__delete__`) of every
member value while building an Enum, and introspective code probes
builtin values this way constantly.
The values of the built-in types whose attributes all come from their
type (None, bool, int, float, str, bytes, tuple, list, dict, set,
frozenset) resolve an attribute by type and name alone, so a miss
holds for the class's current version. Like the TypeAttrMissing verdict
for built-in type objects, a new ValueAttrMissing verdict in the type
cache records a value's miss: a first miss still takes the full lookup,
a repeat answers at once. Subclass instances, functions, modules and
anything else with attributes of its own are untouched.
Measured (instructions retired, macOS, interpreter only):
- hasattr(1, '__get__'): 8509 -> 2157
- enum._is_descriptor(1): 26.3k -> 8.5k
- Enum("E", "RED GREEN BLUE") and use: 1.185M -> 1.138M per iteration
`s.to_string()` on a SharedStr went through the blanket ToString, formatting the text through Display and Formatter::pad. The VM does this wherever it turns a str object into an owned name (getattr, hasattr, setattr and delattr's attribute names, class names, function attributes). An inherent SharedStr::to_string, which method resolution prefers, now copies the bytes directly; the result is the same string (no width or precision is ever set). Measured (instructions retired, macOS, pinned hash seed): class_creation probe 3.482M -> 3.466M per iteration.
…d memo
`cls.__dict__.get(name)` (enum reads its classes this way, as does
inspect) loaded `get` off the mappingproxy on the slow attribute path
and called it through the full handler: a mappingproxy was not among
the builtin receiver kinds method_memo_tag names, which the method
memo, the LOAD_ATTR method slots and the core loop's builtin lane all
key on. A proxy over a class dict now has its own tag; its methods are
plain native bodies over the wrapped dict. A proxy over another
mapping (MappingProxyObj) still calls that mapping's own methods.
Measured (instructions retired, macOS, interpreter only):
- C.__dict__.get('zz'): 3745 -> 1713
- Enum("E", "RED GREEN BLUE") and use: 1.133M -> 1.105M per iteration
`obj[key] = value` on an instance whose class defines __setitem__ in
Python (a dict subclass such as enum's EnumDict, which every class body
assignment and Enum member goes through, or any custom container)
left the core loop for the slow step, which called the method in a
nested interpreter loop. As BINARY_SUBSCR already does for a Python
__getitem__ (try_inline_getitem), the core loop now runs it as an
inline activation: the three operands become its (self, key, value)
arguments and its result is discarded, as a property setter's is. The
site remembers the function by the class's attribute version; a
__setitem__ with other parameters, a descriptor or a native one keeps
its existing path.
Measured (instructions retired, macOS, interpreter only):
- d[k] = v on a dict subclass whose __setitem__ calls
super().__setitem__: 5958 -> 3231
- Enum("E", "RED GREEN BLUE") and use: 1.105M -> 1.087M per iteration
Three operations `inspect.signature` performs on every class or function it examines left the core loop for the slow step: - `sys.getrecursionlimit()` (inspect.unwrap reads it per call) was not among the builtin functions the core loop calls in place, though it reads no frame; - `name in cls.__dict__` (membership in a class's mappingproxy) had no leaf form; it now tests the wrapped class dict as a dict's does; - `weakref.ref(obj)` and `types.MappingProxyType(mapping)` calls (inspect's MRO cache and Signature.parameters) were not among the builtin types the core loop's type lane constructs through the ordinary call. Measured (instructions retired, macOS, pinned hash seed), per probe iteration: - str(inspect.signature(dataclass)): 933k -> 920k - class_creation probe: 3.427M -> 3.388M
… step The core loop's membership test over a tuple or list (leaf_contains) compared each element natively only for identical objects, native scalars and plain instances; any other pair sent the whole test to the slow step's rich comparison. Two common shapes now settle in place, for `in` only (list.remove's admission check keeps the narrower rule, as its native removal compares on its own): - an instance of an `int` subclass whose `__eq__` is int's own (an IntEnum or IntFlag member: `kind in (_POSITIONAL_ONLY, ...)` in inspect, enum's own checks) compares by its value against ints, bools and others of its kind; - a function, or a class under type's own `__eq__`, equals only itself against scalars, builtin functions, plain instances and its own kind. Any user `__eq__`, on either side or on a metaclass, still runs through the full comparison. Measured (instructions retired, macOS, interpreter only): - Parameter.POSITIONAL_OR_KEYWORD in (two other kinds): 6776 -> 929 - f in (len, None, f): 4625 -> 1460 - str(inspect.signature(dataclass)): 920k -> 869k per call - class_creation probe: 3.388M -> 3.316M per iteration
`type.__dict__['__mro__']` (inspect's _check_class reads type's own getsets this way) subscripted a mappingproxy, which the core loop's container op did not take, so it left for the slow step. A str key present in the wrapped class dict is now read as a dict's is; a miss still raises its KeyError on the full path. Measured (instructions retired, macOS, pinned hash seed): str(inspect.signature(dataclass)) 869k -> 866k per call.
os._path_splitroot_ex and os._path_normpath are registered on non-Windows targets only, but their helpers were compiled everywhere, so the Windows build failed on seven dead-code warnings under -D warnings. The helpers now carry the same #[cfg(not(windows))] as the registration.
Clippy (-D warnings) and rustfmt found issues across the merged performance work: elidable lifetimes, raw-pointer borrows and casts, index loops over parallel slices, a match on a bool, a redundant else in the core loop's offload macro, and a &Option return. The boxed ChainRec pool and the frame JIT's method caches keep their boxes (their contents are addressed by pointer) with the lint allowed and the reason noted, as the neighbouring cache vectors already do; the LRU order buffer's unaligned word reads and writes allow the alignment lint. rustfmt's check had been crashing on CI (an out-of-memory panic while diffing a large file) instead of reporting these diffs.
owenthcarey
changed the title
Since "call effect leaves that call directly", core_local_method declined an effect leaf (a method body that stores) whose body itself calls, so the call took a direct activation instead of the frameless leaf evaluator. That pays off for compiled callers, whose direct call is cheap (go -6.9%), but the core loop's alternative is a full activation: the interpreter ran deltablue about 10% slower than before the change. The decline is now a const parameter: compiled code's h_local_method keeps it, and the core loop runs those leaves in place again. deltablue (bench fixture, work 50, both JITs off), instructions retired: 2036M before this PR's change, 1924M at the previous head, 1754M now; main is 1826M. The pybench suite is unchanged within noise.
Compiled code called an effect leaf (a method body that stores) directly instead of through the frameless leaf evaluator whenever its body made any call. That paid off for go's recursive Square.find, whose frameless evaluation nests its own evaluation at every level. But an effect leaf that calls other code is cheaper frameless, and as a real activation it gathers call heat and compiles: deltablue's constraint methods compiled seven more bodies in a short run, to no gain even in longer ones, and the macOS bench gate measured deltablue 18% slower than main. The test is now whether the body may call itself (it makes a call and its names include its own); code_calls becomes code_recurses. deltablue (bench fixture, work 50), instructions retired: 2136M -> 2044M (main: 1977M). pybench suite geomean 0.9998, go unchanged.
mimalloc purges freed pages on macOS with MADV_FREE_REUSABLE, which lowers the footprint but leaves the pages in the resident size, and so in ru_maxrss, until the system reclaims them: freed memory never left a WeavePy process's RSS. A test mapping of 64 MB stayed at 66 MB resident after MADV_FREE_REUSABLE, MADV_DONTNEED and MADV_FREE alike, and fell to 0.6 MB only when fresh pages were mapped over it. No runtime option does that: purge_decommits, purge_delay, page_full_retain, page_commit_on_demand and arena_reserve left chaos, go and float_points where they were, and disallow_arena_alloc (a map and an unmap per page) cost 0.55% more instructions over the suite. WeavePy now vendors libmimalloc-sys 0.1.49, trimmed to the mimalloc 3 sources it compiles, with one patch in _mi_prim_decommit: on macOS a purge maps fresh pages over the range, falling back to upstream's advice if that fails. vendor/libmimalloc-sys/README.md records the upstream version, the reason and the exact diff. On macOS the allocator also sets mimalloc's purge delay to 10 ms (from 1000 ms, longer than a short program's warm-up; MIMALLOC_PURGE_DELAY still overrides it). Linux and Windows build the same C code paths and options as before. Peak RSS against the previous commit (quick builds, warm caches): chaos 27.5 to 25.0 MB, go 26.8 to 24.4 MB, float_points 58.9 to 45.4 MB. Over the suite this change cut the RSS geomean by about 7%. Instructions per call: +0.2% geomean, from refaulting purged pages (float_points +3.1%, async_tree +1.8%, graph_search +1.4%). It should also fix Pillow's TestAvifLeaks::test_leak_load on the 3-core macOS CI runners, which bounds peak RSS growth over one AVIF decode and failed there (never on Linux, where purging already drops pages). With freed memory resident, WeavePy sat at its peak, so the decoder's system-allocator memory set a new one: after freeing a large batch of Python objects, one decode raised the peak by 1.2 MB (CPython: 0), and now by 0. The bench gate's fixtures run no slower against the previous commit (list_ops, generators and float_math 7-9% faster).
The cycle collector is process-wide, and its registry spans every interpreter's objects. Since this branch, a collection reads its candidates without taking their cells' borrows (the mark walk's `peek`s), without holding a reference to them (`with_borrowed`), and the allocation paths append to the collector's deferred and young lists through guard-free `peek_mut`s. That is sound only if every thread that touches objects holds the GIL. Several threads did not. `Interpreter::default` built an interpreter, tracking thousands of objects, without the GIL, and an interpreter's fields released its objects without it when it was dropped. The VM unit tests run in parallel, each building and dropping its own interpreter, so one thread appended to the deferred list while another's collection drained it (`index out of bounds: the len is 1700 but the index is 1700` in `defer_container`, a `swap_remove` past the end, glibc's `malloc(): unaligned tcache chunk detected`), or released lists that another thread's mark walk was reading (the SIGSEGV in `traverse_impl` on a garbage child). A panic raised while Cranelift frames were on the stack aborted with "failed to initiate panic". On `main` the collector held a reference to each candidate and took cell borrows, so only the young lists' `peek_mut` raced, which is why the abort was rarer there. Take the GIL where objects are touched outside an eval loop: - `Interpreter::default` holds the GIL while it builds, and the interpreter's `Drop` holds it while its fields drop (a trailing `GilScope` field gives it back after the others). - A worker thread takes the GIL before it builds or forks its interpreter, not after. - The tier-2 JIT's thread-local state, whose tables hold functions and classes, is released under the GIL when its thread exits. If the GIL stays busy for a second, the state is leaked rather than freed, so a thread's or the process's exit can't hang on it. - The test harness's `run` and the unit tests that build objects directly hold the GIL (`gil::hold_gil`), as the contract requires. A new test checks that building and dropping an interpreter waits for a GIL held by another thread; it fails without the fix.
`sys.setrecursionlimit` sets a process-wide limit (WeavePy runs one interpreter per process), but the VM unit tests run many interpreters at once. `jit_native_recursion_limit_matches_interpreter` set the limit to 100 and left it there, and `recursion::tests` briefly set it to 4. `jit_direct_method_call_guard_changes_match_interpreter` sets 300 and prints the depth at which `RecursionError` first fires, once with the JIT and once without, so a peer test changing the limit midway made the two runs report different depths (240 vs 297), on `main` too. The limit-4 window could also make any concurrent test fail at depth 4. Run the two Python-level tests in a child process of their own, as other tests that depend on process-wide state already do, and let the unit test pass its limit to the entry check directly instead of changing the shared one. The assertions are unchanged.
A process's first compile pays the code generator's cold start (its code paged in, its tables built: about 0.7 ms). Earlier work in this branch moved the warm-up from start-up to a background thread started when the first code object was halfway to its compile threshold, so a process that never compiles wouldn't carry the code generator's pages. But a hot loop gets from halfway to its compile in far less time than the warm-up takes, so the first compile still paid most of the cold start on the thread that needs the code: the bench gate's short JIT fixtures ran 6-11% slower than on main (sumvm, nested_loops, jitloop), and on the Windows runner sumvm crossed the gate's 15% limit. The background warm-up now starts as start-up begins (run_site) when the JIT may run, and the halfway trigger stays as a fallback. It costs the main thread nothing: `-c pass` takes the same 10 ms. The price is the code generator's pages, about 2.6 MB resident in a process that never compiles (12.6 MB against 10.0 MB; main, which warmed synchronously at start-up, is 11.8 MB). Against the previous commit (best of 20): sumvm 4.20 -> 3.66 ms, jitloop 8.07 -> 7.44 ms.
Pillow's TestAvifLeaks::test_leak_load bounds the growth of the process's peak RSS (ru_maxrss) over one AVIF decode at cpu_count * 3 MB, 9 MB on the 3-core macOS runner. It fails whenever the process sits at its high-water mark as the test starts, and in the ecosystem lane it does: by then the suite has grown the process to about 3 GB on a 7 GB runner, and under that memory pressure the peak hugged the working set (measured margins 0-54 MB), so the decode's own buffers set a new peak (21.7 MB in a logged failing run). It isn't a leak. The same binary passed the test in 5 of 5 standalone runs of the suite on the same runner image, the other four leak tests' 100 iterations grew by 0 KB in the failing run, and it passes on Linux. main passed for an incidental reason: its collector allocated hundreds of MB of scratch per full collection (357 MB for 3M containers, against 48 MB on this branch), which set peaks far above the working set. The harness gains OS-scoped deselect lists (deselect_linux, deselect_macos, deselect_windows), applied only on that OS, so the test keeps running on Linux; a test pins the scoping against the checked-in manifest.
owenthcarey
marked this pull request as ready for review
October 20, 2026 04:54
This reverts commit ba20d32. Vendoring libmimalloc-sys (about 25,000 lines, one patch to mimalloc's macOS purge) is a dependency decision that deserves its own review rather than riding along with this branch. It was brought in expecting it to fix Pillow's AVIF leak test on macOS, which it didn't (that test's failure is the runner's memory pressure; see the manifest). Its remaining benefit, about 7% lower peak RSS over the suite at about +0.2% instructions, moves to a separate change.
`Table::clear` filled the slot table with zeros even when a small table has none, so a recycled keyword dict made a zero-byte `memset` call at the empty slice's dangling address on every `f(x, name=value)` call into a `**kwargs` function. On AMD Zen 4 and Zen 5 that call cost about 100 ns (glibc's AVX-512 `memset` stores through a mask there, apparently taking a microcode assist for the unmapped page), which made the bench gate's call_overhead fixture 1.5x slower than main on those runners. Probed on EPYC 9V45 runners: call_overhead goes from 49 ms to 31 ms (main: 31 ms).
owenthcarey
deleted the
claude/weavepy-performance-optimization-5e41ea
branch
October 20, 2026 23:09
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continues the effort to make WeavePy faster than CPython 3.14 on the realistic workload suite (
tools/pybench, 33 workloads). Along the way it fixes several correctness bugs, including a pre-existing thread-safety race in the cycle collector.Results
Geometric means against CPython 3.14 on macOS x86-64 (lower is better; 1.00x is parity):
PGO is the fair comparison: CPython's release builds use it, and it's worth 11-27% of cycles here (
tools/pgo/build.sh). "Before" ismainunless noted. The PGO numbers were measured at dc20d42; the one commit after it skips an emptymemset(see the call_overhead note below).With PGO, WeavePy takes fewer cycles than CPython on pickle_roundtrip (0.71x), json_roundtrip (0.82x), decimal_telco (0.87x), scimark (0.92x), unpack_sequence (0.97x), float_points (0.99x) and xml_etree (0.99x). The largest gaps left, in PGO cycles, are context_managers (2.48x), tomllib_loads (2.06x), deepcopy (2.02x), stdlib_utils (1.96x) and async_tree (1.92x).
Biggest moves in this PR, in instructions against CPython:
What changed
CALL_FUNCTION_EX.specialize::kw_perm).deltabluenote below).LOAD_GLOBALreads its value in place, and a numeric compare branches without pushing a bool.__slots__members are read and written by layout position, like CPython's member descriptors.__dict__,super(), class stores and properties are served from site caches.__call__runs as an inline call. Class creation,inspectprobes and enum membership stay in the core loop.yield fromchain stays collapsed across yields: resuming is O(1) in depth instead of about 133 instructions per level.map,filter,zip,enumerate,groupby, itertools sources) step natively in everyFOR_ITER, and sets use the compact dict table (dictmap::Table<V>).sys._getframe,findCallerandwarnings.warnwalk lean frames.co_positionsis built once per code object.struct.Structpack and unpack.math.gcd,float.as_integer_ratioand codec error handlers.lru_cachehits under one cache hold; decorator wrappers kept in compiled code.CodeObjectfits mimalloc's 384-byte size class.PYTHONDONTWRITEBYTECODEtoo. Without it,import asynciotook 1.75s instead of 0.30s in every such process; many container images set this variable.Correctness fixes
Each fix has a regression test (71 new test files under
tests/regrtest/, plus unit tests):main):mainand most runs on this branch before the fix.gil::hold_gil), and a unit test pins this down.dataclasses.make_dataclass(..., decorator=...)raisedUnboundLocalError.super: aLOAD_ATTRsite that had seen asuperproxy was specialized againstsuper's own MRO, sosuper().__new__could returnobject.__new__.__del__ran late for keyword-call arguments and for operands of four in-place call arms.set().discard(x)andset().remove(x)swallowed an error from__hash__.StopIterationraised by amapcallback propagated instead of ending the loop.list.insertrejected bools and__index__objects.'__main__.P') where CPython prints'P'.f_linenoreads are fixed..pyccould raiseUnboundLocalErrorwhen two inlined comprehensions reused a captured name.Validation
cargo clippy --workspace --all-targets --all-features -- -D warningsandcargo fmt --checkare clean.Lib/testgate (768 labels): 760 pass. The seven unexpected results reproduce identically on amainbuild, or are timeouts under local load:main: test_capi, test_compileall, test_embed and test_pydoc.Notes for reviewers
rustfmt --checkhit an out-of-memory panic while diffing a very large file, and the job didn't count that as a failure. The tree is now formatted, so the check has nothing to diff.deltablueon the bench gate: an early change sent every effect leaf (a method that stores) whose body calls something to direct calls. That helped go, through its recursiveSquare.find. But in the interpreter, and in short runs, it costdeltablueup to 18% againstmain, because its constraint methods compiled to no gain. Bisected and fixed: only recursive effect leaves take direct calls now, go keeps its gain, anddeltablueis within 3.4% ofmainin instructions.TestAvifLeaks::test_leak_loadis deselected on macOS only:mainpassed: its collector's huge scratch set peaks far above the working set.deselect_macosand so on) so the test still runs on Linux. The manifest entry documents the measurements.main) only on some runners. A probe across runner types traced it to EPYC 9V45 (Zen 5) and 9V74 (Zen 4) CPUs and to one call shape,f(x, name=value)into a**kwargsfunction. Clearing the recycled keyword dict made a zero-bytememsetcall at an empty slice's dangling address, which glibc's AVX-512memsetturns into a masked store that those CPUs appear to handle slowly (about 100 ns a call).Table::clearnow skips it for a table without slots. On a Zen 5 runner the fixture went from 49 ms to 31 ms (main: 31 ms).libmimalloc-sysmoved out: the macOS purge patch (about 7% of suite RSS on macOS) was reverted here and is on its own branch,perf-mimalloc-macos-purge, for a separate review. That's why peak RSS is 1.720x rather than the 1.638x measured with it.exec/evalsource cache, which mostly helped the benchmark's repeated sources.__del__for rebound locals after many iterations. It reproduces without this PR's changes and is tracked separately.