Repository navigation
perf: make forwarding, partial, keyword and method call shapes cheaper - #97
Merged
Merged
Conversation
A positional call of a function with `*args`, `**kwargs`, or keyword-only parameters (a decorator's `def wrapper(*args, **kwargs)`, `def f(a, *, key=None)`) took the general inline call every time: `core_call_plain` and the frame JIT's direct call declined extended parameters and closures, and `core_call_cached` re-derived the shape on each call. The cached plain call and the direct call now bind these shapes themselves (`fill_extended_raw`: the surplus as the `*args` tuple, the compiled keyword-only defaults, an empty `**kwargs` dictionary), along with the callee's cells, so `core_call_cached` goes away. A direct site carries the extended-parameter flag beside its cells flag, so the common shape pays nothing for the check. Frameless leaves with keyword-only parameters take their defaults at frame JIT leaf sites and fused method sites too, and a shape mismatch no longer reaches `pure_leaf_call`, whose bail counter could otherwise retire a leaf for good. Per call (instructions retired, quick build): the forwarding wrapper 3368 -> ~2460, `fva(1, 2, 3)` 1817 -> ~1245, `fkw(i)` 835 -> ~760. Suite geomean 0.999 (closures_decorators -1.3%, context_managers -1.8%).
Keyword calls that the call site's permutation can't bind (a callee with `*args`, a closure, a native caller) now share one by-name binder, `lean_bind_keywords`, which takes its positionals as slices: the `*args` surplus builds its tuple in place, the first keyword is tried at the parameter after the positionals, and keyword-only defaults are looked up only for the parameters left unbound. `call_python_kw_names` binds through it instead of computing a permutation per call, and a tier-2 dynamic keyword call of a plain function now goes there too instead of converting each keyword name into a `String` for the generic binder. `f(*args, **kwargs)` of a frameless leaf (the forwarding wrapper's inner call) evaluates without allocating: the operands leave the stack as a fixed array instead of a collected vector, the function and code are borrowed, and keyword-only parameters take their defaults. A static or class method read through an instance is remembered in the method-load site's slot (`MethodSlotFn::ClassFn` beside `Static`), so the frame JIT's method load serves it and the call stays native. Per call (instructions retired): the forwarding wrapper ~2460 -> ~2270, `o.sm(i)` 1280 -> ~790, `o.cm(i)` 1361 -> ~960, `fvk(1, 2, x=3)` in a tier-2 loop 5526 -> ~4530. Suite geomean 0.998 against main (closures_decorators -2.0%, context_managers -2.3%).
A `functools.partial` with more than one stored positional, or with stored keywords, was called through its native `__call__` from the slow step: the builtin lane's leaf probes, `dispatch_call`, and a nested call of the function. The `CALL` arms now send it down their Python path, where `core_partial_call` binds the stored positionals, the call's, and the stored keywords by name (as CPython's `partial_vectorcall` does) and runs the function as an inline activation, or a warm pure leaf in place. Anything that binder declines, errors included, still takes the native `__call__`. A pure leaf's keyword binding now has a pointer form, `leaf_bind_keywords`: each parameter points into the positionals, the keyword values, or the function's defaults, with no reference taken. `f(*args, **kwargs)` of a pure leaf with a non-empty mapping (the forwarding wrapper called with keywords) and a partial over a pure leaf evaluate through it in place. `core_call_ex` classifies its callee once instead of probing the generator and class paths for every function. `SlotStorage::leading_interned` reads a `__slots__` layout's leading values by key identity (its keys are interned), which the partial's field reads take on every call. Per call (instructions retired): `partial(f3, 1, 2)(3)` 4511 -> ~1510 (CPython 914), `partial(fkw, c=5)(i)` 7222 -> ~1910 (CPython 2134), the forwarding wrapper called with a keyword 4187 -> ~3400. Suite geomean 0.994 against main (closures_decorators -5.8%, context_managers -2.8%, xml_etree -1.9%).
Tier 2's dynamic call helper treated a keyword call of a Python function as free: the comment held that tier 1 paid the same generic binder. Tier 1 now binds those calls inline (by the site's permutation, by name, or as a frameless leaf), so a native loop that makes one per iteration is a thin driver around interpreter round trips, each paying the activation shell, the guard revalidation and the pin traffic. Such calls now count toward the call-density budget that retires the loop to tier 1, as positional generic calls already do. Keyword calls of native callees stay uncharged. Per call (instructions retired, microops loop): `fvk(1, 2, x=3)` 4530 -> ~2630 (it ran 5526 on main), `fkw(i, c=4)` 1226 -> ~905. The suite is unchanged (geomean 0.9995 against the previous commit, within noise).
A `CALL_KW` site's `CallPyKwNames` permutation (the keyword values' slots, computed once per site) refused callees with `*args` and closures, so `wrapper(x=1)` against a decorator's `def wrapper(*args, **kwargs)` and `f(1, 2, key=3)` against `def f(*args, key=None)` bound by name on every call and switched through the general inline call. The permutation now covers surplus positionals (they go to the `*args` tuple, which `kw_names_fill_locals` builds) and closures whose code has no cell variables of its own (their cells are the function's, bound as the lean cells), and the frame JIT's direct keyword call takes both, so these calls bind by moving their operands and run as direct calls. The full handler's keyword fast path and tier 2's site binding now run a closure's frame with its cells (`call_python_bound`), which they could not before because no closure ever got a permutation. Per call (instructions retired): `fvk(1, 2, x=3)` ~2630 -> ~2270 (5526 on main), the forwarding wrapper called with a keyword ~3360 -> ~3130 (4187 on main). Suite unchanged (geomean 1.0008 against the previous commit, within noise; 0.994 against main).
A native caller's call of a Python function (an `lru_cache` miss, a class's `__init__` from `instantiate`, a `key=` function that isn't a frameless leaf) copied its arguments into a fresh vector for `call_python_owned`, which only resized it for the exact positional shape. `call_python` now binds that shape itself into a pooled vector, so the call allocates nothing for its locals. A lean `__init__` with `*args` (`contextlib.suppress.__init__(self, *exceptions)`) built its surplus tuple from a collected vector; it now builds the tuple from the remaining arguments in place. Per call (instructions retired): a recursive `lru_cache(maxsize=0)` call pair 5604 -> ~5440, an `lru_cache` miss of a looping function 4864 -> ~4590. Suite geomean 0.998 against the previous commit (context_managers -0.7%), 0.992 against main.
This reverts eedc1bd ("perf: leave keyword-call-dense loops to tier 1"). Charging tier 2's keyword calls against the call-density budget retired whole native loops whose other calls tier 2 runs natively: the `call_overhead` bench fixture (positional, default, keyword and method calls in one loop) fell back to tier 1 and ran 2.2 to 2.6 times slower than main (3390 -> 8200 instructions per iteration). A keyword call costs tier 1 less only when it is nearly all the loop does, so the budget isn't the place to judge it.
# Conflicts: # crates/weavepy-vm/src/frame_jit.rs
# Conflicts: # crates/weavepy-vm/src/frame_jit.rs # crates/weavepy-vm/src/lib.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the call shapes wrappers and decorators use cheaper:
f(*args, **kwargs),functools.partial, keyword calls,*argscallees, and static and class methods.What
core_call_plain) and the frame JIT's direct call bind*args,**kwargs, keyword-only defaults and closures through one sharedfill_extended_raw;core_call_cachedis gone. A call of the wrong shape can no longer trip the leaf bail counter and stop a leaf being evaluated frameless for good.lean_bind_keywords) serves every keyword path;call_python_kw_namesand tier 2 no longer convert keyword names toStrings.f(*args, **kwargs)of a frameless leaf allocates nothing.MethodSlotFn::ClassFn).functools.partialwith two or more stored arguments, or with stored keywords, binds inline (core_partial_call) instead of taking the slow step; leaves with keywords bind through a pointer binder (leaf_bind_keywords);__slots__reads are cheaper (SlotStorage::leading_interned).CALL_KW's per-site keyword permutation covers*argscallees and closures without cell variables of their own, the frame JIT's direct keyword call included; the full handler and tier 2 run a closure's frame with its cells.lru_cachemisses,instantiate's__init__) bind into pooled locals; a lean__init__with*argsbuilds its tuple in place.Measurements
Instructions per call (
microops.py --instructions):f(*args, **kwargs)functools.partialf(*seq)*argscalleeGeomean over all 26
call_*operations: 1.415 to 1.200 times CPython; a partial with stored keywords goes from 7,222 to about 1,910, below CPython's 2,134. Whole suite: geomean 0.992 ofmain;closures_decorators0.942,context_managers0.966,xml_etree0.98; nothing regressed past noise (an eager method-slot allocation that costoo_patterns3%, and checks that costgoandraytrace1% to 2% on the direct-call path, were fixed before committing).Validation
-D warningsis clean.test_call,test_extcall,test_functools,test_keywordonlyarg,test_positional_only_argandtest_inspectpass with no unexpected results; probes of keyword errors and messages,**kwargsmutation safety,__kwdefaults__,__defaults__and__code__rebinding after warm-up, the tracer-on path, partial keyword merging, and static and class method rebinding match CPython.tests/regrtest.Revised after CI
The bench gate caught
call_overheadrunning 2.2 to 2.6 times slower: charging keyword calls to tier 2's call-density budget retired the fixture's mixed loop to tier 1 as a whole (8,240 instructions an iteration against tier 2's 3,390). That commit is reverted; the fixture is back tomain's count (3,559 against 3,574), every other fixture is within 1%, andf(*args, **kwargs)(4,415) and keyword calls (1,212) in tier-2 loops keepmain's cost. The 26 call micro-ops' geomean is 1.242 times CPython's (1.415 onmain).Merging
This touches the direct-call and binding code that #86, #94 and #95 change too; whichever lands later needs a rebase.
What's left
Shapes that need a real activation still pay 450 to 600 instructions in the frame JIT's call and return machinery (RFC 0078, stage 3); leaves with
*argsaren't evaluated frameless yet; keyword-only defaults are still found by string compare (50 to 70 instructions a call).