From a783d34f0ee70eb62b1be1c9a8d55a160a7295b9 Mon Sep 17 00:00:00 2001 From: Artur Shiriev Date: Mon, 3 Aug 2026 13:42:22 +0300 Subject: [PATCH] docs(performance): republish the comparative table at 3.2.0 Regenerated with just bench-report (5 paired runs, reproduced by a second independent 5-run pass). Every absolute is 23-29% above the 2026-07-28 publication for all five frameworks on identical library, rival, CPython and OS versions -- the machine was slower on the day -- so the page now states that outright and reads the ratios against it. Against that shift, C2 and C6 moved toward modern-di and C1/C3/C4 did not, which matches exactly what 3.2.0 touched. modern-di's warm singleton is now faster than dishka's on dishka's own by-type basis (0.98), the first time that cell has gone below 1.0. The event-loop floor inside every C4 cell was re-measured on the same machine (0.31 -> 0.35 us per request). The alias hop has no cell here; the suite has no alias scenario. Co-Authored-By: Claude Opus 5 --- docs/introduction/performance.md | 183 ++++++++++++++++++------------- 1 file changed, 106 insertions(+), 77 deletions(-) diff --git a/docs/introduction/performance.md b/docs/introduction/performance.md index c5bc949..b376efb 100644 --- a/docs/introduction/performance.md +++ b/docs/introduction/performance.md @@ -34,13 +34,13 @@ C1-C3 are published twice for modern-di: once resolved by provider reference lookup; that-depends and dependency-injector only by-reference. Each C1-C3 table compares one modern-di variant against the rivals whose API matches it, because a single column would flatter modern-di against half the set. By-type resolution adds a fixed dict-lookup cost on top -of `resolve_provider` — 46 ns on C1, 43 ns on C2 and 48 ns on C3 in the cells below, close to the -same absolute cost each time but 15%, 23% and 6% of the respective baselines. +of `resolve_provider` — 60 ns on C1, 54 ns on C2 and 65 ns on C3 in the cells below, close to the +same absolute cost each time but 15%, 26% and 6% of the respective baselines. C4 does not split this way: modern-di's C4 body resolves **by reference** throughout, while dishka and wireup can only be measured by type. That asymmetry cuts **against** modern-di's -C4 ratios, not for them — levelling it would add the ~46 ns by-type lookup to modern-di's cell -and move the dishka ratio below from 1.26 to about 1.29. The C1-C3 leveling does not apply to C4. +C4 ratios, not for them — levelling it would add the ~60 ns by-type lookup to modern-di's cell +and move the dishka ratio below from 1.24 to about 1.27. The C1-C3 leveling does not apply to C4. C6 does not split either, for the same reason: modern-di's C6 body resolves by reference and there is no by-type C6 variant to pair against dishka and wireup, so a split would leave that half of the @@ -56,7 +56,7 @@ for why that matters and which cells it moved. ## Results -Measured 2026-07-28 with modern-di 3.1.2 on an Apple M4 (macOS 26.5), CPython 3.14.6, +Measured 2026-08-03 with modern-di 3.2.0 on an Apple M4 (macOS 26.5), CPython 3.14.6, median over 5 runs (ratios paired within each run); the footnote under each table bounds the across-run dispersion of each side's own median. Rival versions: dishka 1.10.1, dependency-injector 4.49.1, that-depends 4.0.2, wireup 2.12.0. @@ -73,117 +73,135 @@ as a verdict. | Scenario | modern-di | vs dependency-injector | vs that-depends | |---|---|---|---| -| C1 transient | 279 ns ±0.6% | **0.76** ±1.2% | **0.89** ±1.6% | -| C2 warm singleton | 142 ns ±1.2% | 2.94 ±1.7% | 2.13 ±1.9% | -| C3 deep chain (6) | 768 ns ±0.9% | **0.54** ±3.0% | **0.72** ±1.1% | +| C1 transient | 353 ns ±0.7% | **0.74** ±2.1% | **0.91** ±1.3% | +| C2 warm singleton | 157 ns ±0.5% | 2.63 ±2.2% | 1.90 ±1.0% | +| C3 deep chain (6) | 965 ns ±0.3% | **0.53** ±0.3% | **0.72** ±0.6% | -_Across-run IQR of each side's own median (5 runs): modern-di ≤1.2%, rivals ≤2.0%. The ± on each ratio cell is a different quantity: the spread of the paired per-run ratios._ +_Across-run IQR of each side's own median (5 runs): modern-di ≤0.7%, rivals ≤1.9%. The ± on each ratio cell is a different quantity: the spread of the paired per-run ratios._ ### By-type resolution | Scenario | modern-di | vs dishka | vs wireup | |---|---|---|---| -| C1 transient | 323 ns ±1.4% | 1.34 ±1.7% | 1.50 ±1.0% | -| C2 warm singleton | 182 ns ±0.2% | 1.06 ±0.9% | 2.40 ±2.0% | -| C3 deep chain (6) | 814 ns ±0.9% | 1.82 ±1.8% | 1.26 ±0.2% | +| C1 transient | 413 ns ±0.9% | 1.37 ±0.7% | 1.54 ±0.6% | +| C2 warm singleton | 211 ns ±0.9% | **0.98** ±1.9% | 2.22 ±0.6% | +| C3 deep chain (6) | 1.03 µs ±0.8% | 1.82 ±0.8% | 1.29 ±1.9% | -_Across-run IQR of each side's own median (5 runs): modern-di ≤1.4%, rivals ≤1.0%. The ± on each ratio cell is a different quantity: the spread of the paired per-run ratios._ +_Across-run IQR of each side's own median (5 runs): modern-di ≤0.9%, rivals ≤1.3%. The ± on each ratio cell is a different quantity: the spread of the paired per-run ratios._ ### Request lifecycle (batched, published per request) | Scenario | modern-di | vs dependency-injector | vs that-depends | vs dishka | vs wireup | |---|---|---|---|---|---| -| C4 request lifecycle | 1.92 µs ±2.9% | **0.02** ±2.3% | **0.20** ±1.2% | 1.28 ±3.4% | **0.13** ±2.0% | +| C4 request lifecycle | 2.42 µs ±1.0% | **0.02** ±0.9% | **0.20** ±1.3% | 1.24 ±1.8% | **0.13** ±0.7% | -_Across-run IQR of each side's own median (5 runs): modern-di ≤2.9%, rivals ≤2.5%. The ± on each ratio cell is a different quantity: the spread of the paired per-run ratios._ +_Across-run IQR of each side's own median (5 runs): modern-di ≤1.0%, rivals ≤0.7%. The ± on each ratio cell is a different quantity: the spread of the paired per-run ratios._ ### Per-request context | Scenario | modern-di | vs dependency-injector | vs that-depends | vs dishka | vs wireup | |---|---|---|---|---|---| -| C6 context | 1.41 µs ±0.9% | **0.58** ±1.6% | **0.63** ±3.9% | 1.65 ±1.1% | 1.38 ±1.4% | +| C6 context | 1.68 µs ±0.6% | **0.55** ±1.9% | **0.60** ±0.8% | 1.56 ±1.2% | 1.27 ±1.2% | -_Across-run IQR of each side's own median (5 runs): modern-di ≤0.9%, rivals ≤4.6%. The ± on each ratio cell is a different quantity: the spread of the paired per-run ratios._ +_Across-run IQR of each side's own median (5 runs): modern-di ≤0.6%, rivals ≤1.3%. The ± on each ratio cell is a different quantity: the spread of the paired per-run ratios._ ## What the numbers show -- Against `dependency-injector`, modern-di is faster by reference on C1 (**0.76**) and - C3 (**0.54**), and far faster on the batched C4 request lifecycle (**0.02**). +- Against `dependency-injector`, modern-di is faster by reference on C1 (**0.74**) and + C3 (**0.53**), and far faster on the batched C4 request lifecycle (**0.02**). dependency-injector's C4 body calls `init_resources()`/`shutdown_resources()` every cycle in addition to resolving; the suite doesn't decompose how much of its per-request cost is that lifecycle work versus the resolve itself, so C4 should be read as a whole-lifecycle comparison, not an isolated resolve (see the caveat below). dependency-injector is still - faster on C2 warm-singleton (2.94, an implied ~48 ns cache hit against modern-di's 142 ns): + faster on C2 warm-singleton (2.63, an implied ~60 ns cache hit against modern-di's 157 ns): its hit is a C-level slot read on a Cython-compiled core, where modern-di's is a Python - dict lookup behind an override guard. Pure Python does not reach ~48 ns, so this cell is + dict lookup behind an override guard. Pure Python does not reach ~60 ns, so this cell is expected to stay above 1.0 however much of modern-di's own overhead is removed. -- Against `that-depends`, modern-di now leads by reference on C1 by 11% (**0.89**, - interquartile width 1.6%) — the first publication where that cell's spread does not cover - 1.00. Treat it as a real but modest lead: it has sat on both sides of parity across - publications (1.08, 1.12, 0.98, 0.98, 0.97, now 0.89), and this move is explained by - modern-di getting ~28 ns faster rather than by that-depends changing — its implied C2 - absolute is unchanged at ~67 ns. modern-di is also faster on C3 (**0.72**) and far faster - on C4 (**0.20**). that-depends remains faster on C2 warm-singleton (2.13); the suite does - not decompose its `resolve_sync` cache-hit path, so no mechanism is asserted here. -- Against `dishka` and `wireup` on the by-type table, modern-di is slower on all three - synchronous scenarios (1.06–1.82x dishka, 1.26–2.40x wireup). Both inline dependency calls - into `exec`-generated source, which removes a per-node function-call frame that modern-di - keeps; modern-di does not generate code (a - [documented non-goal](design-decisions.md#non-goals)). That mechanism is asserted **for - dishka only**, where the ratio grows with node count as a per-node cost should: 1.34 on C1 - (2 nodes) against 1.82 on C3 (6 nodes). wireup's ratio moves the other way (1.50 on C1 down - to 1.26 on C3), which a per-node cost does not explain, and the suite does not decompose - wireup's resolve further — so no mechanism is claimed for it. -- The two rivals also diverge on C2, and modern-di is now **level with dishka** there (1.06, - interquartile width 0.9%). Most of what remains is the fixed by-type lookup: dividing - modern-di's **by-reference** C2 (142 ns) by dishka's implied C2 absolute (182 ÷ 1.06 = - 171.7 ns) gives **0.83** — on the same by-reference basis modern-di's warm hit is now the - faster of the two. wireup's largest gap is C2 (2.40), and the same division still gives 1.87 - (142 ÷ 75.8 ns), so most of wireup's C2 gap exists at the by-reference baseline and the - suite does not decompose the rest. Both are **cross-basis figures**, comparing modern-di by - reference against a rival that can only be measured by type, which is exactly the mixed - basis the split tables above avoid; they are offered as decomposition, not as published - ratios. -- On C6 (per-request context) modern-di is faster than `dependency-injector` (**0.58**) and - `that-depends` (**0.63**), and slower than `dishka` (1.65) and `wireup` (1.38). The direction +- Against `that-depends`, modern-di leads by reference on C1 by 9% (**0.91**, + interquartile width 1.3%) — the second consecutive publication where that cell's spread does + not cover 1.00. Treat it as a real but modest lead: it has sat on both sides of parity across + publications (1.08, 1.12, 0.98, 0.98, 0.97, 0.89, now 0.91). modern-di is also faster on + C3 (**0.72**) and far faster on C4 (**0.20**). that-depends remains faster on C2 + warm-singleton (1.90, down from 2.13); the suite does not decompose its `resolve_sync` + cache-hit path, so no mechanism is asserted for the remaining gap. +- Against `dishka` and `wireup` on the by-type table, modern-di is slower on the two wiring + scenarios (1.37 and 1.82 against dishka, 1.54 and 1.29 against wireup) and **faster than + dishka on C2** (**0.98**). Both rivals inline dependency calls into `exec`-generated source, + which removes a per-node function-call frame that modern-di keeps; modern-di does not + generate code (a [documented non-goal](design-decisions.md#non-goals)). That mechanism is + asserted **for dishka only**, where the ratio grows with node count as a per-node cost + should: 1.37 on C1 (2 nodes) against 1.82 on C3 (6 nodes). wireup's ratio moves the other + way (1.54 on C1 down to 1.29 on C3), which a per-node cost does not explain, and the suite + does not decompose wireup's resolve further — so no mechanism is claimed for it. +- **The C2 cell against dishka has crossed below 1.0** (**0.98**, interquartile width 1.9%), + the first publication where modern-di's warm singleton is faster than dishka's *on dishka's + own by-type basis*. The spread does not cover 1.00, but it comes close enough that the + honest reading is "level, marginally ahead" rather than a decisive lead. On the + by-reference basis the margin is wider: dividing modern-di's **by-reference** C2 (157 ns) by + dishka's implied C2 absolute (211 ÷ 0.98 = 215.3 ns) gives **0.73**. wireup's largest gap is + still C2 (2.22), and the same division gives 1.65 (157 ÷ 95.0 ns), so most of wireup's C2 gap + exists at the by-reference baseline and the suite does not decompose the rest. Both are + **cross-basis figures**, comparing modern-di by reference against a rival that can only be + measured by type, which is exactly the mixed basis the split tables above avoid; they are + offered as decomposition, not as published ratios. +- On C6 (per-request context) modern-di is faster than `dependency-injector` (**0.55**) and + `that-depends` (**0.60**), and slower than `dishka` (1.56) and `wireup` (1.27). The direction matches the by-type table — the two codegen frameworks lead, the two others trail — but the cells are **not** on one basis: each framework supplies the request value through its own idiom, and two of those are structural analogs rather than equivalents (see the caveat below). No mechanism is asserted for the gaps; the suite does not decompose any framework's context lookup. -- On C4 (request lifecycle), the corrected batching does not *remove* the ~27 µs asyncio floor +- On C4 (request lifecycle), the corrected batching does not *remove* the ~35 µs asyncio floor — it amortizes it. The guard tier's `test_g7c_event_loop_floor_control` times the same batch - shape with an empty body and puts the residual at **~0.31 µs per request** still inside every - C4 cell (~16% of the equivalent guard-tier batch, ~16% of modern-di's C4 figure), shared + shape with an empty body and puts the residual at **~0.35 µs per request** still inside every + C4 cell (~15% of the equivalent guard-tier batch, ~15% of modern-di's C4 figure), shared identically by all five frameworks. Before batching, that floor was ~93% of every cell and compressed the real differences toward 1.0; the page used to read modern-di as "level with dishka (1.00)" on that basis. With the floor amortized, dishka is measurably **faster** than - modern-di here (1.28 — modern-di is the slower side of that cell), not tied. modern-di + modern-di here (1.24 — modern-di is the slower side of that cell), not tied. modern-di remains far faster than that-depends, dependency-injector, and wireup on this scenario. -**What moved in this publication.** A library change, not a benchmark one: the warm-hit path -lost two Python method frames. `ProvidersRegistry.resolver_for` and -`CacheRegistry.fetch_cache_item` both open with a dict lookup that hits and returns; both are -now inlined at their call sites, with the method called only on a miss, where it still owns the -cycle guard, the memo write and the `setdefault` that makes concurrent first-resolvers share one -`CacheItem`. Measured in isolation, that removes ~42 ns from a warm hit. - -Because the first of the two sits in `resolve_provider`, every top-level resolve benefits, not -just cached ones. C2 by reference falls from 186 ns to **142 ns** and C1 from 301 ns to -**279 ns**; the C2 ratios move 3.88 → **2.94** against dependency-injector, 2.80 → **2.13** -against that-depends, 1.32 → **1.06** against dishka, and 3.02 → **2.40** against wireup. Both -rivals whose implied C2 absolutes this page prints are unchanged (~48 ns and ~67 ns), which is -the check that the movement is modern-di's and not drift. - -The bound stated when this was planned still holds: it trims the cell, it does not close it. -dependency-injector's ~48 ns hit is a C-level slot read on a Cython core. A third step — -closing an APP-scoped resolver over its `CacheItem` to reach ~16 ns — is deliberately not taken, -because it would require the providers registry to reference its root container, reintroducing -the reference cycle removed in 3.1.1. - -Every other cell moved within run-to-run noise, and several by more than their own published -`±` — that annotation bounds dispersion within one publication's five runs, not drift between -publications, so do not read a cell's `±` as a bound on how much it may move next time. +**Compare this publication's ratios to the last one, not its absolutes.** Every absolute figure +here is 23–29% higher than 2026-07-28's — **for all five frameworks**, on the same machine, the +same macOS 26.5 and CPython 3.14.6, and the same pinned rival versions. The implied rival +absolutes this page prints moved as a block: dependency-injector's C2 cache hit ~48 → **~60 ns**, +that-depends' ~67 → **~83 ns**, dishka's ~172 → **~215 ns**, wireup's ~76 → **~95 ns**. Nothing +in any of those libraries changed; the machine was simply slower on the day. Ratios are paired +within each run, so they absorb that shift — the absolute column does not, which is exactly why +this page leads with ratios and warns that absolutes will differ on your machine. + +**What moved on top of that shift is C2 and C6 — the two paths 3.2.0 touched.** + +- **C2 warm singleton.** modern-di's by-reference cell rose only 10.6% (142 → **157 ns**) + against a pack that rose 24–25%. Had it not changed it would sit near 176 ns; 3.2.0 removed a + `MAKE_CELL` from the cached resolver's prologue, worth −11.3% measured in isolation, and + 176 × 0.887 = 156. The ratios follow: 2.94 → **2.63** against dependency-injector, + 2.13 → **1.90** against that-depends, 1.06 → **0.98** against dishka, and 2.40 → **2.22** + against wireup. +- **C6 per-request context.** modern-di rose 19% (1.41 → **1.68 µs**) against a pack that rose + 25–29%. 3.2.0 front-guarded the override lookup on the context-kwarg path (−6.0% measured in + isolation) and trimmed `find_context`. Ratios: 0.58 → **0.55**, 0.63 → **0.60**, + 1.65 → **1.56**, 1.38 → **1.27**. +- **C1, C3 and C4 are the control**, and they behave like one: 3.2.0 changed nothing on the + plain transient, deep-chain, or lifecycle paths, and their twelve ratio cells moved by at most + 0.04 with no consistent direction — four up, three down, five unchanged. That is + re-measurement scatter. C2's four cells moved 0.08–0.31 and C6's 0.03–0.11, every one of them + toward modern-di, which scatter does not do. + +**One 3.2.0 improvement is invisible here.** An `Alias` hop went from four Python frames to one, +measured ad hoc at ~322 → ~252 ns per hop. The comparative suite has no alias scenario, so no +cell on this page reflects it. + +The bound stated when the warm-hit work was planned still holds: it trims the cell, it does not +close it. dependency-injector's ~60 ns hit is a C-level slot read on a Cython core. A further +step — closing an APP-scoped resolver over its `CacheItem` — is deliberately not taken, because +it would require the providers registry to reference its root container, reintroducing the +reference cycle removed in 3.1.1. + +This publication is itself the reason not to read a cell's `±` as a bound on how much it may +move next time: that annotation bounds dispersion within one publication's five runs, not drift +between publications, and here the between-publication drift is an order of magnitude larger +than any cell's `±`. **The C4 gain recorded at 3.1.1 was a library fix**, and it stands: every `Container` used to store itself in its own `_scope_map`, making it a reference cycle that reference counting could @@ -254,6 +272,17 @@ still owns the cycle guard, the memo write, and the `setdefault` that makes conc first-resolvers share one `CacheItem`. Worth ~42 ns on a warm hit, and because the first sits in `resolve_provider` it applies to every top-level resolve rather than only cached ones. +3.2.0 trimmed three more paths rather than one. The cached resolver's cold-miss thunk is now +built with `functools.partial` instead of a lambda closing over the target container: a closure +promotes that variable to a cell for the *whole* resolver, so `MAKE_CELL` ran in the prologue on +every call — including the warm hit that returns two lines later and the override hit that never +reaches it (−11.3% on a warm hit). The context-kwarg path front-guards its override lookup on +`has_overrides` (−6.0%), which is the path every framework integration takes for its per-request +values. And an `Alias` stopped routing through `Alias._find_source` and `Container.resolve_provider` +on every hop: it now inlines both lookups and calls its source's compiled resolver directly, one +Python frame per hop instead of four (~322 → ~252 ns). The alias change has no cell on this page +— there is no alias scenario in the comparative suite. + ## Reproduce it yourself ```bash