docs(performance): republish the comparative table at 3.3.0 - #423
Merged
Conversation
Second publication of the day on the same machine, same macOS and CPython, and the same four pinned rival versions -- so the attribution is unusually clean: every rival's implied absolute is unchanged (dependency-injector's C2 hit 59.7 -> 59.5 ns, that-depends 82.6 -> 82.6, dishka 215.3 -> 214.8, wireup 95.0 -> 94.6). Every cell that moved is 3.3.0's. The by-type table crossed over: modern-di is now faster than dishka on C1 (0.91) and C2 (0.81) and faster than wireup on C3 (0.90), having been slower than both on all three one publication earlier. dishka keeps C3 (1.30), the deepest graph, which is where its per-node codegen advantage should show. C2 by reference is unchanged at 157 ns, and that is the control -- a warm cached hit touches neither the arity ladder (cold-miss path) nor the by-type inline. It was predicted flat before the run. The by-type surcharge fell from 54-65 ns to 21/17/23 ns, which is the by-type inline showing up: what remains is close to the bare registry dict lookup. Also files the staleness pattern: this page went stale twice today, and nothing detects it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: 76a0f21 | Previous: fc0408b | Ratio |
|---|---|---|---|
benchmarks/test_guard_by_type.py::test_g16_resolve_by_type |
3893158.342595616 iter/sec (stddev: 1.7426908785202336e-8) |
2974477.053465562 iter/sec (stddev: 1.5184437632491462e-8) |
0.76 |
benchmarks/test_guard_by_type.py::test_g17_resolve_by_type_large_registry |
4013370.302383028 iter/sec (stddev: 1.2902682187892631e-8) |
3078699.3443489377 iter/sec (stddev: 8.485649643283638e-9) |
0.77 |
benchmarks/test_guard_cold.py::test_g8_cold_first_resolve |
25480.461736045043 iter/sec (stddev: 0.000004554918562579273) |
26375.17618110752 iter/sec (stddev: 0.000006208465271841761) |
1.04 |
benchmarks/test_guard_cold.py::test_g8b_cold_first_resolve_cached |
19360.90757388759 iter/sec (stddev: 0.000004015789091386459) |
20736.251386595064 iter/sec (stddev: 0.000004058615252225284) |
1.07 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[1] |
610.255989031789 iter/sec (stddev: 0.0000803826807575259) |
428.5147813411539 iter/sec (stddev: 0.00010254622477433882) |
0.70 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[2] |
555.4882992532574 iter/sec (stddev: 0.00006540920413141217) |
376.77106674117016 iter/sec (stddev: 0.00035543855792948654) |
0.68 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[4] |
505.2054749461316 iter/sec (stddev: 0.00006422642650404175) |
377.2858800396892 iter/sec (stddev: 0.000034144260395994674) |
0.75 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[1] |
2469.6434005752926 iter/sec (stddev: 0.0000311348829465428) |
2424.6352641391813 iter/sec (stddev: 0.00003653454738877614) |
0.98 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[2] |
1900.457669802246 iter/sec (stddev: 0.00003260751184710583) |
1783.8086630934508 iter/sec (stddev: 0.000060021848060854465) |
0.94 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[4] |
1289.0585132324027 iter/sec (stddev: 0.000039400770385496954) |
1216.5765119266011 iter/sec (stddev: 0.00003353887421580802) |
0.94 |
benchmarks/test_guard_lifecycle.py::test_g6_build_child_container |
858846.0715966525 iter/sec (stddev: 7.776539621387235e-8) |
745182.1505544861 iter/sec (stddev: 4.8974923964532066e-8) |
0.87 |
benchmarks/test_guard_lifecycle.py::test_g6b_build_child_container_auto_scope |
828916.7612138398 iter/sec (stddev: 7.410346972878011e-8) |
693144.6057681259 iter/sec (stddev: 3.80363466658199e-8) |
0.84 |
benchmarks/test_guard_lifecycle.py::test_g7_request_lifecycle_batch |
2415.961199229644 iter/sec (stddev: 0.000020029374947003524) |
2264.7112474654778 iter/sec (stddev: 0.000008868774909574978) |
0.94 |
benchmarks/test_guard_lifecycle.py::test_g7c_event_loop_floor_control |
78145.42560886804 iter/sec (stddev: 0.000001380123055744086) |
61202.45202487152 iter/sec (stddev: 0.0000015397182589624615) |
0.78 |
benchmarks/test_guard_lifecycle.py::test_g13_teardown_at_scale |
51243.31095707005 iter/sec (stddev: 0.0000017157358628023505) |
43473.18267474752 iter/sec (stddev: 0.0000016264450467753485) |
0.85 |
benchmarks/test_guard_resolve.py::test_g1_transient_resolve |
2728012.2046210202 iter/sec (stddev: 1.962148625665897e-8) |
2096784.4238307995 iter/sec (stddev: 2.210574219125219e-8) |
0.77 |
benchmarks/test_guard_resolve.py::test_g2_cached_resolve |
4846824.590798828 iter/sec (stddev: 9.727712768382865e-9) |
3319457.447961124 iter/sec (stddev: 9.785522261164457e-9) |
0.68 |
benchmarks/test_guard_resolve.py::test_g3_deep_chain |
966867.4350755507 iter/sec (stddev: 3.6517743223403184e-8) |
754371.5928102633 iter/sec (stddev: 4.737505809434614e-8) |
0.78 |
benchmarks/test_guard_resolve.py::test_g4_wide_resolve |
522519.0150541757 iter/sec (stddev: 1.276199135017494e-7) |
406521.25130448455 iter/sec (stddev: 1.0282302454945122e-7) |
0.78 |
benchmarks/test_guard_resolve.py::test_g5_cross_scope |
2064073.2721239494 iter/sec (stddev: 3.1389763424671634e-8) |
1677547.0973457035 iter/sec (stddev: 2.549541288127291e-8) |
0.81 |
benchmarks/test_guard_resolve.py::test_g9_context_resolve |
1069286.4469778233 iter/sec (stddev: 4.659274870543929e-8) |
871244.5981753406 iter/sec (stddev: 6.035691811674111e-8) |
0.81 |
benchmarks/test_guard_resolve.py::test_g12_override_active_resolve |
651748.2576741417 iter/sec (stddev: 8.409676982055975e-8) |
497874.49908264866 iter/sec (stddev: 6.248013077045816e-8) |
0.76 |
benchmarks/test_guard_resolve.py::test_g18_alias_hop |
3030850.9714095206 iter/sec (stddev: 2.670245951110558e-8) |
2304425.854946363 iter/sec (stddev: 9.1771260444836e-9) |
0.76 |
benchmarks/test_guard_validate.py::test_g10_validate_deep_chain |
29199.849375398873 iter/sec (stddev: 0.000002629174114152341) |
30927.208217573934 iter/sec (stddev: 0.0000024290139961712183) |
1.06 |
benchmarks/test_guard_validate.py::test_g11_validate_wide |
18721.022197837487 iter/sec (stddev: 0.0000032935957396834396) |
18179.72751316095 iter/sec (stddev: 0.000003903350506011211) |
0.97 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
docs/introduction/performance.mdreported 3.2.0 and predated 3.3.0's work, which moved the by-type column by 17-34%. It is the page carrying our named comparative claims against four other frameworks, so a stale one understates the library publicly.Design
Tables regenerated with
just bench-report(5 paired runs) and published verbatim. Run twice, independently, before writing any prose — the two passes agree within noise on every cell.The attribution is unusually clean this time. This is the second publication of the day: same machine, same macOS 26.5, same CPython 3.14.6, same four pinned rival versions, a few hours apart. Every rival's implied absolute is unchanged — dependency-injector's C2 cache hit 59.7 → 59.5 ns, that-depends 82.6 → 82.6, dishka 215.3 → 214.8, wireup 95.0 → 94.6. Nothing drifted, so every cell that moved is 3.3.0's. The 3.2.0 publication this morning could not make that claim: the machine had shifted 23-29% and the prose had to reason around it.
The by-type table crossed over
modern-di is now faster than dishka on C1 (0.91) and C2 (0.81), and faster than wireup on C3 (0.90) while level on C1 (1.01). One publication earlier it was slower than both on all three. dishka keeps a clear lead on C3 (1.30) — the deepest graph, which is exactly where the per-node call frame that
exec-inlined source removes should tell, and the mechanism this page has always asserted for dishka specifically.A framework that generates no code overtaking two that do is a strong claim, which is why I ran it twice before writing it down.
The control held
That last row was predicted flat before the run and is flat by construction: a warm cached hit returns before the cold-miss path the arity ladder touches, and never enters the by-type path the inline touches.
The by-type surcharge collapsed
The page said by-type adds "60 ns on C1, 54 on C2, 65 on C3". It is now 21 / 17 / 23 ns — the by-type inline removing
resolve_provider's frame, leaving something close to the bare registry dict lookup. That is why the by-type column moved further than the by-reference one.Non-goals
Also filed
planning/deferred/2026-08-03-performance-page-staleness.md. This page went stale twice today — at 3.2.0 and again at 3.3.0 — and both times a person noticed rather than a check. The one-line check is easy; where it belongs is not, because the republish needs the comparative environment thatlint-cidoes not install, so there is a legitimate window between a release and the republish. The item lays out the candidates and says: if it happens a third time, stop treating it as a checklist item.Verification
just lint-ci: clean.just docs-build: builds, no broken links or nav warnings (network was available this time, unlike on #415).just check-links: OK.Every ratio, absolute and derived figure in the prose was recomputed from the published run; I grepped for surviving 3.2.0-era numbers and fixed one unrelated inconsistency I introduced this morning — the asyncio floor was updated to ~35 µs in one place and left at ~27 µs in another.
Before merging
just lint-cipasses;just test-ciunaffected (docs only).