perf(jit): consolidate feedback and tier-state lookups - #20
Merged
Conversation
2 tasks
huacnlee
added a commit
that referenced
this pull request
Sep 6, 2026
### Issue # (if available) N/A ### Description of changes Archive a fresh comparison of repository-pinned QuickJS-ng, quickjs-jit automatic mode, and Bun 1.4.0 default configuration after #19 and #20 merged into clean main `47aeb11`. Include forced Tier 1/2 diagnostics, raw samples, methodology/provenance, CSV, confidence intervals, charts, and an evidence-based next-stage optimization roadmap. No runtime behavior changes. Automatic JIT is faster than the interpreter in 8 workloads, statistically tied in 1, and slower in 13, based on per-workload paired 95% intervals. Selected results below express **speed relative to the QuickJS-ng interpreter** (1x = parity): | Workload | JIT automatic speed [95% CI] | Bun default speed | |---|---:|---:| | scalar-loop | 24.024x [23.382, 24.316] | 47.26x | | float64-dense | 7.819x [7.708, 7.895] | 6.27x | | property-heavy | 0.268x [0.267, 0.271] | 3.87x | | generic-call-entry | 0.135x [0.134, 0.136] | 57.85x | | arrays-typed | 0.656x [0.654, 0.658] | 10.41x | These are current-protocol results, not equally warmed peak-engine measurements: QuickJS and Bun differ in process-internal warmup and host timing boundaries. Native counters include warmup. Bun uses a recorded launcher that removes the runner's `--smol` flag. The unmerged property candidate `9ccda01` is excluded. The roadmap prioritizes aligned measurement and steady-state counters, property and generic-call costs, automatic tiering/fallback overhead, then cold start and library hotspots. Property and generic-call stages propose a paired speed lower bound of 1.5x versus the same-harness main baseline, with eventual interpreter parity or better. Evidence: [full comparison](benchmarks/results/main-47aeb11-engines.md), [raw samples](benchmarks/results/main-47aeb11-engines.json), [methodology](benchmarks/results/main-47aeb11-methodology.json), [next targets](docs/PERFORMANCE_NEXT.md). Validation: - Completed 22 workloads × 5 modes × 30 latency samples, with 5 discarded warmup processes and 10 one-second throughput windows per workload/mode. - Audited all 3,300 latency samples, checksums, entry/exit counters, source/binary hashes, derived results, links, and chart rendering; independent report review found no substantive issue. - Staged whitespace check passes; CSV/SVG formatting normalization preserves data/path tokens. No runtime tests rerun for this documentation/data-only commit. - **Performance acceptance remains failing** (runner exit 2 after complete output): compute speed lower bound is below 5x, automatic decision evidence is missing, and startup/definition-eval proxy budgets fail. The gpui-shell gate is inconclusive because external host evidence is absent. Raw gate output is preserved. ### Checklist - [x] Changelog assessed: not applicable to benchmark evidence and planning documents. - [x] Unit tests assessed: no runtime changes; artifact and data validation completed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue # (if available)
Follow-up to merged #19; review this diff against
main.Description of changes
Consolidate
observe_typeandobserve_callinto one B-tree entry lookup, preserving occupied-at-capacity updates, rejected-entry accounting, exact feedback versions, and independent call-slot growth. Cache baseline tier state once after maintenance inrecord_hot; successful direct refresh returns before that cached value is reused.Add three characterization tests covering the capacity, generation, version and megamorphic edges. Deliberate capacity and arity mutations fail the corresponding tests. Validation: 518 release runtime tests, 82 targeted ASAN/LSAN tests, workspace Clippy and formatting pass; independent code review found no issues.
Measured performance
Compared with
a38bd01(merged #18), the cumulative #19 + #20 changes improve generic-call Tier 1 speed by 19.3%, generic-call automatic by 18.1%, and recursive Tier 1 by 18.7%. The measured candidate is99a1e78; the conflict-resolution merge retains that exact content tree. These are cumulative results, not the incremental gain of #20 over the currentmaincontaining #19.The values below are steady-state latency converted to speed relative to the named baseline: 1.00x is parity; higher is faster. Intervals are paired 95% bootstrap confidence intervals.
a38bd01Controls and limitations:
Method: five discarded warmup processes, 30 fresh-process latency pairs per mode, ten throughput windows of at least one second per engine/mode, and 10,000 paired bootstrap resamples, pinned to CPU 0. Both runtimes use the identical corrected benchmark harness. Engine order is balanced independently of mode order (15/15 first positions for latency; 5/5 for throughput). Checksums, native entry/exit balance, and executable/source hashes were audited. The recorded runs use dirty-source diagnostic provenance; throughput windows count fresh workers including setup and warmup, not JavaScript operations.
Raw samples, exact driver/harness source, hashes, and intervals:
generic calls,
direct calls,
recursive Fibonacci,
scalar loop,
property access.
Archive raw evidence and a property profile: helper-frame validation remains the main sampled property cost. Document the next bounded ownership optimization and its safety constraints; property lowering itself is unchanged in this PR.
Checklist