perf(jit): reuse native timing storage and settle Tier 2 benchmarks - #19
Merged
Conversation
2 tasks
huacnlee
added a commit
that referenced
this pull request
Sep 6, 2026
huacnlee
added a commit
that referenced
this pull request
Sep 6, 2026
### Issue # (if available) Follow-up to merged #19; review this diff against `main`. ### Description of changes Consolidate `observe_type` and `observe_call` into one B-tree entry lookup, preserving occupied-at-capacity updates, rejected-entry accounting, exact feedback versions, and independent call-slot growth. Cache baseline tier state once after maintenance in `record_hot`; successful direct refresh returns before that cached value is reused. Add three characterization tests covering the capacity, generation, version and megamorphic edges. Deliberate capacity and arity mutations fail the corresponding tests. Validation: 518 release runtime tests, 82 targeted ASAN/LSAN tests, workspace Clippy and formatting pass; independent code review found no issues. ### Measured performance **Compared with `a38bd01` (merged #18), the cumulative #19 + #20 changes improve generic-call Tier 1 speed by 19.3%, generic-call automatic by 18.1%, and recursive Tier 1 by 18.7%.** The measured candidate is `99a1e78`; the conflict-resolution merge retains that exact content tree. These are cumulative results, not the incremental gain of #20 over the current `main` containing #19. The values below are steady-state latency converted to speed relative to the named baseline: **1.00x is parity; higher is faster**. Intervals are paired 95% bootstrap confidence intervals. | Workload | Requested mode / actual execution | Speed vs. `a38bd01` | Speed improvement | 95% interval | |---|---|---:|---:|---:| | Generic calls | Tier 1 | 1.193x | 19.3% faster | 1.190x–1.197x | | Generic calls | Tier 2 / actually Tier 1 | 1.163x | 16.3% faster | 1.160x–1.165x | | Generic calls | Automatic / actually Tier 1 | 1.181x | 18.1% faster | 1.175x–1.185x | | Recursive Fibonacci | Tier 1 | 1.187x | 18.7% faster | 1.183x–1.189x | | Scalar loop | Tier 2 | 1.035x | 3.5% faster | 1.028x–1.038x | | Scalar loop | Automatic | 1.041x | 4.1% faster | 1.037x–1.047x | Controls and limitations: - All direct-call modes, property-access modes, and interpreter comparisons are **statistically tied**. For example, direct-call Tier 2 ranges from 1.9% slower to 0.2% faster; property automatic ranges from 0.24% slower to 0.15% faster. - Scalar Tier 1 is tied, between 2.3% slower and 0.6% faster. Recursive requested Tier 2 and automatic are also tied; automatic recursion has zero native entries and remains an interpreter-path control. - The scalar-loop improvements have no isolated causal attribution. These measurements neither establish an overall no-regression result nor replace the complete clean-source acceptance matrix. - Later rooted-property-guard changes and their measurements are outside this PR and are not included in this table. Method: five discarded warmup processes, 30 fresh-process latency pairs per mode, ten throughput windows of at least one second per engine/mode, and 10,000 paired bootstrap resamples, pinned to CPU 0. Both runtimes use the identical corrected benchmark harness. Engine order is balanced independently of mode order (15/15 first positions for latency; 5/5 for throughput). Checksums, native entry/exit balance, and executable/source hashes were audited. The recorded runs use dirty-source diagnostic provenance; throughput windows count fresh workers including setup and warmup, not JavaScript operations. Raw samples, exact driver/harness source, hashes, and intervals: [generic calls](https://github.com/longbridge/quickjs-jit/blob/99a1e78/benchmarks/results/m3-feedback-lookups-generic-call-entry-paired.json), [direct calls](https://github.com/longbridge/quickjs-jit/blob/99a1e78/benchmarks/results/m3-feedback-lookups-call-heavy-paired.json), [recursive Fibonacci](https://github.com/longbridge/quickjs-jit/blob/99a1e78/benchmarks/results/m3-feedback-lookups-fibonacci-recursive-paired.json), [scalar loop](https://github.com/longbridge/quickjs-jit/blob/99a1e78/benchmarks/results/m3-feedback-lookups-scalar-loop-paired.json), [property access](https://github.com/longbridge/quickjs-jit/blob/99a1e78/benchmarks/results/m3-feedback-lookups-property-heavy-paired.json). Archive raw evidence and a property profile: helper-frame validation remains the main sampled property cost. Document the next bounded ownership optimization and its safety constraints; property lowering itself is unchanged in this PR. ### Checklist - [x] Added change to the changelog - [x] Created unit tests for my feature if needed
2 tasks
huacnlee
added a commit
that referenced
this pull request
Sep 6, 2026
### Issue # (if available) N/A ### Description of changes Archive a fresh comparison of repository-pinned QuickJS-ng, quickjs-jit automatic mode, and Bun 1.4.0 default configuration after #19 and #20 merged into clean main `47aeb11`. Include forced Tier 1/2 diagnostics, raw samples, methodology/provenance, CSV, confidence intervals, charts, and an evidence-based next-stage optimization roadmap. No runtime behavior changes. Automatic JIT is faster than the interpreter in 8 workloads, statistically tied in 1, and slower in 13, based on per-workload paired 95% intervals. Selected results below express **speed relative to the QuickJS-ng interpreter** (1x = parity): | Workload | JIT automatic speed [95% CI] | Bun default speed | |---|---:|---:| | scalar-loop | 24.024x [23.382, 24.316] | 47.26x | | float64-dense | 7.819x [7.708, 7.895] | 6.27x | | property-heavy | 0.268x [0.267, 0.271] | 3.87x | | generic-call-entry | 0.135x [0.134, 0.136] | 57.85x | | arrays-typed | 0.656x [0.654, 0.658] | 10.41x | These are current-protocol results, not equally warmed peak-engine measurements: QuickJS and Bun differ in process-internal warmup and host timing boundaries. Native counters include warmup. Bun uses a recorded launcher that removes the runner's `--smol` flag. The unmerged property candidate `9ccda01` is excluded. The roadmap prioritizes aligned measurement and steady-state counters, property and generic-call costs, automatic tiering/fallback overhead, then cold start and library hotspots. Property and generic-call stages propose a paired speed lower bound of 1.5x versus the same-harness main baseline, with eventual interpreter parity or better. Evidence: [full comparison](benchmarks/results/main-47aeb11-engines.md), [raw samples](benchmarks/results/main-47aeb11-engines.json), [methodology](benchmarks/results/main-47aeb11-methodology.json), [next targets](docs/PERFORMANCE_NEXT.md). Validation: - Completed 22 workloads × 5 modes × 30 latency samples, with 5 discarded warmup processes and 10 one-second throughput windows per workload/mode. - Audited all 3,300 latency samples, checksums, entry/exit counters, source/binary hashes, derived results, links, and chart rendering; independent report review found no substantive issue. - Staged whitespace check passes; CSV/SVG formatting normalization preserves data/path tokens. No runtime tests rerun for this documentation/data-only commit. - **Performance acceptance remains failing** (runner exit 2 after complete output): compute speed lower bound is below 5x, automatic decision evidence is missing, and startup/definition-eval proxy budgets fail. The gpui-shell gate is inconclusive because external host evidence is absent. Raw gate output is preserved. ### Checklist - [x] Changelog assessed: not applicable to benchmark evidence and planning documents. - [x] Unit tests assessed: no runtime changes; artifact and data validation completed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue # (if available)
Follow-up to #18.
Description of changes
Native entry timing used a hash map of per-function vectors, adding lookups and allocations to callback bookkeeping. Replace it with a shared LIFO stack while preserving the original function key and tier through recursion, retirement, OSR, and all exit kinds. Empty or mismatched exits preserve other active records. After warming one slot, inserting timers for 100 sequential distinct functions now performs zero allocations instead of 105.
The performance controls also exposed forced Tier 2 timing before its caller finished compiling. Add an explicit publication requirement for call-heavy, stable compilation settling, and rejection of compilation during required-Tier2 timing. Refresh the changed script checksum and three pre-existing stale local manifest hashes.
Archive profiles and four paired comparisons against merged M3 with identical corrected harnesses and balanced engine order. Generic automatic measures 1.011x baseline speed, with its interval only narrowly excluding parity. Generic Tier 1 and recursive Tier 1 are statistically tied with broad intervals; direct-call modes are also tied. These dirty-source diagnostics do not establish overall speedup or complete the broader clean-source performance acceptance gate. Documentation retains the original readiness-defect evidence and explains the superseded measurement limitations.
Validation:
Checklist