Skip to content

Gate server SDK overhead in CI and reduce per-request cost under span streaming #25678

Open

Description

@mydea

Server SDK overhead has no regression gate. The timing-based node-overhead-gh-action was removed as too flaky (#20246). Kiln (#23302) compares versions on demand, but nothing runs on PRs. Regressions such as #24820 (CPU up from 10.56 to 10.74+) are found by users and are hard to bisect afterwards.

Now that span streaming is the default, a few per-request and per-span paths cost noticeably more than they need to. Local measurements on Node 24.21, using an in-process fetch against an http server so client and server instrumentation both run:

Setup Time per request Scope clones per request spanStart per request
No SDK 0.056 ms 0 0
init() without tracesSampleRate 0.088 ms 4 0
tracesSampleRate: 0 0.130 ms 6 2
tracesSampleRate: 1 0.183 ms 6 2

Proposal

  1. A CI gate based on counts, not wall-clock time. Add a dev-packages suite that records stable counters for a few scenarios (raw http, express, a span-heavy loop):

    • scope clones per request
    • spanStart per request
    • bytes retained per buffered trace after gc()
    • modules loaded by require('@sentry/node')
    • heap after N requests, to catch leaks

    It should fail when a counter grows past a small threshold. Timings stay informational only. The point is to avoid the flakiness that got the old action removed.

  2. A cheaper sampled-out path. A request that is sampled out still costs about 70% as much as a sampled one.

    • The http server subscription builds the full attribute set (headers, URL parsing, filtering) before any sampling decision (server-utils server-subscription.ts).
    • An unsampled root still becomes a full SentrySpan (core/src/tracing/trace.ts, sampleSpan → new SentrySpan) and emits spanStart.

    When no tracesSampler or beforeSampling hook is registered, the decision depends only on tracesSampleRate, the parent decision and sampleRand, so it can be made first. On "drop", skip building attributes and return a non-recording span. That span must still record the dropped-span outcome and keep propagation working. With a sampler registered, behavior stays as it is.

  3. Less work per span in captureSpan. Under streaming this runs on every span end. It calls getCombinedScopeData (copying the breadcrumb and event-processor arrays and deep-merging request data) and spanToJSON(segmentSpan) for every child (core/src/tracing/spans/captureSpan.ts). However, applyCommonSpanAttributes only reads user, attributes, contexts and the segment's name and id. A narrow merge plus direct reads would give the same output.

  4. A bounded span buffer. SpanBuffer.add (core/src/tracing/spans/spanBuffer.ts) creates one bucket and one setTimeout per trace. It caps size per trace but has no global cap. Each buffered span keeps its segment span alive, along with that span's scopes and request data. Locally, about 7 KB was retained per trace, and 20k buffered traces held about 141 MB until they were flushed. Proposed: global limits on buffered traces and bytes (flush the oldest bucket when reached), one shared unref'd interval instead of per-trace timers, and optionally dropping scopes and request data from the segment span once it ends.

None of these change public API or the data that is sent. Related: #15861.

Activity

  1. linear-code commented on Oct 23, 2026

    @linear-code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions