Skip to content

P1: compiled runtime logging lifecycle for OpenTelemetry and Loki #204

Description

@07prajwal2000

Problem

The compiled request path creates logging infrastructure on every request. setupContextVars mutates shared integration configuration with route/project IDs, OpenTelemetryLogs creates a provider/exporter and force-flushes each log, and Loki creates a transport per request. Providers, exporters, timers, and sockets are never shut down. Concurrent requests can race and mislabel logs.

Scope

  • Cache logger, OpenTelemetry provider/exporter, and Loki transport per integration in the execution thread.
  • Treat integration configuration as immutable; attach project and route identifiers as per-log attributes.
  • Batch exports; flush and close all resources during worker shutdown.
  • On integration configuration change, atomically replace the cached logging resources and close the retired provider after it drains.
  • Define the boundary with Admin telemetry worker: trace transport, JetStream stream and span consumer #195 so compiled-worker logs can use the existing telemetry path without creating an exporter for each request.
  • Add lifecycle and concurrency tests for labeling, replacement, batching, and shutdown.

Acceptance

  • Concurrent routes cannot cross-label logs.
  • A stable integration creates bounded providers/transports under sustained load.
  • Log exports batch normally and all resources flush/close on shutdown.
  • Replacing integration config eventually closes the old provider and uses the new credentials.

Split from #192. Part of #194.

Metadata

Metadata

Assignees

No one assigned

    Labels

    server/engineExecution engine & runtime issues

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions