Repository navigation
feat: add otlp resource attributes and http span conventions - #182
Conversation
✅ Deploy Preview for lunorash ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Thank you for confirming the Contributor License Agreement! 🙏 |
|
Thank you for following the naming conventions! 🙏 |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughRuntime dispatch now uses centralized W3C trace handling, request metadata extraction, configurable inbound trace trust, deferred telemetry export, and shared OTLP encoders. Runtime and container sinks merge detected and explicit resource attributes into exported traces, metrics, logs, and spans. ChangesRuntime trace and dispatch telemetry
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Request
participant WorkerDispatch
participant TraceContext
participant ObservabilitySink
participant OTLPExporter
Request->>WorkerDispatch: RPC or REST request
WorkerDispatch->>TraceContext: beginDispatchTrace(request)
TraceContext-->>WorkerDispatch: dispatch trace and sampling decision
WorkerDispatch->>ObservabilitySink: event, request metadata, and sink context
ObservabilitySink->>OTLPExporter: encoded OTLP body with resources
OTLPExporter-->>ObservabilitySink: deferred HTTP export
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merging this PR will improve performance by 23.44%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ⚡ | broadcastWhisper to 1024 members |
1,281.2 µs | 936.1 µs | +36.87% |
| ⚡ | baseline (Object.keys + toInternal + path spread per field) |
72.4 µs | 65 µs | +11.34% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing feat/otel-resource-attributes (23945cc) with alpha (7acf42b)2
Footnotes
-
1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩
-
No successful run was found on
alpha(a07c091) during the generation of this report, so 7acf42b was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩
There was a problem hiding this comment.
Actionable comments posted: 11
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/runtime/src/create-worker.ts (1)
3240-3254: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
sinkContextnow hands rawenvandrequestto every registered observability sink.Previously sinks only got
waitUntil. Now every sink (not justotlpSink, which carefully extracts only specific derived fields viadetectResourceAttributes) receives the full rawrequest(includingAuthorization/Cookieheaders) and the full rawenv(including secret bindings). Any custom/console/webhook sink that logs or serializes itscontextargument for debugging would leak secrets and PII. This pattern is repeated at 3 more sites in this file (handleBatchRpc,serverQuery,invokeExposed) — see the consolidated comment for all sites and a suggested narrower context/helper.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/runtime/src/create-worker.ts` around lines 3240 - 3254, Stop passing raw env and request objects through sinkContext, since custom sinks may serialize secrets and PII. Update the sink context construction in the current request path and the corresponding handleBatchRpc, serverQuery, and invokeExposed sites to expose only waitUntil plus sanitized, derived telemetry attributes needed by otlpSink. Reuse a shared helper or narrow context shape so all four call sites apply the same filtering while preserving OTLP resource detection.
🧹 Nitpick comments (2)
packages/platform-conformance/packem.config.ts (1)
7-8: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winKeep build warnings visible and actionable.
Line 8 disables failure for every Packem warning, which can allow unresolved imports or declaration-generation problems to reach published artifacts. Prefer failing on warnings or narrowly allowlisting known benign warnings, and verify the behavior against the pinned Packem version.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/platform-conformance/packem.config.ts` around lines 7 - 8, Update the Packem configuration near the runtime setting to remove the blanket failOnWarn: false behavior. Prefer enabling warning failures, or replace it with a narrowly scoped allowlist for verified benign warnings compatible with the pinned Packem version, so unresolved imports and declaration-generation warnings fail the build.packages/runtime/src/observability-sinks.ts (1)
939-954: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winResource-attribute detection recomputed on every single telemetry event.
resourceAttributesForre-runsdetectResourceAttributes(env/string scans) and rebuilds a fresh merged object on everyonLog/onMetric/onRpc/onSpancall. Most detected values (service.version, deployment.environment, host.name, k8s.pod.name, process.pid) are static for the lifetime of the sink/isolate; onlycf.colocan vary per request. Consider memoizing the non-request-derived detection once at sink construction (or per-request via aWeakMapkeyed oncontext.request) instead of recomputing on every event, especially for chatty call sites that emit manyctx.log/ctx.traceevents per request.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/runtime/src/observability-sinks.ts` around lines 939 - 954, Update resourceAttributesFor to avoid recomputing stable resource detection and rebuilding the merged attributes for every telemetry event. Cache the sink-level or per-request results while preserving request-specific values such as cf.colo, and continue allowing staticResourceAttributes to override detected attributes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/container/src/otel.ts`:
- Around line 91-92: Update the documentation for deploymentEnvironment and
serviceVersion to state that their environment-variable fallbacks are applied
only when detectResources is enabled, matching the gated behavior in
detectContainerResources(); do not change the fallback implementation.
In `@packages/do/package.json`:
- Line 66: Update the `@lunora/shard-engine` dependency in
packages/do/package.json to use the configured pnpm catalog entry, such as
catalog:pnpm-workspace:shard-engine, instead of the hard-coded 1.0.0-alpha.1
version.
In `@packages/do/src/ctx-db-cdc.ts`:
- Line 15: Remove the locally defined ConflictError class in ctx-db.ts and use
the shared ConflictError imported from `@lunora/shard-engine` for all
duplicate-insert throws, including the paths checked by CDC replay. Ensure every
thrower references the single shared constructor so instanceof checks trigger
the existing replace fallback.
In `@packages/platform-conformance/package.json`:
- Around line 61-63: Update the `@lunora/platform` dependency in
packages/platform-conformance/package.json from the hard-coded version to the
workspace reference workspace:*, matching the pnpm-workspace.yaml override.
In `@packages/platform-conformance/README.md`:
- Around line 17-19: Update the documented defineHostContractSuite example to
import describe, expect, and it from vitest, then pass them as the third
argument using the { describe, expect, it } object. Keep the existing
"reference" suite name and createReferenceHost registration unchanged.
In `@packages/platform/src/capabilities.ts`:
- Around line 77-78: Update the Cloudflare capability matrix entries for
crossShardFanout and mail to mark them as emulated rather than native,
reflecting their coordinator/relay and Resend queue implementations. Preserve
all other capability classifications and the existing matrix structure.
In `@packages/platform/src/shard-directory.ts`:
- Around line 40-57: The ShardDirectory interface currently requires both lookup
strategies despite documenting them as alternatives. Replace it with a
discriminated union representing either a direct getByName(name) provider or a
two-step provider containing idForName(name) and get(id), while preserving
provider-neutral return types and shared documentation.
- Around line 17-26: Update the ShardJurisdiction type contract to match its
provider-neutral, extensible semantics: allow arbitrary string jurisdiction
values rather than restricting callers to the current "eu", "fedramp", and "us"
literals. Preserve those known values as documented suggestions or compatible
literals without keeping the type closed.
In `@packages/platform/src/shard-host.ts`:
- Around line 67-73: The ShardAlarms.set contract currently rejects the
documented null-clearing value. Update the set signature in the ShardAlarms
interface to accept null and ensure implementations honor null by clearing any
pending alarm, while preserving existing timestamp and Date scheduling behavior.
In `@packages/platform/src/socket-host.ts`:
- Around line 45-64: Ensure tagged fan-out never uses an unfiltered socket list:
update the SocketHost contract around getSockets and its tag-support behavior so
callers either receive only sockets matching the requested tag or fail closed
when exact filtering is unavailable. Do not permit hosts to silently ignore tag
and return every socket; expose and enforce an explicit capability if needed,
preserving untagged enumeration behavior.
In `@packages/runtime/src/observability-sinks.ts`:
- Around line 782-794: Update the Cloudflare detection block around `cf` and
`stringFrom` so the typed `request.cf` value is converted to a plain object
before being passed to `stringFrom`, such as by spreading it into a new object.
Preserve the existing `colo` fallback order and Cloudflare detection behavior.
---
Outside diff comments:
In `@packages/runtime/src/create-worker.ts`:
- Around line 3240-3254: Stop passing raw env and request objects through
sinkContext, since custom sinks may serialize secrets and PII. Update the sink
context construction in the current request path and the corresponding
handleBatchRpc, serverQuery, and invokeExposed sites to expose only waitUntil
plus sanitized, derived telemetry attributes needed by otlpSink. Reuse a shared
helper or narrow context shape so all four call sites apply the same filtering
while preserving OTLP resource detection.
---
Nitpick comments:
In `@packages/platform-conformance/packem.config.ts`:
- Around line 7-8: Update the Packem configuration near the runtime setting to
remove the blanket failOnWarn: false behavior. Prefer enabling warning failures,
or replace it with a narrowly scoped allowlist for verified benign warnings
compatible with the pinned Packem version, so unresolved imports and
declaration-generation warnings fail the build.
In `@packages/runtime/src/observability-sinks.ts`:
- Around line 939-954: Update resourceAttributesFor to avoid recomputing stable
resource detection and rebuilding the merged attributes for every telemetry
event. Cache the sink-level or per-request results while preserving
request-specific values such as cf.colo, and continue allowing
staticResourceAttributes to override detected attributes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 3e7f8d76-d499-428c-a5ab-fcc2fb514d48
⛔ Files ignored due to path filters (20)
AGENTS.mdis excluded by none and included by nonelabeler-config.ymlis excluded by none and included by nonepackages/container/__tests__/otel.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/container/__tests__/workerd/container-smoke.workerd.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/do/__tests__/ctx-db.constraints.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/do/__tests__/function-metrics.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/do/__tests__/reactive-cache.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/do/__tests__/transaction.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/platform-conformance/__assets__/package-og.svgis excluded by!**/*.svgand included bypackages/**packages/platform-conformance/__tests__/suite.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/platform/__assets__/package-og.svgis excluded by!**/*.svgand included bypackages/**packages/platform/__tests__/contracts.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/observability-sinks.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/shard-engine/__assets__/package-og.svgis excluded by!**/*.svgand included bypackages/**packages/shard-engine/__tests__/smoke.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**plans/multi-platform-portability-assessment.mdis excluded by none and included by nonepnpm-lock.yamlis excluded by!**/pnpm-lock.yamland included by nonepnpm-workspace.yamlis excluded by none and included by noneshared/log-event.tsis excluded by none and included by noneshared/otlp.tsis excluded by none and included by none
📒 Files selected for processing (55)
packages/container/src/otel.tspackages/do/__bench__/reactive-cache.bench.tspackages/do/package.jsonpackages/do/src/ctx-db-cdc.tspackages/do/src/ctx-db-rank-page.tspackages/do/src/ctx-db.tspackages/do/src/index.tspackages/do/src/reactive-cache.tspackages/do/src/shard-do.tspackages/platform-conformance/.releaserc.jsonpackages/platform-conformance/LICENSE.mdpackages/platform-conformance/README.mdpackages/platform-conformance/eslint.config.jspackages/platform-conformance/package.jsonpackages/platform-conformance/packem.config.tspackages/platform-conformance/prettier.config.jspackages/platform-conformance/project.jsonpackages/platform-conformance/src/index.tspackages/platform-conformance/src/reference-host.tspackages/platform-conformance/src/suite.tspackages/platform-conformance/tsconfig.jsonpackages/platform-conformance/vitest.config.tspackages/platform/.releaserc.jsonpackages/platform/LICENSE.mdpackages/platform/README.mdpackages/platform/eslint.config.jspackages/platform/package.jsonpackages/platform/packem.config.tspackages/platform/prettier.config.jspackages/platform/project.jsonpackages/platform/src/bindings.tspackages/platform/src/capabilities.tspackages/platform/src/index.tspackages/platform/src/scheduler-host.tspackages/platform/src/shard-directory.tspackages/platform/src/shard-host.tspackages/platform/src/socket-host.tspackages/platform/tsconfig.jsonpackages/platform/vitest.config.tspackages/runtime/src/create-worker.tspackages/runtime/src/observability-sinks.tspackages/runtime/src/observability.tspackages/shard-engine/.releaserc.jsonpackages/shard-engine/LICENSE.mdpackages/shard-engine/README.mdpackages/shard-engine/eslint.config.jspackages/shard-engine/package.jsonpackages/shard-engine/packem.config.tspackages/shard-engine/prettier.config.jspackages/shard-engine/project.jsonpackages/shard-engine/src/dependency-tracker.tspackages/shard-engine/src/index.tspackages/shard-engine/src/transaction.tspackages/shard-engine/tsconfig.jsonpackages/shard-engine/vitest.config.ts
a9b95c1 to
a0bb629
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/runtime/src/otel-trace.ts`:
- Around line 62-67: Update createDispatchSpanContext to mark the returned
dispatch context as local with isRemote: false, and preserve the extracted
parent’s traceState when present so injectTraceContext propagates it. Keep the
existing traceId, spanId, and traceFlags behavior unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: b701b995-0d7e-4979-9a8a-a83baa461492
⛔ Files ignored due to path filters (10)
packages/container/__tests__/otel.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/container/__tests__/workerd/container-smoke.workerd.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/create-worker.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/observability-sinks.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/otel-trace.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/shard-engine/__tests__/smoke.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**pnpm-lock.yamlis excluded by!**/pnpm-lock.yamland included by nonepnpm-workspace.yamlis excluded by none and included by noneshared/log-event.tsis excluded by none and included by noneshared/otlp.tsis excluded by none and included by none
📒 Files selected for processing (7)
packages/container/package.jsonpackages/container/src/otel.tspackages/runtime/package.jsonpackages/runtime/src/create-worker.tspackages/runtime/src/observability-sinks.tspackages/runtime/src/observability.tspackages/runtime/src/otel-trace.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/runtime/src/create-worker.ts
- packages/runtime/src/observability-sinks.ts
a0bb629 to
e99930f
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (2)
packages/runtime/src/observability-sinks.ts (2)
768-781: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
stringEnvandstringFromare identical implementations.Both functions have the exact same body (
typeof value === "string" && value.length > 0 ? value : undefined). Consider collapsing to one helper.As per path instructions,
packages/**/*.tsshould "Follow DRY principles."♻️ Proposed consolidation
-const stringEnv = (env: Record<string, unknown> | undefined, key: string): string | undefined => { - const value = env?.[key]; - - return typeof value === "string" && value.length > 0 ? value : undefined; -}; - /** Read a string property from an object, or `undefined` when missing / not a string. */ const stringFrom = (object: Record<string, unknown> | undefined, key: string): string | undefined => { const value = object?.[key]; return typeof value === "string" && value.length > 0 ? value : undefined; };Then replace all
stringEnv(env, ...)call sites withstringFrom(env, ...).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/runtime/src/observability-sinks.ts` around lines 768 - 781, Consolidate the duplicate stringEnv and stringFrom helpers into a single helper, retaining the existing non-empty-string behavior. Remove the redundant helper and update every stringEnv(env, ...) call site to use the retained helper, including any references near the observability sink configuration logic.Source: Path instructions
941-964: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win
resourceAttributesForre-derives the full detected resource on every log/metric/span/RPC emission.
detectResource(context?.env, context?.request)re-parses every env var (SERVICE_VERSION,HOSTNAME,KUBERNETES_*, etc.) and rebuilds aResourceon every singleonLog/onMetric/onSpan/onRpccall, even though the env-derived attributes (service.version,deployment.environment,host.name,k8s.pod.name,process.pid) are constant for the sink's lifetime. Only therequest.cf-derived fields (cloud.provider/cloud.region) genuinely vary per event. On a busy worker emitting many log lines per request, this repeats the same string comparisons unnecessarily.Consider splitting detection into an env-only part computed once at sink construction (merged into
staticResource) and a request-only part (cf) computed per event, rather than recomputing everything per call.As per path instructions,
packages/**/src/**/*.tsshould adhere to "best practices recommended for performance."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/runtime/src/observability-sinks.ts` around lines 941 - 964, Update otlpSink and resourceAttributesFor to detect environment-derived resources once during sink construction and merge them into staticResource, while retaining only request.cf-derived resource detection per event. Ensure explicit resourceAttributes continue to override detected and convenience attributes, and preserve per-event behavior for cloud.provider and cloud.region.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/runtime/src/observability-sinks.ts`:
- Around line 768-781: Consolidate the duplicate stringEnv and stringFrom
helpers into a single helper, retaining the existing non-empty-string behavior.
Remove the redundant helper and update every stringEnv(env, ...) call site to
use the retained helper, including any references near the observability sink
configuration logic.
- Around line 941-964: Update otlpSink and resourceAttributesFor to detect
environment-derived resources once during sink construction and merge them into
staticResource, while retaining only request.cf-derived resource detection per
event. Ensure explicit resourceAttributes continue to override detected and
convenience attributes, and preserve per-event behavior for cloud.provider and
cloud.region.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: eb3a5904-eb1e-4f92-b77b-f5eb665302c8
⛔ Files ignored due to path filters (8)
packages/container/__tests__/otel.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/create-worker.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/observability-sinks.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/otel-trace.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**pnpm-lock.yamlis excluded by!**/pnpm-lock.yamland included by nonepnpm-workspace.yamlis excluded by none and included by noneshared/log-event.tsis excluded by none and included by noneshared/otlp.tsis excluded by none and included by none
📒 Files selected for processing (7)
packages/container/package.jsonpackages/container/src/otel.tspackages/runtime/package.jsonpackages/runtime/src/create-worker.tspackages/runtime/src/observability-sinks.tspackages/runtime/src/observability.tspackages/runtime/src/otel-trace.ts
🚧 Files skipped from review as they are similar to previous changes (6)
- packages/container/package.json
- packages/runtime/package.json
- packages/runtime/src/otel-trace.ts
- packages/runtime/src/observability.ts
- packages/container/src/otel.ts
- packages/runtime/src/create-worker.ts
Addresses CodeRabbit review + the thermo review on PR #182. - `createDispatchSpanContext` carries the upstream `traceState` and marks the dispatch span `isRemote: false`. Without it the W3C propagator emits no `tracestate` header, so vendor correlation keys died at this hop — a conformance gap in a change whose point is OTel compatibility. - Container option docs no longer promise unconditional env fallbacks for `serviceVersion` / `deploymentEnvironment`; both only fire under `detectResources: true`, unlike `serviceName`. - Export `OtlpResourceAttributes`: it is the type of the public `OtlpSinkOptions.resourceAttributes` but had no exported name. - Regenerate `api-snapshots/runtime.api.md` (`Lint (api surface)` was red). - Run Prettier over the branch (`Lint (prettier)` was red). CodeRabbit's claim that `stringFrom(cf, "colo")` fails to type-check does not reproduce: `@cloudflare/workers-types` declares `cf?: any`, and `tsc --noEmit` is green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Addresses the six findings from the thermo review of PR #182. **Inbound trace context is no longer trusted by default.** The `traceparent` header is caller-supplied on any public Worker, and its trace id was steering both the head-sampling verdict and which trace a request's spans and `ctx.log` lines joined — so a client could send `00-ffffffff…` to drop itself out of every trace, or graft entries into another tenant's waterfall. Continuation is now opt-in via `trustInboundTraceContext`, and head sampling keys on a server-minted span id whenever the upstream is untrusted. The whole-trace invariant is unaffected: downstream hops read the verdict off the propagated `traceparent` rather than re-deriving it. `tracestate` is validated against the W3C grammar and size limits before being echoed onward. **Removed `@opentelemetry/api`, `/core`, and `/resources`** from `@lunora/runtime` and `@lunora/container`. They were doing two string operations and an object merge, in Workers bundles, via Node-oriented packages. `otel-trace.ts` now builds on the zero-dep `parseTraceparent`/`buildTraceparent` in `shared/otlp.ts` — the encoder `@lunora/do` and `@lunora/container` already share — which also restores the forward-compatible parsing an SDK propagator rejects, and un-strands `buildTraceparent`. The `Resource.merge` round-trip and the two casts that claimed OTel's `Attributes` was a `Record<string, string|number|boolean>` are gone. `@lunora/container`'s module doc claimed "no `@opentelemetry/*` dependency" throughout; that is true again. **One resource detector, not two.** `shared/otlp-resource.ts` holds the policy both exporters were implementing separately, with the environment injected as a reader so it stays pure and testable. The detectors are split by where they can fire, so the Worker no longer probes for `HOSTNAME`, `KUBERNETES_*` and a pid that cannot exist in workerd. **Resource attributes resolve once per request, not once per event.** `resourceAttributesFor` re-ran detection and rebuilt a merged bag on every log line, metric and span on the hot dispatch path. Detection is now lazy and memoized per request, and with `detectResources` off (the default) the sink returns its static bag by reference and allocates nothing. **`LogSinkContext` no longer carries raw `env` and `request`.** That context is fanned out to every registered sink including user-authored ones, so it exposed every secret binding and the caller's `Authorization` and `Cookie` to anything that logged its arguments while debugging. Sinks now receive a thunk resolving a small allowlisted attribute bag. **`waitUntil` reaches `serverQuery` and the REST surface.** Error spans now exceed the gzip threshold, making their export asynchronous — without a `waitUntil` those two paths could lose exactly the telemetry that matters. **Structure.** `observability-sinks.ts` is back under 1000 lines (1056 → ~700): the OTLP encoders and transport moved to `otlp-export.ts` and detection to `resource-detect.ts`, and the file-wide `import/exports-last` suppression is no longer needed. The six-field `requestMeta` copy at seven dispatch sites is one spread, `traceEventFields` replaces the hand-copied trace spreads that had already diverged, and `dispatchSingleShard` loses the `dispatchContext` parameter that was always derivable from its own `request`. Tests: restored the exact `expect.assertions(n)` counts, added an assertion-free poll so every OTLP test awaits its send rather than racing the gzip threshold, covered all three metric-resource branches, and added suites for the trust policy, the shared detectors, and `waitUntil` threading. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`trustInboundTraceContext` was a whole-worker boolean, which asked the user
to make a security judgement with no help and then couldn't express the
common case: one worker serving both public browser traffic and a trusted
internal caller.
It now takes a named signal, a list of them, `true`/`false`, or a custom
predicate:
trustInboundTraceContext: true // internal-only
trustInboundTraceContext: "mtls" // edge-verified certs
trustInboundTraceContext: ["mtls", "cloudflare-access"] // either
trustInboundTraceContext: (request) => … // escape hatch
`"mtls"` reads Cloudflare's `cf.tlsClientAuth.certVerified`, which the edge
sets and a caller cannot forge — the one signal that carries its own proof.
`"cloudflare-access"` checks the Access assertion, which is sound only when
Access actually fronts the worker; both are documented as such so the
choice is informed rather than guessed. Unknown signals fail closed, and
the policy resolves to a single predicate at construction, so a dispatch
pays one call.
The custom form receives only the `Request`. Handing it `env` would put
every secret binding behind a telemetry callback — the same boundary
`LogSinkContext` was just narrowed to avoid.
Also: dropping an inbound `traceparent` was completely silent, which reads
as a mysteriously broken waterfall with nothing to search for. When the
option is unset and a well-formed upstream trace is actually discarded, the
worker now logs one hint naming the option. It fires at most once, only on
a real drop, and setting the option to `false` silences it — an explicit
`false` is a decision, `undefined` is an unanswered question.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/runtime/src/trace-trust.ts`:
- Around line 92-94: Update the custom predicate branch in isTrustedUpstream so
the returned function invokes the user-supplied option inside a try/catch and
returns false when it throws. Preserve the existing predicate result for
successful calls, ensuring faulty trust callbacks fail closed without
propagating exceptions during dispatch.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 8d472a1e-7ff9-4c2d-8bea-389c7dbe68f2
⛔ Files ignored due to path filters (12)
api-snapshots/lunora.api.mdis excluded by none and included by noneapi-snapshots/runtime.api.mdis excluded by none and included by nonepackages/runtime/__tests__/create-worker.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/observability-sinks.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/otel-trace.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/otlp-resource.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/rest-surface.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/server-query.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**packages/runtime/__tests__/trace-trust.test.tsis excluded by!**/__tests__/**,!**/*.test.tsand included bypackages/**shared/log-event.tsis excluded by none and included by noneshared/otlp-resource.tsis excluded by none and included by noneshared/otlp.tsis excluded by none and included by none
📒 Files selected for processing (9)
packages/container/src/otel.tspackages/runtime/src/create-worker.tspackages/runtime/src/index.tspackages/runtime/src/observability-sinks.tspackages/runtime/src/otel-trace.tspackages/runtime/src/otlp-export.tspackages/runtime/src/resource-detect.tspackages/runtime/src/rest-routes.tspackages/runtime/src/trace-trust.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/container/src/otel.ts
| if (typeof option === "function") { | ||
| return option; | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Fail closed if a custom trust predicate throws.
isTrustedUpstream(request) runs on every dispatch (create-worker.ts Line 3173) and is evaluated before beginDispatchTrace's try scope, so a throwing user predicate turns every request into a 500 rather than merely declining to continue the trace. Wrapping the escape-hatch form keeps a faulty telemetry-trust callback from taking down the data path.
🛡️ Fail-closed wrapper for the custom form
if (typeof option === "function") {
- return option;
+ return (request) => {
+ try {
+ return option(request);
+ } catch {
+ // A faulty trust predicate must not fault the dispatch — decline the trace.
+ return false;
+ }
+ };
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (typeof option === "function") { | |
| return option; | |
| } | |
| if (typeof option === "function") { | |
| return (request) => { | |
| try { | |
| return option(request); | |
| } catch { | |
| // A faulty trust predicate must not fault the dispatch — decline the trace. | |
| return false; | |
| } | |
| }; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/runtime/src/trace-trust.ts` around lines 92 - 94, Update the custom
predicate branch in isTrustedUpstream so the returned function invokes the
user-supplied option inside a try/catch and returns false when it throws.
Preserve the existing predicate result for successful calls, ensuring faulty
trust callbacks fail closed without propagating exceptions during dispatch.
The check was `headers.get("cf-access-jwt-assertion") !== null` — a header
presence test wearing the name of a verified one. In every deployment
topology it was either redundant or unsound:
- Worker reachable only through Access: every caller has already passed it,
so `true` is correct and the header check adds nothing.
- Worker also reachable directly (a `*.workers.dev` route left enabled, a
hostname outside the Access policy): anyone sets the header and is
trusted.
- Mixed routes — the only case wanting per-request discrimination — is
exactly where a presence test fails, since the direct caller forges it.
Verifying the assertion properly means a JWKS fetch and an audience check.
That is async, so it would put a network call on the RPC dispatch path, and
`@lunora/cloudflare-access` already does it correctly where it belongs, in
`resolveIdentity`. A predicate covers anyone who wants to wire it in.
The name was the real hazard: it reads as "verified by Cloudflare Access",
so it was the option someone would reach for *because* it sounded like the
secure one, while being the weakest thing in the union. The module doc now
records why it is absent so it does not get re-added.
Also drops the array form. With one signal left, `TraceTrustSignal[]` is an
array of a single-member union — an API artifact, not a feature. Widening
back to accept a list is non-breaking whenever a second signal earns its
place.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The observability guide documented `otlpSink` and trace propagation but not the options this PR adds, and — more importantly — not the behaviour change that inbound trace continuation is now opt-in. Someone whose waterfall stopped stitching had nothing to search for. - New "Resource attributes" section: the four convenience options plus `resourceAttributes`, what `detectResources` reads and why it is opt-in, and the caveat that shard-side signals (`ctx.log` / `ctx.trace` / `ctx.metrics`) carry only the explicit attributes. - New "Joining a trace from upstream" section: the `trustInboundTraceContext` values as a table, why it defaults off, why `true` is the right answer behind a front door you control, and the reminder to check that a `*.workers.dev` route is not an ungated second one. - Wire contract: the HTTP semantic-convention span attributes, why `http.route` carries the function path rather than `/_lunora/rpc`, the `exception` span event, and `parentSpanId`/`flags`. - Container telemetry: the matching resource options, and that detection there additionally resolves host/k8s/pid because it reads a real process environment. - Privacy: `url.path` is raw and can carry ids on the REST surface (`http.route` is the safe one to group on), plus a note for custom-sink authors that the sink context deliberately exposes neither `env` nor the raw `Request`. - Production checklist: one item pointing at the trust decision. Verified with `pnpm --filter @lunora/docs run build` — every new anchor resolves (the site's link validator silently downgrades broken ones to plain text, so this was checked in the built chunks, not just the source). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`sampling: { headRate, alwaysSampleErrors }` has been a `createWorker`
option with no user-facing documentation anywhere — the previous commit
wanted to link to it twice and had to describe it inline instead.
Covers the parts that are surprising rather than just the field names:
- It governs trace spans only. `ctx.log` and `ctx.metrics` are never
sampled — logs are what you go looking for when something broke, and a
sampled metric is a wrong metric.
- Traces are kept or dropped *whole*. The verdict is derived from the trace
id rather than a coin flip, so the worker, the shard, and any container
agree and you never get half a waterfall.
- Errors are never sampled away. `headRate: 0.01` does not mean 1% of your
errors; it means 1% of successful traffic plus every failure, because the
shard holds a sampled-out trace's spans until the dispatch settles and
flushes them if anything failed.
- Which trace id decides, which is now load-bearing: untrusted inbound
context keys the decision on a server-minted id so a caller cannot pick
its own sampling outcome, and a trusted upstream keys on its trace id so a
distributed trace stays whole.
Also distinguishes `sampling` from a sink's `onlyErrors`, which are easy to
confuse: one keeps a representative fraction of whole traces, the other
keeps no successful spans at all and so loses the baseline you would compare
a regression against.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`.expose({ rest: true })` (plan 167) shipped with no user-facing docs: the
only description of the URL scheme, argument decoding, and shard selection
was in the source. Adds `concepts/rest-api` and registers it in the nav.
Covers what a caller cannot infer from the type signature:
- Default-closed, and why an unexposed procedure 404s rather than 403s —
probing cannot enumerate your functions. `.expose` is public-builder-only,
since an internal function is never client-callable; `stream` is never
exposed because it is a WebSocket surface.
- The derived URL and method per kind, including that a query also accepts
POST for argument sets too large for a query string, and that a wrong
method gets a 405 with `Allow`.
- GET argument decoding, which has a real edge: each query value is
JSON-parsed when it looks like JSON, so `?limit=5` is a number but
`?q=null` is `null` rather than `"null"`. Documented with the workaround.
- `shardKey` via `?shardKey=` or `x-lunora-shard-key` (query wins), and that
it is reserved so it never collides with a handler argument.
- That auth/RLS/validators are the same code path, not a parallel one —
exposing a procedure publishes its URL, not its data.
- `restRateLimit`, which runs before dispatch and defaults to keying on
`CF-Connecting-IP`.
- The generated OpenAPI, and why it cannot drift from the live router (both
derive the mapping from `shared/rest-surface.ts`).
Also disambiguates this from `httpRoute`/`httpAction` on both pages, since
`http-endpoints` previously listed "a REST surface for a non-Lunora client"
as a reason to hand-write a route — which is now the case `.expose` covers.
Verified with a docs build; every cross-page link checked in the built
chunks, since the site's validator downgrades broken links to plain text
rather than failing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`Lint (prettier)` was failing on `concepts/rest-api.mdx`. When the page was
first written its frontmatter was still invalid YAML (an unquoted
`{ rest: true }` parsed as a nested mapping), so Prettier could not parse the
file and skipped it; the frontmatter was fixed afterwards but Prettier was
never re-run over it.
Reflows the over-padded URL column in the methods table and normalises
`*emphasis*` to `_emphasis_`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sync the cloud observability + security-hardening branch with alpha (OTLP resource attributes #182, notify observability #183, do perf #184, workerd custom-spans test #180, ai-gateway correlation #181, and release bumps). Clean auto-merge — no conflicts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0197KjhmBDB3PXAc6TXiPqZ9
Addresses CodeRabbit review on PR #182. - `ShardDirectory` becomes a union of `DirectShardDirectory` (name lookup) and `TwoStepShardDirectory` (`idForName` + `get`). The docs already described these as alternatives, but the interface required all three members, so a name-only registry had to stub out methods it has no concept of. Adds `resolveShard` so callers do not narrow by hand. - `ShardJurisdiction` widens as its doc comment claims, so a new Cloudflare jurisdiction is not a compile error here. - `SocketHost.getSockets(tag)` must now filter exactly. Permitting a host to ignore the tag and return every socket turns a tagged fan-out into a cross-tenant broadcast; hosts that cannot filter omit `setTag` instead. - `ShardAlarms.set` documented `null` clearing that its signature rejected; the doc now points at `delete()`. - Cloudflare capability matrix reports `crossShardFanout` and `mail` as emulated: both are Lunora-implemented over primitives (coordinator tier, Resend), not first-class platform products. - Conformance README example passes the required Vitest API argument. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
alpha (#182) and this branch independently built OTLP export, and the rebase put them in the same files. Resolved per the decisions taken during the rebase: **Resource layer: alpha's.** `shared/otlp.ts` keeps `buildResourceAttributes` / `OtlpResourceAttributes` and drops this branch's `OtlpResource` model, so `otlp-resource.ts` stays the single detection policy shared by the Worker sink and the container exporter. `wrapResource*` keep this branch's array-accepting shape — batching — but take alpha's `serviceName` + attribute-bag inputs. **Export path: this branch's batching.** Per-event resource detection (`detectResources`) is dropped, and that is a real trade, not an oversight: a batch ships N events under ONE `Resource`, so attributes that vary per request have nowhere to attach unless batches are keyed by resource identity. Its tests go with it. **Span encoding: both.** alpha's HTTP server semantic conventions (`http.request.method`, `http.route`, `http.response.status_code`, `url.*`, `server.*`, `user_agent.original`) and its OTel `exception` event are ported onto this branch's `encodeRpcSpan`, which the merge had otherwise dropped. `deployment.environment` keeps alpha's spelling rather than the newer `deployment.environment.name`. Emitting both was tried and reverted: the extra attribute pushed envelopes past `OTLP_GZIP_THRESHOLD`, silently switching the wire format from JSON to gzip. That threshold also broke three tests that indexed `fetch.mock.calls` positionally — a body over 1 KiB compresses first and so POSTs a microtask later than a smaller one. They now await `otlpCalls` and select by endpoint, which is what the helper's own docstring prescribes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Summary
Makes Lunora's OTLP export genuinely OTel-compatible — resource attributes, HTTP semantic-convention span attributes, exception events, and W3C trace-context propagation — across
@lunora/runtimeand@lunora/container, with the wire contract shared throughshared/otlp.ts.Feature work
otlpSinkgainsserviceVersion,serviceNamespace,deploymentEnvironment,resourceAttributes, and an opt-indetectResources. The container exporter gains the matching options. Explicit configuration always wins over detection.http.request.method,url.path,url.scheme,server.address,server.port,user_agent.original,http.route, andhttp.response.status_code; errors additionally emit a standardexceptionspan event.traceparent/tracestate, mints this dispatch's span, and propagates the result to the shard and any container beneath it.Behaviour changes (details under Notes for reviewers)
traceparentis now opt-in viatrustInboundTraceContext.LogSinkContextno longer carries rawenv/request; it carries a resolvedresourceAttributesthunk instead.Dependencies: none added. The
@opentelemetry/api//core//resourcespackages introduced mid-review were removed again in favour of the existing zero-dependency helpers inshared/otlp.ts— see below.Linked issues
Test plan
pnpm --filter @lunora/runtime run test— 692 testspnpm --filter @lunora/container run test— 138 testspnpm --filter @lunora/do run test— 1208 tests (sharesLogSinkContext)pnpm run lint:typesacrossruntime,container,do,d1,studio,cli,vite,codegenpnpm run lint:eslint,pnpm run lint:prettierpnpm run api:check— public API surface regenerated and committedWorkerd integrationjobs green)Reviewers may want to re-run
pnpm run api:checkand the workerd jobs specifically, sinceshared/log-event.tsis inlined into both@lunora/runtimeand@lunora/do.Checklist
feat(scope): subject)apps/docspackage.jsonfiles inpackages/*modified outside the touched packageNotes for reviewers
1. Inbound
traceparentis no longer trusted by defaultThe header is caller-supplied. Trusting it let any client pick the trace their spans and
ctx.loglines join — grafting entries into another tenant's waterfall in a shared collector — and, becauseshared/samplingderives the head verdict from the trace id, pick their own sampling outcome (00-ffffffff…to disappear from traces,00-000000…to force capture and inflate ingest cost).Continuation is now opt-in, and head sampling keys on a server-minted span id whenever the upstream is untrusted:
Error traces are unaffected either way: the tail bias is evaluated from the worker's own decision, never the caller's.
tracestateis validated against the W3C grammar and size limits before being echoed onward.Upgrade note: if you rely on end-to-end waterfalls stitching to an upstream, set
trustInboundTraceContext. Behind a gateway, mesh, or Cloudflare Access that setstraceparentitself,trueis the answer — but confirm the worker is genuinely unreachable otherwise (a*.workers.devroute left enabled is an ungated second front door). Leaving the option unset logs a one-time hint if an inbound trace is actually dropped, so this doesn't fail silently; setting it tofalsekeeps the behaviour and silences that.2.
LogSinkContextshape change — affects custom sinksAn earlier revision put raw
envandrequeston the sink context. That context is fanned out to every registered sink, including user-authored ones, so it exposed every secret binding and the caller'sAuthorization/Cookieto anything that logged its arguments while debugging.It now carries a lazy, memoized
resourceAttributes()thunk returning a small allowlisted bag. A custom sink readingcontext.envorcontext.requestmust be updated — nothing in-tree did.3. The OpenTelemetry SDK dependencies were added, then removed
A mid-review revision pulled in
@opentelemetry/api,/core, and/resources. They were doing two string operations and an object merge — inside Workers bundles, via Node-oriented packages that resolveos/fs/child_processwithoutnodejs_compat. All three are gone again:traceparentparsing/formatting now uses the zero-depparseTraceparent/buildTraceparentalready shared with@lunora/doand@lunora/container. This also restores the spec's forward-compatible parsing (an SDK propagator rejects>= 4-field future versions) and un-strandsbuildTraceparent, which the SDK swap had made dead code.Attributeswas aRecord<string, string | number | boolean>.@lunora/container's module doc claims "no@opentelemetry/*dependency" — true again.4. Structure
observability-sinks.tshad grown past 1000 lines; the OTLP encoders and transport moved tootlp-export.tsand resource detection toresource-detect.ts, taking it back to ~700 and making its file-wideimport/exports-lastsuppression unnecessary. The two divergent resource detectors (worker and container) are now one policy inshared/otlp-resource.tswith the environment injected, so it stays pure and testable.5. Performance
The original CodSpeed regression on the hot RPC path (
rpc-dispatch :: no auth: bare envelope to __root__, −12%) is resolved — resource detection is now lazy and memoized per request rather than re-run per telemetry event, and the OTel propagator's context round-trip is gone.CodSpeed still flags
import-throughput :: 1 shard × 1000 rows. That benchmark exercisesquery-coordinator.ts, which this PR does not touch and which imports nothing this PR touches; the bench stubs the shardfetch, so it never enters dispatch or observability code. The same report shows improvements of +34% and +11% in@lunora/doand@lunora/values— packages with no changes here — and warns of "different runtime environments" and a substituted comparison base. Reads as baseline noise.6. Docs
apps/docs/src/content/docs/concepts/observability.mdxgains a Resource attributes section and a Joining a trace from upstream section (thetrustInboundTraceContexttable and why it defaults off), plus the new HTTP semantic-convention attributes andexceptionevent in the wire contract, the container exporter's matching options, and a note for custom-sink authors that the sink context exposes neitherenvnor the rawRequest. The production checklist gains one item pointing at the trust decision.Verified with a docs build — the site's link validator silently downgrades broken links to plain text, so every new anchor was checked in the built chunks rather than the source.
Contributor License Agreement
Summary by CodeRabbit
New Features
waitUntilplumbing for telemetry timing (RPC, REST, and SSR fast-path).Improvements