Skip to content

feat: distributed tracing with OpenTelemetry + Grafana Tempo - #79

Merged
arek-e merged 1 commit into
mainfrom
feat/otel-tracing
Apr 4, 2026
Merged

feat: distributed tracing with OpenTelemetry + Grafana Tempo#79
arek-e merged 1 commit into
mainfrom
feat/otel-tracing

Conversation

@arek-e

@arek-e arek-e commented Apr 4, 2026

Copy link
Copy Markdown
Owner

Summary

  • New @paws/telemetry package — OpenTelemetry SDK setup for Bun (programmatic init, no --require), Hono tracing middleware that creates spans per request with HTTP semantic conventions, W3C trace context propagation helpers
  • Grafana Tempo added to docker-compose — accepts OTLP traces on 4317/4318, generates span-metrics and service graphs to VictoriaMetrics
  • Control-plane + worker instrumented — tracing middleware on all routes, cross-service trace propagation in worker-client fetch calls
  • Log-trace correlation — logger enriched with traceId/spanId fields via pluggable enricher pattern (no hard dep on telemetry)
  • Grafana datasource wiring — Tempo with trace-to-logs (Loki) and trace-to-metrics (VictoriaMetrics) correlation, derived fields for clickable trace links in log lines
  • Alerting rules — VictoriaMetrics rules for: worker down, degraded workers, control plane down, high session failure rate, queue backlog, capacity exhaustion, high API latency, high session duration

What this enables

  • Trace a single request across control-plane → worker → VM execution
  • Click from a log line to its trace in Grafana
  • Service dependency graph auto-generated from trace data
  • Alerts on SLI violations (availability, latency, capacity)

Test plan

  • bun run typecheck — 41/41 tasks pass
  • bun run test — 30/30 tasks pass
  • Deploy with docker-compose up and verify Tempo receives traces
  • Verify Grafana Explore → Tempo shows traces from both services
  • Verify log lines contain traceId fields and link to Tempo

🤖 Generated with Claude Code

- New @paws/telemetry package: OTel SDK init for Bun, Hono tracing
  middleware, trace context propagation (W3C traceparent), helper utils
- Grafana Tempo added to docker-compose for trace storage (OTLP HTTP)
- Control-plane + worker instrumented with tracing middleware
- Cross-service trace propagation via worker-client fetch headers
- Logger enriched with trace/span IDs (pluggable enricher pattern)
- Grafana datasources: Tempo with trace-to-logs (Loki) and
  trace-to-metrics (VictoriaMetrics) correlation
- VictoriaMetrics alerting rules: worker down, high failure rate,
  capacity exhaustion, API latency, session duration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Apr 4, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@arek-e has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 2 minutes and 10 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 2 minutes and 10 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c7973466-b8db-4d1b-a931-daed853ce7dd

📥 Commits

Reviewing files that changed from the base of the PR and between d4f2ccb and 36914d0.

⛔ Files ignored due to path filters (1)
  • bun.lock is excluded by !**/*.lock
📒 Files selected for processing (21)
  • apps/control-plane/package.json
  • apps/control-plane/src/app.ts
  • apps/control-plane/src/server.ts
  • apps/control-plane/src/worker-client.ts
  • apps/control-plane/tsconfig.json
  • apps/worker/package.json
  • apps/worker/src/routes.ts
  • apps/worker/src/server.ts
  • apps/worker/tsconfig.json
  • docker-compose.yml
  • infra/grafana/datasources.yml
  • infra/tempo/tempo-config.yml
  • infra/victoriametrics/alerts.yml
  • packages/logger/src/index.ts
  • packages/logger/src/logger.ts
  • packages/telemetry/package.json
  • packages/telemetry/src/index.ts
  • packages/telemetry/src/middleware.ts
  • packages/telemetry/src/tracing.ts
  • packages/telemetry/tsconfig.json
  • tsconfig.json
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/otel-tracing

Comment @coderabbitai help to get the list of available commands and usage tips.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying getpaws with  Cloudflare Pages  Cloudflare Pages

Latest commit: 36914d0
Status: ✅  Deploy successful!
Preview URL: https://28236fda.getpaws-6m4.pages.dev
Branch Preview URL: https://feat-otel-tracing.getpaws-6m4.pages.dev

View logs

@arek-e
arek-e merged commit 7da56e6 into main Apr 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant