feat(observability): default-on OTLP export beside JSONL traces

[divo12]

User description

Summary

OpenTelemetry OTLP/HTTP export is default-on, not opt-in: the SDK ships in core dependencies and every session fans its Spec 12a JSONL trace to OTel through CompositeTracer + BatchSpanProcessor. JSONL stays the durable substrate; OTel is the additive sink.

load_otel_config()           # enabled unless OTEL_SDK_DISABLED is truthy
  endpoint = OTEL_EXPORTER_OTLP_ENDPOINT or "http://localhost:4318"

Opt out with OTEL_SDK_DISABLED=true (no provider, no exporter, JSONL only). Two consequences a reader should know, both measured on this branch with no collector listening on :4318:

• Span creation stays cheap — ~0.3 ms/session with OTel on vs ~0.002 ms off; export is async and never blocks a span.
• Shutdown is not free: the exporter retries (~0.9 s / 2 s / 3.3 s backoff) and then logs Failed to export span batch due to timeout, max retries or shutdown, adding ~6 s per process. Set OTEL_SDK_DISABLED=true where no collector exists.

A missing/broken OTel provider degrades to JSONL-only and warns exactly once per process via warnings.warn — logging is banned in src/ by an invariant test, so this path deliberately uses warnings, not a logger.

Gap analysis vs hermes-otel / Paperclip / chorus §4 lives in docs/specs/divo/otel-architecture-gap.md; the cross-repo eval resolves its repo roots from DREAM_REPO_ROOT / CHORUS_REPO_ROOT (defaulting to this checkout and its ../chorus sibling) rather than hard-coded developer paths.

Type

• Bug fix
• Feature
• Refactor / docs / chore

Checklist

• Tests added or updated for behaviour changes
• tests/test_public_api.py updated if the public API changed
• CHANGELOG.md updated for user-visible changes
• No secrets, .env.local, or credentials in the diff
• Scope stays within dream (org/company features belong in sibling repos)

Test plan

• uv sync --all-extras
• uv run pytest tests/test_observability/ -q — 76 passed
• uv run python evals/otel/eval_step1_foundation.py
• uv run ruff check src tests / uv run mypy
• CI: 4/4 green (3.11–3.14)
• Measured default-on cost with and without a collector (numbers above)

Link to Devin session: https://app.devin.ai/sessions/4f7deb4af0914056a24a4d660a661713
Requested by: @divo12

---

CodeAnt-AI Description

Export session traces to OpenTelemetry by default while preserving JSONL traces

What Changed

• Sessions now send the same trace lifecycle to OTLP/HTTP in addition to the durable JSONL trace.
• OTLP uses http://localhost:4318 by default and supports custom endpoints and service metadata through environment settings.
• Set OTEL_SDK_DISABLED=true to keep JSONL-only tracing; if the OTel provider cannot start, sessions continue with JSONL and emit one warning.
• Trace export runs asynchronously, preserves span nesting and attributes, records errors, and does not prevent JSONL events when the OTLP sink fails.
• Shutdown waits no longer than five seconds when a collector is unavailable.

Impact

✅ Traces available in OpenTelemetry collectors by default
✅ JSONL traces remain available when OTLP is disabled or unavailable
✅ Fewer shutdown hangs from unavailable collectors

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

[codeant-ai]

🤖 CodeAnt AI — Review Status

Status	Commit	Started (UTC)	Finished (UTC)
✅ Incremental review completed	9652a21	Aug 16, 2026 · 16:50	16:51
✅ Reviewed your PR	ddcb13e	Aug 07, 2026 · 09:46	09:49

[codeant-ai]

---

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

[divo12]

@greptileai please review

[greptile-apps]

Greptile Summary

The PR adds default-on OTLP/HTTP export alongside the durable JSONL trace and wires each session through a composite tracer.

• Adds typed OTel configuration, provider, tracer, attribute conversion, and public factory APIs.
• Adds core OpenTelemetry dependencies, evaluation scripts, documentation, and observability tests.
• Preserves JSONL as the primary trace while asynchronously exporting sibling OTel spans.

Confidence Score: 4/5

The partial-entry cleanup defect should be fixed before merging because an OTel span-start failure can mask its cause, leave the JSONL span unclosed, and abort the session.

CompositeTracer tracks successful entries separately but unwinds the full manager list, so a later child entry failure invokes cleanup on an unentered context manager before the successfully entered JSONL manager can be closed.

Files Needing Attention: src/dream/observability/_composite.py

Important Files Changed

Filename	Overview
src/dream/observability/_composite.py	Adds multi-sink span and event fan-out, but its partial-entry exception cleanup exits unentered managers and can leave entered spans unclosed.
src/dream/observability/_otel_provider.py	Adds lazy process-wide provider construction, batch export, caching, and atexit shutdown.
src/dream/observability/_otel_tracer.py	Adapts Dream tracer spans and events to OTel spans with nesting, attributes, and error status.
src/dream/observability/_factory.py	Builds JSONL-only or composite session tracers according to the OTel opt-out configuration.
src/dream/_factory.py	Replaces direct JSONL tracer construction with the new session tracer factory.
src/dream/observability/_otel_config.py	Implements default-on environment configuration with standard SDK disable handling and endpoint defaults.
evals/otel/eval_cross_repo.py	Adds a manually invoked cross-repository evaluation runner with configurable repository roots.
pyproject.toml	Promotes the OpenTelemetry API, SDK, and OTLP HTTP exporter to core dependencies.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    S[Session engine] --> C[CompositeTracer]
    C --> J[JsonlTracer]
    C --> O[OtelTracer]
    J --> F[Durable JSONL trace]
    O --> B[BatchSpanProcessor]
    B --> E[OTLP HTTP endpoint]

Loading

Prompt To Fix All With AI

### Issue 1
src/dream/observability/_composite.py:42-45
**Unentered span managers are exited**

When a later tracer raises during `__enter__`, this branch calls `__exit__` on every manager, including those that never entered. The resulting context-manager error masks the original failure and prevents the already-entered JSONL span from closing, leaving an incomplete trace and aborting the session.

```suggestion
        except BaseException as exc:
            for manager in reversed(managers[: len(spans)]):
                manager.__exit__(type(exc), exc, exc.__traceback__)
            raise
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "fix(otel): make cross-repo eval paths po..." | Re-trigger Greptile}

[greptile-apps]

Unentered span managers are exited

When a later tracer raises during __enter__, this branch calls __exit__ on every manager, including those that never entered. The resulting context-manager error masks the original failure and prevents the already-entered JSONL span from closing, leaving an incomplete trace and aborting the session.

Suggested change

	except BaseException as exc:
	for manager in reversed(managers):
	manager.__exit__(type(exc), exc, exc.__traceback__)
	raise
	except BaseException as exc:
	for manager in reversed(managers[: len(spans)]):
	manager.__exit__(type(exc), exc, exc.__traceback__)
	raise

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/dream/observability/_composite.py
Line: 42-45

Comment:
**Unentered span managers are exited**

When a later tracer raises during `__enter__`, this branch calls `__exit__` on every manager, including those that never entered. The resulting context-manager error masks the original failure and prevents the already-entered JSONL span from closing, leaving an incomplete trace and aborting the session.

```suggestion
        except BaseException as exc:
            for manager in reversed(managers[: len(spans)]):
                manager.__exit__(type(exc), exc, exc.__traceback__)
            raise
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[codeant-ai]

---

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

[greptile-apps]

divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.