Skip to content

Observability: integrate sc-observability 2.0 full OTel (Tokio) for task-activity visibility; tracks Phase D prerequisites #1600

Description

@randlee

Tracking issue linking atm-core's next observability phase to the sc-observability work it depends on. Rand, 2026-09-26: full visibility on all task activity (every assignment, start, reminder, reset, escalation, close, prompt handoff) plus other daemon metrics, using the full OTel implementation (Tokio), not the lightweight Python-oriented one.

What atm-core has today (develop)

  • Cargo.toml: sc-observability, sc-observability-log, sc-observability-types pinned =1.4.1. Logging only; no sc-observe, no sc-observability-otlp, no spans or metrics.
  • crates/atm-observability/src/lib.rs: RetainedLogger adapter over Logger::builder_typed/try_log_typed/flush_typed.
  • crates/atm-observability/src/tracing_bridge.rs: hand-rolled 700-line tracing-subscriber layer (TracingBridgeLayer), installed by crates/atm-daemon-bootstrap/src/daemon_observability.rs::install_tracing_bridge and registered by diagnostic_timeline.rs for its counters. Exists because 1.4.1 has no tracing/log bridge.
  • Task activity is already typed but never reaches ObservabilityPort: TaskEventKind, TaskTransition, PromptHandoff in crates/atm-storage/src/task_state.rs; writers in atm-storage-rusqlite/src/task_store.rs, atm-http-runtime/src/herdr_queue_wake_escalation.rs, atm-http-runtime/src/prompt_handoff_record.rs.
  • Related fixes in flight: task reminders: budget counts from task start while the assignee is between turns; lead_notified nudge sat unread #1598 (reminder reset, every-10 escalation, deferred escalation nudge, new reminders_reset ledger event), atm task history: the past N tasks of a team, including completed ones #1599 (atm task history).

sc-observability prerequisites (all in Phase D, ships as 2.0; plan on PR randlee/sc-observability#223, beads import randlee/sc-observability#221)

Need sc-observability item Status
Constructor accepting a host-owned Logger so info!/warn!/#[instrument] share the daemon's retained logger (replaces the hand-rolled bridge) randlee/sc-observability#204 → sprint d-2 attach_logger(Arc<Logger>, AttachmentOptions) open, planned
Typed sink registration without the deprecated LogSinkError randlee/sc-observability#203 → sprint d-3 register_typed_sink open, planned
Error enums 2.0 (breaks TryLogFailure/LogFailure/*_typed matches in atm-observability and crates/atm/src/main.rs) randlee/sc-observability#92 → sprints d-4, d-12 open, planned
Neutral signal model (SpanKind, MetricValue::{Sum,Histogram}, ExporterBackend::OpenTelemetrySdk, TelemetryHealth) sprint d-5, d-12 planned
Full OTel backend on Tokio: official opentelemetry SDK, gRPC/HTTP-protobuf, shutdown_async_typed sprints d-6, d-7 planned
Public consumer surface, 2.0 release, migration guide sprint d-18 (d-9 conformance after) planned

Today's sc-observability-otlp 1.4.1 buffers all three signals into no-op exporters with crate-private exporter traits, so nothing can be exported before d-18. The legacy HTTP/JSON path (d-8) is the lightweight implementation and is not atm-core's target. Note: the legacy OTLP source Phase D restores came from atm-core (docs/plans/phase-d/legacy-otlp-provenance.json → randlee/agent-team-mail 7b39f4e7f).

atm-core work that does not wait for Phase D

  1. Define the task-activity Observable payload and the ATM-owned mapping spec (sc-observability docs/atm-adapter-mapping-spec.md §4.2, §4.3, §9 leave span rules to ATM): which task events open/annotate/close a span, counters per TaskEventKind, latency histograms assign→start and start→close, prompt-handoff attempts.
  2. Route the ledger writers above through ObservabilityPort so the events exist as observations before an exporter does.
  3. Prepare the 1.4.1 → 2.0 adapter migration (error enums, attach_logger replacing TracingBridgeLayer while keeping the retained field allowlist and the diagnostic-timeline counters).

Process

Rand's sequence: import the sc-obs skill/agent system into atm-core in one PR, then write this phase's plan as beads with the atm-beads skill. This issue is the anchor the plan links to.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions