[EPIC] Establish suite-wide observability and CLI experience

[szmyty]

Outcome

Establish one privacy-preserving, automation-safe observability model for flow, aniflow, optiflow, and renderflow, covering operational logs, metrics, distributed traces, diagnostics, and correlation across in-process and bounded-process execution.

The suite should be easy to debug locally, observable through OpenTelemetry when explicitly enabled, and consistent for both people and orchestration systems without confusing telemetry with durable run evidence.

This epic is a horizontal capability track under flow suite roadmap #11. It coordinates shared semantics here while preserving independent holon ownership and release lifecycles.

Why this matters

The suite performs long-running, resource-intensive, resumable work across provider and process boundaries. Operators need to answer:

• Which product, command, run, stage, provider, capability, and artifact was involved?
• Where was time spent and what resource pressure was observed?
• Which dependency, validation, or lifecycle boundary failed?
• What work remains valid and what is the exact safe recovery action?
• Can one suite run be followed across flow and participating holon processes?

Current behavior is uneven: renderflow already uses Rust tracing, while aniflow and optiflow have stronger typed result/evidence boundaries than general-purpose operational telemetry. This track converges the experience without replacing their domain contracts.

Non-negotiable boundaries

• Own suite policy and conventions; use established instrumentation and OpenTelemetry ecosystems instead of building a logging engine.
• Public libraries remain subscriber-agnostic. CLI/application boundaries install subscribers or exporters.
• Versioned command results own standard output. Human logs, progress, and diagnostics use standard error or an explicitly selected sink.
• Machine-result streams never contain banners, color, progress, or log records.
• Telemetry is operational observation, not artifact evidence, provider success, validation proof, checkpoint compatibility, provenance authority, or plan input.
• Timestamps, span IDs, trace IDs, ephemeral run IDs, exporter state, and sampling decisions never affect deterministic plans, cache keys, compatibility fingerprints, or artifact digests.
• Source content, prompts, captions, media bytes, secrets, environment values, authorization headers, complete argv, and unnecessary absolute paths are excluded by default.
• High-cardinality attributes are bounded and reviewed. Raw artifact or user identifiers do not become metric labels.
• Export is disabled by default. Local operation remains fully functional without network access, a collector, an account, or telemetry configuration.
• Exporter failure cannot silently change a successful media operation into success or failure. Any strict observability policy must be explicit and typed.
• Cross-process context propagation is explicit, allowlisted, versioned where contractual, and sent only to participating suite components.
• No mutable branch, path dependency, copied sibling source, or direct sibling-holon dependency.
• Accessibility does not depend on color, animation, icons, or decorative ASCII.

Target capability

The completed track should provide:

• structured Rust spans and events with a reviewed event/attribute taxonomy;
• concise human logs and optional structured JSON Lines logs;
• consistent verbosity, filtering, color, terminal detection, and sink selection;
• trace/log correlation across flow and opted-in holon processes;
• OpenTelemetry logs, metrics, and traces through optional OTLP exporters;
• W3C trace-context compatibility where cross-process propagation is selected;
• stable service/resource attributes and bounded domain-neutral correlation fields;
• latency, throughput, retry, cancellation, validation, provider, queue, and resource-pressure measurements with controlled cardinality;
• safe shutdown and exporter flushing without hanging CLI completion;
• in-memory and local-collector conformance tests requiring no public network;
• privacy/redaction tests and an explicit diagnostic-bundle preview workflow;
• consistent, accessible CLI help presentation, including optional compact static wordmarks for interactive root help;
• operational documentation for local debugging and collector integration.

Architecture and repository decision gate

Do not create a shared crate or repository merely from anticipated duplication.

First define the suite profile and prove it in at least two materially different CLIs. Then record an ADR choosing among:

1. small repository-local adapters governed by one conformance profile;
2. a separately published, semver-versioned neutral crate;
3. a new independently released observability/CLI-support repository.

The decision must evaluate MSRV spread, dependency weight, feature flags, release coupling, embedding APIs, OpenTelemetry version churn, supply-chain/security ownership, and rollback. If a shared crate is selected, consumers pin immutable releases rather than mutable Git branches.

Planned sequence

Execute the child issues in order unless their dependency sections explicitly allow parallel work:

• FLO-OBS-01 / #15 — Define the suite observability, privacy, and stream contract.
• FLO-OBS-02 / #16 — Prove the observability profile in two holon CLIs.
• FLO-OBS-03 / #17 — Prove opt-in OpenTelemetry logs, metrics, and traces.
• FLO-OBS-04 / #18 — Decide shared observability crate and repository ownership, including explicit evaluation of the existing egohygiene/observatory boundary.
• FLO-OBS-05 / #19 — Build the selected reusable observability package if the ADR chooses extraction.
• FLO-OBS-06 / #20 — Migrate the suite and enforce cross-product conformance.
• FLO-OBS-07 / #21 — Add privacy-safe diagnostics and operator runbooks.
• FLO-OBS-08 / #22 — Add accessible branded root-help presentation.

The shared-package checkpoint is deliberately conditional. A repository must not be created merely to satisfy the checklist; #18 may select repository-local adapters or place the capability in an existing neutral owner.

Suite definition of done

• Every supported CLI preserves its documented stdout/stderr and exit contracts under human, JSON, redirected, and non-interactive execution.
• One run can be correlated across participating flow and holon boundaries without parsing human prose.
• OpenTelemetry logs, metrics, and traces can reach a local collector when explicitly configured and remain entirely inactive otherwise.
• Collector absence, exporter backpressure, shutdown, and partial export have tested, honest behavior.
• Redaction and bounded-cardinality tests cover representative paths, provider diagnostics, configuration, and process failures.
• Telemetry cannot establish stage completion, validated output, provenance, or checkpoint reuse.
• Libraries remain embeddable without installing or replacing the caller's global subscriber.
• Stable/MSRV/platform tests and local collector integration tests pass without requiring internet access.
• Any shared dependency is independently versioned, pinned, documented, and reversible.
• Help branding remains human-only, compact, accessible, snapshot-tested, and absent from machine streams.
• Documentation explains local debugging, opt-in export, retention/privacy implications, and recovery from observability failures.

Explicit non-goals

• A proprietary telemetry backend or hosted observability service.
• Export enabled by default.
• Uploading diagnostic bundles automatically.
• Treating logs or spans as durable domain evidence.
• Persisting source content or secrets for debugging convenience.
• Replacing holon-native result, event, run-manifest, or validation contracts.
• Kernel/container isolation, distributed scheduling, or remote provider execution.
• Continuous profiling or high-volume media-content capture without a separately justified issue.
• One cross-repository mega-PR.

Working protocol

Each child issue must pin current repository state, keep implementation changes in the owning repository, use one focused PR per repository, run repository-local validation plus suite conformance, and link exact evidence back here. Do not close this epic until every surviving child is delivered or explicitly superseded with evidence.