Skip to content

docs: add delta-v telemetryd architecture document - #401

Merged
pbrane merged 1 commit into
developfrom
docs/telemetryd-architecture
Jul 17, 2026
Merged

pbrane merged 1 commit into
developfrom
docs/telemetryd-architecture

Conversation

@pbrane

@pbrane pbrane commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary

Adds docs/architecture/deltav-telemetryd-architecture.md documenting the telemetry ingestion architecture:

  • The three-tier pipeline: Minion (UDP listener + 4-byte protocol sniff) → minion-gateway (per-protocol gRPC → per-protocol Kafka topics) → flow-enricher (SCS consumer → ClickHouse), with telemetryd as a vestigial/extension-seat bridge (all adapters disabled).
  • Protocol-to-protocol coupling analysis: per-protocol isolation end-to-end (topics, gRPC methods, sink modules, processors) vs. the three shared chokepoints (single UDP port, shared gateway plumbing, deploy-time config).
  • Minion coupling: the two thin wire contracts (TelemetryService gRPC proto, DeltaV.Sink.* + SinkMessage framing) and the content-blind design.
  • Horizontal scaling via Spring Cloud Stream: flow-enricher and prometheus-writer as SCS consumers (consumer-group scale-out), the SCS producers feeding them (collectd, pollerd/perspectivepollerd via poller-timeseries-common, provisiond), and the scaling ceilings (4 partitions per topic, <location>@<minion-id> key skew, raw kafka-clients services that scale manually).
  • Mermaid diagram of the full pipeline.

Documentation only — no code changes.

https://claude.ai/code/session_01E1cRKma4wRcjD6FmtVriA3

Describes the Minion -> minion-gateway -> Kafka -> flow-enricher ingestion
pipeline, per-protocol coupling and shared chokepoints, the Minion wire
contracts, and which backend components scale horizontally via Spring
Cloud Stream consumer groups.

Claude-Session: https://claude.ai/code/session_01E1cRKma4wRcjD6FmtVriA3
@github-actions github-actions Bot added the docs label Jul 17, 2026
@pbrane
pbrane merged commit e2834f9 into develop Jul 17, 2026
1 check passed
@pbrane
pbrane deleted the docs/telemetryd-architecture branch July 17, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant