Add usage metrics collector for Splunk ingestion#33
Open
Conversation
Implements a data collector that aggregates per-workspace and per-tenant usage metrics (components, relationships, recommendations, requirements) from internal CATIO services and emits structured JSON events to a log file for Splunk ingestion. - New `kportforward collect --once` command for manual/scripted runs - Background collection integrated into kportforward: triggers automatically once services are healthy, runs once per day with idempotency state tracking - gRPC clients wired to architecture-inventory, recommendations, and requirements services using local proto definitions - HTTP client for workspace discovery via environment service - JSON-only output file (~/.config/kportforward/usage-metrics.log) with diagnostic text discarded to keep Splunk ingestion clean - Remote config fallback for collector section to handle deployments where the remote config predates this feature
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
~/.config/kportforward/usage-metrics.logfor Splunk ingestion via the existing stdout logging pipelineWhat's included
kportforward collect --oncecommand for manual or scripted runscatio-tech/protosCI status
CI is currently failing because this public repo depends on
catio-tech/protos(private) via areplacedirective ingo.mod. The GitHub Actions runner does not have access to it. The code works correctly locally. This will be resolved when the collector is moved to a private repo as discussed.Test plan
.\bin\kportforward.exe collect --once --force 2>$nulland verify JSON events are emitted to stdout.\bin\kportforward.exeand verify~/.config/kportforward/usage-metrics.logis created and contains only JSON lines after services come upcomponent_countandrelationship_countare non-zerogo test ./...— all tests pass