Skip to content

Refactor BYOO collector configuration for log chunking and resource overrides #273

Description

@sbaum1994

Summary

Refactor NVCA BYOO collector configuration so BYOO log chunking and BYOO telemetry resource overrides are first-class configurable settings. Operators should be able to tune these values through the NVCA operator configuration without patching generated workloads or relying on scattered implementation-only environment variables.

This Epic is informed by the config surface being added in #194 and should make that surface complete, documented, and safe for supported deployments.

Context from #194

PR #194 is adding more than basic log chunk and resource tuning. It introduces or updates:

  • Structured agent.byooOtelCollector configuration in NVCA operator values.
  • Helm values, schema, and README documentation in both NVCA operator chart locations.
  • Agent config merge support for BYOO OTel collector settings.
  • BYOO_OTEL_COLLECTOR_CONFIG_B64 as the structured BYOO collector config handoff, while preserving existing BYOO log chunking environment variables.
  • Exporter helper timeout, retry, sending queue, and sending queue batch settings.
  • OTel memory_limiter processor overrides.
  • Metrics/traces batch processor overrides.
  • A separate logs-only logBatch processor so log chunk behavior can be tuned independently from metrics and traces.
  • Collector service telemetry log settings.
  • Optional debug exporter fanout for generated pipelines.
  • Config encode/decode, operator merge, and collector renderer tests.

Scope

  • Define the supported NVCA configuration schema for BYOO log chunking, including max body bytes, dry run, exporter batch sizing, and logs-only batch settings.
  • Define the supported NVCA configuration schema for BYOO telemetry resource overrides, including Kubernetes requests and limits for BYOO collector components and related telemetry containers where applicable.
  • Preserve backward compatibility for the existing BYOO log chunking env vars while moving operators toward the structured config path.
  • Plumb the supported settings through NVCA operator Helm values, schema, generated docs, agent config merge, and workload rendering.
  • Validate that log-specific batch tuning does not unintentionally change metrics or traces pipelines.
  • Add validation or guardrails for invalid values where practical, especially resource overrides and batch sizing.
  • Add focused tests for config encoding, decoding, chart value merge behavior, collector rendering, and resource override propagation.
  • Document supported defaults, examples, and upgrade behavior.

Acceptance criteria

  • BYOO log chunking can be configured from NVCA operator configuration without code changes.
  • BYOO telemetry resource requests and limits can be configured from NVCA operator configuration without manual workload patches.
  • Existing deployments that rely on current defaults continue to work without configuration changes.
  • Metrics/traces batching and logs-only batching can be tuned independently.
  • Documentation and tests cover the supported public configuration surface.

Component

NVCA

References

Metadata

Metadata

Labels

No labels
No labels

Type

Fields

No fields configured for Epic.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions