Skip to content

Collector cannot forward profiles: otelcol 0.130 rejects the OTLP profiles schema Pyroscope v2 needs #439

Description

@iamsharduld

Profiles pushed through the OpenTelemetry Collector never reach Pyroscope. The collector rejects the payload outright:

StatusCode.INTERNAL
details = "grpc: error unmarshalling request: proto: wrong wireType = 2 for field LocationsLength"

LocationsLength is a field from the older OTLP profiles schema. Pyroscope v2 parses the newer stack-table shape, where a sample carries a stack_index into a ProfilesDictionary.stack_table rather than locations_start_index/locations_length. So a generator that satisfies Pyroscope v2 cannot satisfy otelcol 0.130, and the reverse holds too. The two are mutually exclusive.

Reproduced by running scripts/profilegen.py directly against a deployed collector's OTLP port. Pointing the same generator at the coordinator works, so it's the collector that can't read it.

The collector charm ships 0.130 on every channel we can use: 0.130/stable is rev 211, and even dev/edge (rev 236) still pins ubuntu/opentelemetry-collector:0.130-26.04. A 0.155 image exists on Docker Hub but only for 24.04, and no charm channel uses it.

So this needs an opentelemetry-collector-k8s image on a version that understands the newer schema. Related: canonical/opentelemetry-collector-k8s-operator#334, and #335 which attempted the bump and was closed.

Until then, four tests are skipped in #435: test_emit_profile_to_collector and test_ingest_profile_from_collector in both test_profiling_with_collector and test_profiling_with_collector_tls. Everything else in those modules passes, so the collector deploys and integrates fine, it just cannot carry profiles.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions