Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 20 additions & 15 deletions docs/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,29 +6,34 @@ The Relay Proxy can export metrics via [OpenTelemetry Protocol (OTLP)](https://o

## Available metrics

| Metric | Type | Description |
|--------|------|-------------|
| `launchdarkly.relay.connections` | UpDownCounter | The number of currently active stream connections from SDKs to the Relay Proxy. |
| `launchdarkly.relay.requests` | Counter | The cumulative number of requests received by the Relay Proxy's [service endpoints](./endpoints.md) since startup. |
| `launchdarkly.relay.request.duration` | Histogram | The duration of requests to the Relay Proxy's service endpoints, in seconds. |
| `launchdarkly.relay.events.received.bytes` | Counter | The cumulative number of event bytes received by the Relay Proxy (measured after decompression). |
| Metric | Type | Unit | Description |
|--------|------|------|-------------|
| `http.server.active_requests` | UpDownCounter | `{request}` | The number of currently active stream connections from SDKs to the Relay Proxy. |
| `http.server.request.duration` | Histogram | `s` | The duration of requests to the Relay Proxy's service endpoints, in seconds. |
| `launchdarkly.relay.events.received.size` | Counter | `By` | The cumulative number of event bytes received by the Relay Proxy (measured after decompression). |
| `launchdarkly.relay.events.sent` | Counter | `{event}` | The cumulative number of events successfully sent to LaunchDarkly. |
| `launchdarkly.relay.events.sent.size` | Counter | `By` | The cumulative bytes of event payloads successfully sent to LaunchDarkly. |
| `launchdarkly.relay.events.send.errors` | Counter | `{event}` | The cumulative number of events that failed to send after all retries. |
| `launchdarkly.relay.events.dropped` | Counter | `{event}` | The cumulative number of events dropped due to capacity overflow. |
| `launchdarkly.relay.events.pending` | Gauge | `{event}` | The current number of events buffered in the queue. |

## Attributes

All metrics include the following attributes:

| Attribute | Description |
|-----------|-------------|
| `relayId` | A unique identifier for this Relay Proxy instance, generated at startup. |
| `env` | The name of the LaunchDarkly environment as configured in the Relay Proxy. In automatic configuration or offline mode, this is the actual project and environment name from LaunchDarkly. Example: `MyApplication Staging` |
| `platformCategory` | The kind of SDK that generated the metric: `server`, `mobile`, or `browser`. |
| `userAgent` | The user agent of the SDK making the request. Example: `Node/3.4.0` |
| `sdkWrapper` | The SDK wrapper identifier, if provided. Example: `flutter-client/2.0.0` |
| `route` | The request URL path template. Variables appear as placeholders rather than actual values. Example: `/sdk/evalx/{envId}/contexts/{context}` |
| `method` | The HTTP method. Example: `GET` |
| `relay.id` | A unique identifier for this Relay Proxy instance, generated at startup. |
| `environment.name` | The name of the LaunchDarkly environment as configured in the Relay Proxy. In automatic configuration or offline mode, this is the actual project and environment name from LaunchDarkly. Example: `MyApplication Staging` |
| `platform.category` | The kind of SDK that generated the metric: `server`, `mobile`, or `browser`. |
| `user_agent` | The user agent of the SDK making the request. Example: `Node/3.4.0` |
| `sdk.wrapper` | The SDK wrapper identifier, if provided. Example: `flutter-client/2.0.0` |
| `http.route` | The request URL path template. Variables appear as placeholders rather than actual values. Example: `/sdk/evalx/{envId}/contexts/{context}` |
| `http.request.method` | The HTTP method. Example: `GET` |
| `url.scheme` | The URL scheme. Example: `https` |
| `application.id` | The application identifier, extracted from the `application-id` field of the `X-LaunchDarkly-Tags` header. |
| `application.version` | The application version, extracted from the `application-version` field of the `X-LaunchDarkly-Tags` header. |
| `instanceId` | The SDK instance identifier from the `X-LaunchDarkly-Instance-Id` header. |
| `instance.id` | The SDK instance identifier from the `X-LaunchDarkly-Instance-Id` header. |

## Backend-specific notes

Expand All @@ -42,7 +47,7 @@ OTEL_EXPORTER_OTLP_ENDPOINT=http://<prometheus-host>:9090/api/v1/otlp/v1/metrics
OTEL_EXPORTER_OTLP_PROTOCOL=http
```

Prometheus converts OpenTelemetry metric names by replacing dots with underscores, so the metrics will appear as `launchdarkly_relay_connections`, `launchdarkly_relay_requests_total`, `launchdarkly_relay_request_duration_seconds`, and `launchdarkly_relay_events_received_bytes_total`.
Prometheus converts OpenTelemetry metric names by replacing dots with underscores, so the metrics will appear as `http_server_active_requests`, `http_server_request_duration_seconds`, `launchdarkly_relay_events_received_size_total`, etc.

### OpenTelemetry Collector

Expand Down
4 changes: 2 additions & 2 deletions internal/application/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ func StartHTTPServer(
}
}

errCh := make(chan error)
errCh := make(chan error, 1)

// Create a channel to listen for signals
sigCh := make(chan os.Signal, 1)
Expand All @@ -61,6 +61,7 @@ func StartHTTPServer(
if err != nil && err != http.ErrServerClosed {
errCh <- err
}
close(errCh)
}()

// Handle graceful shutdown in a separate goroutine
Expand All @@ -83,7 +84,6 @@ func StartHTTPServer(
} else {
loggers.Info("Server gracefully stopped")
}
close(errCh) // Close the error channel after shutdown
}()

return srv, errCh
Expand Down
78 changes: 50 additions & 28 deletions internal/metrics/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,19 @@ import (
"time"

"go.opentelemetry.io/otel/attribute"
semconv "go.opentelemetry.io/otel/semconv/v1.26.0"
)

const (
// Metric instrument names.
connMeasureName = "launchdarkly.relay.connections"
requestMeasureName = "launchdarkly.relay.requests"
requestDurationMeasureName = "launchdarkly.relay.request.duration"
eventsReceivedBytesMeasureName = "launchdarkly.relay.events.received.bytes"
eventsSentCountMeasureName = "launchdarkly.relay.events.sent.count"
eventsSentBytesMeasureName = "launchdarkly.relay.events.sent.bytes"
eventsSentFailuresMeasureName = "launchdarkly.relay.events.sent.failures"
eventsSentDroppedMeasureName = "launchdarkly.relay.events.sent.dropped"
eventsSentPendingMeasureName = "launchdarkly.relay.events.sent.pending"
connMeasureName = "http.server.active_requests"
requestDurationMeasureName = "http.server.request.duration"
eventsReceivedMeasureName = "launchdarkly.relay.events.received.size"
eventsSentMeasureName = "launchdarkly.relay.events.sent"
eventsSentSizeMeasureName = "launchdarkly.relay.events.sent.size"
eventsSendErrorsMeasureName = "launchdarkly.relay.events.send.errors"
eventsDroppedMeasureName = "launchdarkly.relay.events.dropped"
eventsPendingMeasureName = "launchdarkly.relay.events.pending"

defaultFlushInterval = time.Minute

Expand All @@ -27,47 +27,69 @@ const (
)

var (
relayIDAttrKey = attribute.Key("relayId") //nolint:gochecknoglobals
platformCategoryAttrKey = attribute.Key("platformCategory") //nolint:gochecknoglobals
userAgentAttrKey = attribute.Key("userAgent") //nolint:gochecknoglobals
sdkWrapperAttrKey = attribute.Key("sdkWrapper") //nolint:gochecknoglobals
routeAttrKey = attribute.Key("route") //nolint:gochecknoglobals
methodAttrKey = attribute.Key("method") //nolint:gochecknoglobals
envNameAttrKey = attribute.Key("env") //nolint:gochecknoglobals
relayIDAttrKey = attribute.Key("relay.id") //nolint:gochecknoglobals
platformCategoryAttrKey = attribute.Key("platform.category") //nolint:gochecknoglobals
userAgentAttrKey = attribute.Key("user_agent") //nolint:gochecknoglobals
sdkWrapperAttrKey = attribute.Key("sdk.wrapper") //nolint:gochecknoglobals
envNameAttrKey = attribute.Key("environment.name") //nolint:gochecknoglobals
applicationIDAttrKey = attribute.Key("application.id") //nolint:gochecknoglobals
applicationVersionAttrKey = attribute.Key("application.version") //nolint:gochecknoglobals
instanceIDAttrKey = attribute.Key("instanceId") //nolint:gochecknoglobals
statusCodeAttrKey = attribute.Key("statusCode") //nolint:gochecknoglobals
instanceIDAttrKey = attribute.Key("instance.id") //nolint:gochecknoglobals

// OTEL HTTP semantic convention attribute keys (from semconv package)
httpRouteAttrKey = semconv.HTTPRouteKey //nolint:gochecknoglobals
httpRequestMethodAttrKey = semconv.HTTPRequestMethodKey //nolint:gochecknoglobals
httpResponseStatusAttrKey = semconv.HTTPResponseStatusCodeKey //nolint:gochecknoglobals
urlSchemeAttrKey = semconv.URLSchemeKey //nolint:gochecknoglobals
networkProtoVersionAttrKey = semconv.NetworkProtocolVersionKey //nolint:gochecknoglobals
errorTypeAttrKey = semconv.ErrorTypeKey //nolint:gochecknoglobals
statusCodeAttrKey = attribute.Key("status_code") //nolint:gochecknoglobals
)

// buildAttributes creates an OTel attribute set from base key-values plus per-request attributes.
// All string values should be pre-sanitized via sanitizeTagValue before calling this function.
func buildAttributes(baseKVs []attribute.KeyValue, platform, userAgent, sdkWrapper string) attribute.Set {
attrs := make([]attribute.KeyValue, len(baseKVs), len(baseKVs)+3)
// buildRequestAttributes creates an OTel attribute set for request metrics using semconv attribute names
// where applicable. All string values should be pre-sanitized via sanitizeTagValue before calling this function.
func buildRequestAttributes(baseKVs []attribute.KeyValue, platform, userAgent, sdkWrapper, route, method, urlScheme, applicationID, applicationVersion, instanceID string) attribute.Set {
attrs := make([]attribute.KeyValue, len(baseKVs), len(baseKVs)+9)
copy(attrs, baseKVs)
attrs = append(attrs,
platformCategoryAttrKey.String(platform),
userAgentAttrKey.String(userAgent),
sdkWrapperAttrKey.String(sdkWrapper),
httpRouteAttrKey.String(route),
httpRequestMethodAttrKey.String(method),
urlSchemeAttrKey.String(urlScheme),
applicationIDAttrKey.String(applicationID),
applicationVersionAttrKey.String(applicationVersion),
instanceIDAttrKey.String(instanceID),
)
return attribute.NewSet(attrs...)
}

// buildRequestAttributes creates an OTel attribute set for request metrics (includes route and method).
// All string values should be pre-sanitized via sanitizeTagValue before calling this function.
func buildRequestAttributes(baseKVs []attribute.KeyValue, platform, userAgent, sdkWrapper, route, method, applicationID, applicationVersion, instanceID string) attribute.Set {
attrs := make([]attribute.KeyValue, len(baseKVs), len(baseKVs)+8)
// buildDurationAttributes creates an OTel attribute set for the http.server.request.duration histogram,
// including all semconv required/conditionally-required attributes.
func buildDurationAttributes(baseKVs []attribute.KeyValue, platform, userAgent, sdkWrapper, route, method, applicationID, applicationVersion, instanceID, urlScheme, protocolVersion, errorType string, statusCode int) attribute.Set {
attrs := make([]attribute.KeyValue, len(baseKVs), len(baseKVs)+14)
copy(attrs, baseKVs)
attrs = append(attrs,
platformCategoryAttrKey.String(platform),
userAgentAttrKey.String(userAgent),
sdkWrapperAttrKey.String(sdkWrapper),
routeAttrKey.String(route),
methodAttrKey.String(method),
httpRouteAttrKey.String(route),
httpRequestMethodAttrKey.String(method),
urlSchemeAttrKey.String(urlScheme),
applicationIDAttrKey.String(applicationID),
applicationVersionAttrKey.String(applicationVersion),
instanceIDAttrKey.String(instanceID),
)
if protocolVersion != "" {
attrs = append(attrs, networkProtoVersionAttrKey.String(protocolVersion))
}
if statusCode > 0 {
attrs = append(attrs, httpResponseStatusAttrKey.Int(statusCode))
}
if errorType != "" {
attrs = append(attrs, errorTypeAttrKey.String(errorType))
}
return attribute.NewSet(attrs...)
}

Expand Down
Loading
Loading