Skip to content

[bug] Kafka client creation errors print the rejected rdkafka value #774

Description

@MarcusKainth

Where

Kafka sink (spate-kafka)

Version

main (8d023b6)

Cargo features

kafka

Rust version and platform

rustc 1.96, any platform

What happened

When librdkafka rejects an rdkafka: passthrough value, the Kafka sink's startup error prints that value. build (crates/spate-kafka/src/sink/config.rs) formats the error as sink.kafka: producer creation failed: {e}, and the probe producer's error takes the same form. rdkafka's KafkaError::ClientConfig displays as Client config error: {desc} {key} {value}, and librdkafka's own desc quotes it again for an enumerated property. The Kafka source's consumer creation (crates/spate-kafka/src/source.rs, fatal("create consumer")) carries the same error.

${VAR} interpolation runs before parsing, so a secret placed in the wrong property, or in a property with a typed value, ends up in the message. #764 removes values from config type errors in spate-core. This path does not go through that code.

What you expected instead

The error names the property and why librdkafka rejected it, and does not print the value.

Reproduction

// In crates/spate-kafka/src/sink/config.rs tests:
let body = format!("{}  rdkafka:\n    compression.type: hunter2\n", minimal());
let err = build(parse(&body).unwrap()).unwrap_err().to_string();
assert!(!err.contains("hunter2"), "{err}");
// fails: invalid configuration: sink.kafka: producer creation failed: Client config error:
// Invalid value "hunter2" for configuration property "compression.codec" compression.type hunter2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions