[meta] Flakey bundled plugins from #logstash-plugin-ci

[donoghuc]

Flaky test analysis — logstash-plugins scheduled CI failures

Source: oblt-robot-ci Slack failure notifications. Investigation via gh CLI (read-only)
against each run's --log-failed output and job matrix. Date of analysis: 2026-08-20.

Method

For each run the failed-job log was downloaded and the RSpec summary, failing example
name, and error signature extracted. The job matrix was inspected to see whether the
failure was isolated to a single stack/branch cell (a strong flaky signal) or reproduced
across all cells.

Classification key

• Test race — intermittent, timing-dependent assertion or socket/thread race in the spec.
• Concurrency — thread-safety defect (unsynchronized shared state) crashing a worker thread.
• Hang/timeout — job killed by the 60-minute step timeout; test deadlocks/hangs.
• Infra — CI environment failure (Docker image pull i/o timeout, missing DRA build, setup step); not a plugin/test bug.

Summary

Plugin	Runs (IDs)	Failing test / step	Signature	Verdict	Assignee	PR	Merged
logstash-output-tcp	31783808676, 32009100050	spec/outputs/tcp_spec.rb:194 register	register raises ConfigurationError; ECONNREFUSED on client connect	Flaky — test race	@andsel	logstash-plugins/logstash-output-tcp#73	✅
logstash-integration-snmp	31783846781, 32009105537, 32347392451	Setup / install step	Install failed with an unexpected code: 1 (Docker)	Flaky — infra	@andsel	logstash-plugins/logstash-integration-snmp#98	✅
logstash-input-elasticsearch	31784088299	Setup / image pull	docker.elastic.co … i/o timeout (ES 9.5.1)	Flaky — infra	@andsel	NO FIX	spurious
logstash-output-elasticsearch	31784049577, 31873619940, 32347652832	Integration specs (index template / bulk)	Connection refused … port 9200 — ES not ready	Flaky — infra/env	@andsel	logstash-plugins/logstash-output-elasticsearch#1285	✅
logstash-input-tcp	31784048596, 32114939230, 32231316548	Run tests step	timed out after 60 minutes; connect-refused retry loop	Flaky — hang/timeout	@andsel	issue PR
logstash-input-ganglia	31873726335, 31935726847, 32115237611	spec/inputs/ganglia_spec.rb:61/73	UDP "should receive the correct data/value/name"	Flaky — test race
logstash-filter-translate	31873735909, 31935728649, 32115232167	spec/filters/scheduling_spec.rb:197	reload "expected 11, got 1"	Flaky — test race	@andsel	logstash-plugins/logstash-filter-translate#117	✅
logstash-mixin-scheduler	31873773968, 31935751921, 32009785840	Run tests step	timed out after 60 minutes	Flaky — hang/timeout	@andsel	issue
logstash-output-http	31935473675	spec/outputs/http_spec.rb:496	"keeps retrying due a protocol exception"	Flaky — test race	@andsel	investigation PR issue
logstash-input-file	31935589000, 32114901771	spec/inputs/file_read_spec.rb:345	"cleans up sincedb entry"	Flaky — test race
logstash-input-elastic_serverless_forwarder	31935741498	SSL client-auth cert-rejection examples	intermittent bad_certificate/certificate_required	Flaky — test race (TLS)
logstash-output-lumberjack	32009744611	Run tests step	0 failures but exit 1; server thread RuntimeError: crashed	Flaky — thread exception leak	@donoghuc	logstash-plugins/logstash-output-lumberjack#37	✅
logstash-integration-kafka	32009180665, 32409673361	spec/integration/outputs/kafka_spec.rb:177	"distribute events to all partitions"; Unable to poll Kafka consumer + Fatal assertion	Flaky — test race
logstash-input-beats	32009491400	Run tests step	0 failures but exit 1; Zlib::GzipFile::Error: Unexpected end of ZLIB input stream	Flaky — test race	@andsel	logstash-plugins/logstash-input-beats#578	✅
logstash-input-redis	32114916361	spec/inputs/redis_spec.rb:35	"should read events from a list"; events[1000] nil → NoMethodError	Flaky — test race
logstash-input-http_poller	32115257090	Setup test environment	single cell failed in 9s, 15 cells passed	Flaky — infra/setup
logstash-codec-multiline	32231288604	spec/codecs/identity_map_codec_spec.rb	ConcurrencyError: … unsynchronized modifications	Flaky — concurrency	@andsel	logstash-plugins/logstash-codec-multiline#79	✅
logstash-input-http	32347870269	Setup / image pull	docker.elastic.co … i/o timeout (logstash 8.19.20)	Flaky — infra

All 18 plugins are flaky; none show a deterministic, reproducible product bug. They split
into three buckets: test races/concurrency (11), hang/timeout (2), and CI infra
(5).

---

Per-plugin detail

logstash-output-tcp — flaky (test race)

• Runs: 31783808676 (Stack 8.current), 32009100050 (Stack 9.previous snapshot).
• Only one matrix cell failed; every other stack/branch passed the same commit.
• 36 examples, 1 failure. Failing example at spec/outputs/tcp_spec.rb:194:
  expect { subject.register }.to_not raise_error → got LogStash::ConfigurationError.
• Also Errno::ECONNREFUSED connecting to localhost:1000 from tcp_spec.rb:54.
• Signature is a register/SSL-setup + client-connect race in the spec harness, not a code defect.

logstash-integration-snmp — flaky (infra)

• Runs: 31783846781 (unit), 32009105537 (integration), 32347392451 (integration).
• No RSpec failures. Fails at environment setup with Install failed with an unexpected code: 1.
• Docker/DRA image acquisition failure — CI infrastructure, not a test bug.

logstash-input-elasticsearch — flaky (infra)

• Run: 31784088299.
• failed to do request: Head "https://docker.elastic.co/v2/elasticsearch/elasticsearch/manifests/9.5.1": dial tcp … i/o timeout.
• Image-pull timeout during setup. Infra.

logstash-output-elasticsearch — flaky (infra/env)

• Runs: 31784049577, 31873619940, 32347652832 (all integration).
• 136 examples, 1–2 failures. Repeated curl: (7) Failed to connect to elasticsearch port 9200: Connection refused.
• The ES container is not ready/healthy when the integration specs run — environment timing, not plugin logic.

logstash-input-tcp — flaky (hang/timeout)

• Runs: 31784048596 (main snapshot), 32114939230 (9.previous), 32231316548 (9.previous).
• The action 'Run tests' has timed out after 60 minutes.
• Preceded by Connection refused - connect(2) for "127.0.0.1" port … retry loops; a spec deadlocks/hangs on a socket, never completing.

logstash-input-ganglia — flaky (test race)

• Runs: 31873726335, 31935726847, 32115237611.
• 7 examples, 1 failure, and the failing example varies between cells: ganglia_spec.rb:61 "should receive the correct data", :73 "should receive the value", and "should receive the name".
• UDP receive-timing race: the assertion runs before the datagram is processed. The varying failing example across cells confirms non-determinism.

logstash-filter-translate — flaky (test race)

• Runs: 31873735909, 31935728649, 32115232167.
• 55 examples, 1 failure at spec/filters/scheduling_spec.rb:197 — "scheduled reloading merge uses the merged dictionary after reload":
  expected: "11", got: "1".
• The assertion samples the dictionary before the scheduled reload thread has merged the update — scheduler timing race.

logstash-mixin-scheduler — flaky (hang/timeout)

• Runs: 31873773968, 31935751921, 32009785840.
• The action 'Run tests' has timed out after 60 minutes. A scheduler spec hangs and never returns.

logstash-output-http — flaky (test race)

• Run: 31935473675.
• 264 examples, 1 failure at spec/outputs/http_spec.rb:496 — "with supported_protocols set to (disabled) 1.1 keeps retrying due a protocol exception".
• Retry-count/timing expectation that depends on how many retries elapse in a fixed window — race.

logstash-input-file — flaky (test race)

• Runs: 31935589000, 32114901771.
• 143 examples, 1 failure at spec/inputs/file_read_spec.rb:345 — "delete on complete with sincedb cleanup enabled cleans up sincedb entry".
• Filesystem/sincedb-flush timing race between the reader thread and the assertion.

logstash-input-elastic_serverless_forwarder — flaky (test race, TLS)

• Run: 31935741498.
• 36 examples, 1 failure, 2 pending. Failing examples rotate among the SSL client-authentication cert-rejection cases (ssl_client_authentication => required/optional, self-signed / mismatched SAN, ssl_verification_mode => full) expecting a bad_certificate/certificate_required rejection.
• TLS handshake-timing race: the rejection sometimes surfaces as a different/absent error than asserted.

logstash-output-lumberjack — flaky (thread exception leak)

• Run: 32009744611.
• RSpec reports 2 examples, 0 failures but the job exits 1.
• Lumberjack server threads terminate with RuntimeError: crashed / Whoa we shouldn't get this frame: (jls-lumberjack … server.rb:70). These are deliberately-injected failures in lumberjack_spec.rb, but with report_on_exception the thread crash escapes and forces a non-zero process exit — an unclean-teardown flake, not an assertion failure.

logstash-integration-kafka — flaky (test race)

• Runs: 32009180665 (main), 32409673361 (11.x).
• 44 examples, 1 failure at spec/integration/outputs/kafka_spec.rb:177 — "when using multi partition topic should distribute events to all partitions".
• Unable to poll Kafka consumer and Exception in thread "kafka-input-worker-test-0" java.lang.AssertionError: Fatal assertion — broker/partition-assignment timing race; partition distribution not yet balanced when asserted.

logstash-input-beats — flaky (test race)

• Run: 32009491400.
• Both cells report 262 examples, 0 failures, yet the job is red.
• Zlib::GzipFile::Error: Unexpected end of ZLIB input stream — a compressed-payload spec crashed on a truncated/incomplete stream (network/timing), aborting the process after the summary.

logstash-input-redis — flaky (test race)

• Run: 32114916361.
• 6 examples, 1 failure at spec/inputs/redis_spec.rb:35 — "should read events from a list":
  expect(events[1000].get('sequence')).to eq(1000) → NoMethodError (events[1000] is nil).
• Not all 1001 events were delivered before the assertion; also IOError: closed stream during teardown. Data-delivery timing race.

logstash-input-http_poller — flaky (infra/setup)

• Run: 32115257090.
• Only Stack 9.current (5.x branch) failed, at Setup test environment after 9s; the other 15 matrix cells passed. Environment-setup infra flake.

logstash-codec-multiline — flaky (concurrency)

• Run: 32231288604.
• Exit code 1 with ConcurrencyError: Detected invalid array contents due to unsynchronized modifications with concurrent users from the IdentityMapCodec#map_cleanup thread (spec/codecs/identity_map_codec_spec.rb:134).
• Genuine thread-safety defect in the identity-map cleanup path exercised by concurrent spec threads; manifests intermittently.

logstash-input-http — flaky (infra)

• Run: 32347870269.
• docker.elastic.co/logstash/logstash:8.19.20 … Head … i/o timeout. Image-pull timeout during setup. Infra.

---

Notes

• The five infra cases (snmp, input-elasticsearch, output-elasticsearch, http_poller, input-http) are Docker image-pull / container-readiness / setup timeouts and are not fixable in plugin code; they recur across many repos and point at registry/runner flakiness.
• The hang/timeout cases (input-tcp, mixin-scheduler) waste a full 60-minute runner each; worth prioritising as they are the most expensive.
• logstash-codec-multiline's ConcurrencyError is the one signature that reflects a real (thread-safety) bug rather than pure test timing, even though it surfaces intermittently.

[andsel]

logstash-input-elasticsearch

This kind of error

  --------------------
  
     1 |     ARG ELASTIC_STACK_VERSION
  
     2 | >>> FROM docker.elastic.co/elasticsearch/elasticsearch$distribution_suffix:$ELASTIC_STACK_VERSION
  
     3 |     
  
     4 |     ARG plugin_path=/usr/share/plugins/plugin

happens only when the image is not available on the docker repository. I think this is due to some delay in publishing the image.

1. ES image 9.5.1 was created on 2026 Aug 5th https://www.docker.elastic.co/r/elasticsearch/elasticsearch:9.5.1
2. Logstash switched version in CI to 9.5.1 on 2026 Aug 13th 9:45 GTM+2 logstash-plugins/.ci@88fc2c7
3. Test failure on this plugin happened 2026 Aug 14th 10:29 GMT+2 https://github.com/logstash-plugins/logstash-input-elasticsearch/actions/runs/31784088299/job/94716005152

I think there were some lag between docker image creation and publish, but can effectively find the date the ES image was published (not created).

[donoghuc]

Regarding #19452 (comment) I noticed some issues here as well. I believe it was due to a disruption in hosted image availability when there was such fast turn around between releases in the 9.5 stream. Given that seems to have stabilized I think we should be good. Essentially the problem seemed to arisen from SNAPSHOT builds (pointed to in .ci for a new release not being available while the "n - 1" version has been pulled/obsolete.