From 79479397747278c3f8f8d97f304ee64e2c56939d Mon Sep 17 00:00:00 2001 From: logstashmachine <43502315+logstashmachine@users.noreply.github.com> Date: Wed, 15 Oct 2025 23:14:44 +0000 Subject: [PATCH 01/11] Update release notes for 9.2.0 --- docs/release-notes/index.md | 158 ++++++++++++++++++++++++++++++++++++ 1 file changed, 158 insertions(+) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index b9d8ee463a..6398f5e160 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -21,6 +21,164 @@ To check for security updates, go to [Security announcements for the Elastic sta % ### Fixes [logstash-next-fixes] % * +## 9.2.0 [logstash-9.2.0-release-notes] + +### Features and enhancements [logstash-9.2.0-features-enhancements] + +---------- GENERATED CONTENT STARTS HERE ------------ +=== Logstash Pull Requests with label v9.2.0 + +=== Logstash Commits between 9.2 and 9.1.5 + +Computed with "git log --pretty=format:'%h -%d %s (%cr) <%an>' --abbrev-commit --date=relative v9.1.5..9.2" + +3cf4222b4 - (HEAD -> 9.2, origin/9.2) Adds integration test for the `_health_report` and `_node/plugins` APIs. (#18306) (#18311) (8 hours ago) +d32268785 - Update rack to 3.2.3 (#18300) (31 hours ago) +da313459d - Doc: Update docs for testing for boolean fields (#18271) (#18292) (8 days ago) +bf6cf7b88 - Update rack rubygem dep to 3.2.2 (#18293) (8 days ago) +6b7e1cc01 - Remove redundant testing and circular dependency from docker acceptance testing (#18181) (#18255) (8 days ago) +06a09d01e - Downgrade gradle to coninute testing on windows server 2016 (#18263) (#18278) (8 days ago) +ee9a0ec3a - ci: remove sonarqube (#18273) (#18286) (9 days ago) +98c54cf44 - [9.2] (backport #18247) Release notes for 9.0.8 (#18269) (9 days ago) +f333f73ba - Release notes for 9.1.5 (#18248) (#18266) (9 days ago) +e9df375de - Ensure docs gen inserts at correct place in file (#18250) (#18259) (9 days ago) +587d2560b - Take up 9.2 version of elastic_integration plugin (#18261) (13 days ago) +bcdc991e6 - Add lockfile for new 9.2 branch (#18242) (2 weeks ago) +4934f0839 - (origin/mergify/bp/main/pr-18240) Fix heading (#18237) (2 weeks ago) +0f8c7ba06 - Documentation for batch's event metrics (current and average) (#18017) (2 weeks ago) +425478cd6 - metrics: add gauge with compression goal if enabled (#18230) (2 weeks ago) +1b3b3eeb0 - Pq compression user metrics (#18227) (2 weeks ago) +88b853aff - (origin/mergify/bp/main/pr-18168) metric: improve accuracy of timer metric under contention (#18219) (2 weeks ago) +75eca8475 - metrics: add support for user-defined metrics (#18218) (2 weeks ago) +a1522385b - PQ: Add support for event-level compression using ZStandard (ZSTD) (#18121) (2 weeks ago) +0f9b2fe46 - [DOCS] Fix substitution variables (#18224) (3 weeks ago) +46bf75eed - (origin/mergify/bp/main/pr-18223) [Docs] Add hyphen to the pipeline ID restriction description. (#18216) (3 weeks ago) +fdeb6a0b9 - Implements current batch event count and byte size metrics (#18160) (3 weeks ago) +2fab5f469 - Replace buildkite jdk version check w/GH action (#17945) (3 weeks ago) +ecaf1fdd4 - Cover the warn password policy usage of ValidatedPassword setting (#18203) (3 weeks ago) +ad200323b - GH action for updating logstash version (#18035) (3 weeks ago) +132f917ea - Update logstash_project_board.yml (#18116) (3 weeks ago) +e6e7f87a8 - Remove uncommented line erroneously committed. (#18204) (3 weeks ago) +d29021800 - Measure average batch byte size and event count (#18000) (3 weeks ago) +105eecca7 - (origin/mergify/bp/main/pr-18021) Revert "Moved Ruby Password setting to Java implementation (#18183)" (#18199) (3 weeks ago) +c2851a958 - bk: update junit-annotate to run faster (#18193) (3 weeks ago) +ad42337ee - Bump anchore/scan-action from 6 to 7 in the github-actions group across 1 directory (#18195) (3 weeks ago) +f3212484f - PQ settings refactor: propagate builder upward (#18180) (4 weeks ago) +8228fe5dc - Release notes for 9.1.4 (#18159) (#18190) (4 weeks ago) +6aa265dcd - Moved Ruby Password setting to Java implementation (#18183) (4 weeks ago) +cd71a4b18 - Rename Java settings classes to Setting (#18171) (4 weeks ago) +08db1d19f - Adds a required type tag to the open API spec of Logstash APIs. (#18169) (4 weeks ago) +25c2551c8 - Release notes for 9.0.7 (#18158) (#18178) (4 weeks ago) +b8bc4d8de - Move Port and PortRange Ruby settings to Java (#17964) (4 weeks ago) +089361de5 - [main] (backport #18028) Release notes for 9.0.6 (#18173) (4 weeks ago) +041e895c5 - Release notes for 9.1.3 (#18029) (#18174) (4 weeks ago) +809ed0029 - Doc: Refine serverless and hosted content (#18166) (4 weeks ago) +feabee287 - (origin/mergify/bp/main/pr-18165) Doc: Add Logstash-to-serverless page (#18034) (4 weeks ago) +556366d87 - Fix race condition with version bump action (#18150) (5 weeks ago) +096f1a381 - Downgrade jruby to 9.4.13.0 (#18136) (5 weeks ago) +1e127fdda - Fix aarch64 acceptance tests (#18135) (5 weeks ago) +e16200a63 - Bump the github-actions group across 1 directory with 2 updates (#18133) (5 weeks ago) +6a51c82e4 - test: explicitly load ascii fixture as ascii, do line-oriented parsing (#18124) (6 weeks ago) +fe4a735c6 - Update JRuby version to 9.4.14.0 (#18080) (6 weeks ago) +ba0787817 - pq: activate stringref extension for more-compact PQ representation (#17849) (6 weeks ago) +ef97a87d8 - Ensure any file object in a tar archive has an mtime (#18113) (6 weeks ago) +d2baf9fbc - github-actions: support backport labels after being merged (#18102) (6 weeks ago) +d9b95bfd8 - Preserve mtime explicitly when creating tar artifacts (#18091) (6 weeks ago) +23d06e665 - Doc: Provide accurate post-geoip lookup example (#18092) (6 weeks ago) +5fb563473 - Fix gauge type correctly in pipelines metrics. (#18082) (6 weeks ago) +f2621b00e - Ensure acceptance test artifacts are being built for correct arch (#18079) (6 weeks ago) +398a66cbe - upgrade golang to 1.25 (#18083) (6 weeks ago) +386869a69 - make gradle files ready for gradle 10 (#17947) (6 weeks ago) +895cfa5b1 - (origin/mergify/bp/main/pr-18077) Add Mergify rule for forwardporting documentation changes (#17874) (7 weeks ago) +e3055d14e - replace placeholder URLs (#18070) (7 weeks ago) +f2e494843 - Build artifacts only for those that match host arch (#17995) (7 weeks ago) +e676b59ac - Start observabilitySRE container builds immediately on DRA (#18016) (7 weeks ago) +91a9527e5 - Stop waiting on ALL steps before staring junit annotation (#18049) (7 weeks ago) +ef41e83f7 - Add stub workflow file to iterate on (#17993) (7 weeks ago) +19d991416 - Split integration tests into more parts (#18036) (7 weeks ago) +5f7a6cdc1 - fixing thin and zero width white space characters in docs (#18019) (8 weeks ago) +34a7c6769 - [DOCS] Fix nullable type linting error in OpenAPI document (#18018) (8 weeks ago) +75abd8a42 - [main] (backport #17781) Harmonize observability sre acceptance (#18001) (8 weeks ago) +bc4645cee - adding x-metatags to openapi spec (#17854) (8 weeks ago) +26027a1d8 - add pipeline id to thread context during init/start (#17944) (8 weeks ago) +50fc6458d - Bump actions/checkout in the github-actions group across 1 directory (#17996) (8 weeks ago) +7d4f476e6 - Update deploying-scaling-logstash.md (#17999) (#18002) (8 weeks ago) +4512e57e6 - Put back the full start requirement which can be managed according to scenario requirement. (#17994) (9 weeks ago) +74c7ec266 - [health-report CI] Print Logstash logs when pipeline faces an issue. (#17991) (9 weeks ago) +22f6b69f4 - Preserve coerce behavior from Ruby impl (#17992) (9 weeks ago) +332355d68 - Start all exhastive tests in parallel (#17978) (9 weeks ago) +aed28eb32 - [docs] Add `applies_to` labels for 9.1.0 (#17864) (9 weeks ago) +371d49537 - Remove release note (#17966) (#17972) (9 weeks ago) +64e64625e - Add fips config to jvm.options for observabilitySRE (#17958) (9 weeks ago) +fbb65ee34 - Include the prune filter for observabilitySRE image (#17957) (9 weeks ago) +aabf84ba4 - Convert Ruby Integer and PositiveInteger settings classes to Java (#17460) (9 weeks ago) +5cad4367d - Replace reference to Elasticsearch Services with Elastic Cloud Hosted (#17946) (9 weeks ago) +0b4360183 - Update source/target to Java17 (#17943) (10 weeks ago) +95a0466d8 - Move logstash release information to `.ci` (#17924) (10 weeks ago) +9b9790cd1 - pipeline logging: add cause chain when logging (#16677) (10 weeks ago) +90887906f - Update creating-logstash-pipeline.md (#17926) (#17927) (2 months ago) +3b13df1d6 - cli: add hidden command-line flag for overriding settings (#17582) (2 months ago) +f164c2a5a - Release notes for 9.1.1 (#17914) (#17923) (2 months ago) +1d565c0af - Removal of Ruby bridge classes for Gauge and Counter (#17858) (2 months ago) +6b8d0903c - Log *what* components trigger a grype scan (#17905) (2 months ago) +97306065f - Update versions.yml (#17906) (2 months ago) +f7dd12a59 - Removed agent field from collector, because not used (#17893) (2 months ago) +a1a263bcf - Update release matrix based on 9.1.0 and 8.19.0 releases (#17868) (3 months ago) +bc7b74dd3 - [main] (backport #17875) Make sure FIPS comments belong in the platform-sre Dockerfile (#17898) (3 months ago) +b430599c5 - make sure versions.yml is only read once into gradle.ext (#17865) (3 months ago) +3565db81b - Update monkey patch for clamp to 1.3.3 (#17879) (3 months ago) +591437c94 - Standardization of FIPS Java config with ES (#17839) (3 months ago) +6d7d45ce3 - Fail pipeline when observabilitySRE fails to build (#17872) (3 months ago) +58af5e5bb - Release notes for 9.1.0 (#17822) (#17869) (3 months ago) +882021815 - use centrally maintained version variables (#17857) (3 months ago) +f1823d8fa - bump ci release version (#17844) (3 months ago) +caf91cf17 - Restore cgi pin (#17774) (3 months ago) +198adcf84 - [CI] fix benchmark docker name (#17827) (3 months ago) +f4c195ce6 - [CI] Remove UBI docker acceptance test (#17828) (3 months ago) +9583f0436 - Run tests with jruby 9.4.13.0 (#17798) (3 months ago) +207a69751 - Forwardport observability-sre internal distro support from 8.19 to main (#17785) (3 months ago) +521af3bd4 - Obsolete PQ setting queue.checkpoint.interval (#17759) (3 months ago) +d03f2ce33 - Release notes for 9.0.4 (#17784) (#17818) (3 months ago) +aaff3e91b - update commons-lang3 lib (#17812) (3 months ago) +95866a9fd - Pin jar-dependencies to match jruby 9.4.13.0 (#17787) (3 months ago) +9c5092d24 - update ubi9 ironbank base image to 9.6 (#17802) (3 months ago) +8e73c680f - update dockerfile template based on feedback from dockerhub (#17794) (3 months ago) +ddd519cc8 - pq: reduce read contention when caught up (#17765) (3 months ago) +95624abd9 - Fix allow_superuser comment (#17775) (3 months ago) +ba76f9414 - Bump the github-actions group across 1 directory with 3 updates (#17678) (3 months ago) +e9a3b7edb - Moved dependabot reviewers to codeowners (#17745) (3 months ago) +9721b074c - Update mergify with new 9.1 branch (#17740) (3 months ago) +d474b34e3 - mergify: remove duplicated config and use the default ones for the backport labels (#17742) (4 months ago) +ed4022057 - Implement BufferedTokenizer to return an iterable that can verify size limit for every token emitted (#17229) (4 months ago) +443e52728 - Fix deprecation warning with latest jruby/bundler (#17766) (4 months ago) +05da1bbc8 - Doc: Update getting started file to test publishing (#17762) (4 months ago) +5f54c0cc0 - Standardize image artifact name (#17727) (4 months ago) +b23882479 - Doc: Add clarification about API key format for Logstash (#17688) (4 months ago) +792382765 - Bump to 9.2.0 for main (#17739) (4 months ago) + +=== Logstash Plugin Release Changelogs === +Computed from "git diff v9.1.5..9.2 *.release" +Changed plugin versions: +logstash-filter-elastic_integration: 9.1.1 -> 9.2.0 +logstash-filter-translate: 3.4.3 -> 3.5.0 +logstash-integration-snmp: 4.0.7 -> 4.1.0 +---------- GENERATED CONTENT ENDS HERE ------------ + +### Plugins [logstash-plugin-9.2.0-changes] + +**Elastic_integration Filter - 9.2.0** + +404: Not Found + +**Translate Filter - 3.5.0** + +* Introduce opt-in "yaml_load_strategy => streaming" to stream parse YAML dictionaries [#106](https://github.com/logstash-plugins/logstash-filter-translate/pull/106) + +**Snmp Integration - 4.1.0** + +* Add support for SNMPv3 `context engine ID` and `context name` to the `snmptrap` input [#76](https://github.com/logstash-plugins/logstash-integration-snmp/pull/76) + + ## 9.1.5 [logstash-9.1.5-release-notes] No user-facing changes in Logstash core. From 4783737cddb3e00f7fa21e3a3ad98d370799f6bc Mon Sep 17 00:00:00 2001 From: donoghuc Date: Wed, 15 Oct 2025 16:54:52 -0700 Subject: [PATCH 02/11] identify potential RN candidate commits --- docs/release-notes/index.md | 115 ++++-------------------------------- 1 file changed, 11 insertions(+), 104 deletions(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index 6398f5e160..eb7aca5f1a 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -30,131 +30,38 @@ To check for security updates, go to [Security announcements for the Elastic sta === Logstash Commits between 9.2 and 9.1.5 -Computed with "git log --pretty=format:'%h -%d %s (%cr) <%an>' --abbrev-commit --date=relative v9.1.5..9.2" - -3cf4222b4 - (HEAD -> 9.2, origin/9.2) Adds integration test for the `_health_report` and `_node/plugins` APIs. (#18306) (#18311) (8 hours ago) -d32268785 - Update rack to 3.2.3 (#18300) (31 hours ago) -da313459d - Doc: Update docs for testing for boolean fields (#18271) (#18292) (8 days ago) -bf6cf7b88 - Update rack rubygem dep to 3.2.2 (#18293) (8 days ago) -6b7e1cc01 - Remove redundant testing and circular dependency from docker acceptance testing (#18181) (#18255) (8 days ago) -06a09d01e - Downgrade gradle to coninute testing on windows server 2016 (#18263) (#18278) (8 days ago) -ee9a0ec3a - ci: remove sonarqube (#18273) (#18286) (9 days ago) -98c54cf44 - [9.2] (backport #18247) Release notes for 9.0.8 (#18269) (9 days ago) -f333f73ba - Release notes for 9.1.5 (#18248) (#18266) (9 days ago) -e9df375de - Ensure docs gen inserts at correct place in file (#18250) (#18259) (9 days ago) -587d2560b - Take up 9.2 version of elastic_integration plugin (#18261) (13 days ago) -bcdc991e6 - Add lockfile for new 9.2 branch (#18242) (2 weeks ago) -4934f0839 - (origin/mergify/bp/main/pr-18240) Fix heading (#18237) (2 weeks ago) 0f8c7ba06 - Documentation for batch's event metrics (current and average) (#18017) (2 weeks ago) 425478cd6 - metrics: add gauge with compression goal if enabled (#18230) (2 weeks ago) 1b3b3eeb0 - Pq compression user metrics (#18227) (2 weeks ago) 88b853aff - (origin/mergify/bp/main/pr-18168) metric: improve accuracy of timer metric under contention (#18219) (2 weeks ago) 75eca8475 - metrics: add support for user-defined metrics (#18218) (2 weeks ago) a1522385b - PQ: Add support for event-level compression using ZStandard (ZSTD) (#18121) (2 weeks ago) -0f9b2fe46 - [DOCS] Fix substitution variables (#18224) (3 weeks ago) -46bf75eed - (origin/mergify/bp/main/pr-18223) [Docs] Add hyphen to the pipeline ID restriction description. (#18216) (3 weeks ago) fdeb6a0b9 - Implements current batch event count and byte size metrics (#18160) (3 weeks ago) -2fab5f469 - Replace buildkite jdk version check w/GH action (#17945) (3 weeks ago) -ecaf1fdd4 - Cover the warn password policy usage of ValidatedPassword setting (#18203) (3 weeks ago) -ad200323b - GH action for updating logstash version (#18035) (3 weeks ago) -132f917ea - Update logstash_project_board.yml (#18116) (3 weeks ago) -e6e7f87a8 - Remove uncommented line erroneously committed. (#18204) (3 weeks ago) d29021800 - Measure average batch byte size and event count (#18000) (3 weeks ago) -105eecca7 - (origin/mergify/bp/main/pr-18021) Revert "Moved Ruby Password setting to Java implementation (#18183)" (#18199) (3 weeks ago) -c2851a958 - bk: update junit-annotate to run faster (#18193) (3 weeks ago) -ad42337ee - Bump anchore/scan-action from 6 to 7 in the github-actions group across 1 directory (#18195) (3 weeks ago) f3212484f - PQ settings refactor: propagate builder upward (#18180) (4 weeks ago) -8228fe5dc - Release notes for 9.1.4 (#18159) (#18190) (4 weeks ago) + 6aa265dcd - Moved Ruby Password setting to Java implementation (#18183) (4 weeks ago) cd71a4b18 - Rename Java settings classes to Setting (#18171) (4 weeks ago) -08db1d19f - Adds a required type tag to the open API spec of Logstash APIs. (#18169) (4 weeks ago) -25c2551c8 - Release notes for 9.0.7 (#18158) (#18178) (4 weeks ago) b8bc4d8de - Move Port and PortRange Ruby settings to Java (#17964) (4 weeks ago) -089361de5 - [main] (backport #18028) Release notes for 9.0.6 (#18173) (4 weeks ago) -041e895c5 - Release notes for 9.1.3 (#18029) (#18174) (4 weeks ago) -809ed0029 - Doc: Refine serverless and hosted content (#18166) (4 weeks ago) -feabee287 - (origin/mergify/bp/main/pr-18165) Doc: Add Logstash-to-serverless page (#18034) (4 weeks ago) -556366d87 - Fix race condition with version bump action (#18150) (5 weeks ago) -096f1a381 - Downgrade jruby to 9.4.13.0 (#18136) (5 weeks ago) -1e127fdda - Fix aarch64 acceptance tests (#18135) (5 weeks ago) -e16200a63 - Bump the github-actions group across 1 directory with 2 updates (#18133) (5 weeks ago) -6a51c82e4 - test: explicitly load ascii fixture as ascii, do line-oriented parsing (#18124) (6 weeks ago) -fe4a735c6 - Update JRuby version to 9.4.14.0 (#18080) (6 weeks ago) + ba0787817 - pq: activate stringref extension for more-compact PQ representation (#17849) (6 weeks ago) -ef97a87d8 - Ensure any file object in a tar archive has an mtime (#18113) (6 weeks ago) -d2baf9fbc - github-actions: support backport labels after being merged (#18102) (6 weeks ago) -d9b95bfd8 - Preserve mtime explicitly when creating tar artifacts (#18091) (6 weeks ago) -23d06e665 - Doc: Provide accurate post-geoip lookup example (#18092) (6 weeks ago) -5fb563473 - Fix gauge type correctly in pipelines metrics. (#18082) (6 weeks ago) -f2621b00e - Ensure acceptance test artifacts are being built for correct arch (#18079) (6 weeks ago) -398a66cbe - upgrade golang to 1.25 (#18083) (6 weeks ago) -386869a69 - make gradle files ready for gradle 10 (#17947) (6 weeks ago) -895cfa5b1 - (origin/mergify/bp/main/pr-18077) Add Mergify rule for forwardporting documentation changes (#17874) (7 weeks ago) -e3055d14e - replace placeholder URLs (#18070) (7 weeks ago) -f2e494843 - Build artifacts only for those that match host arch (#17995) (7 weeks ago) -e676b59ac - Start observabilitySRE container builds immediately on DRA (#18016) (7 weeks ago) -91a9527e5 - Stop waiting on ALL steps before staring junit annotation (#18049) (7 weeks ago) -ef41e83f7 - Add stub workflow file to iterate on (#17993) (7 weeks ago) -19d991416 - Split integration tests into more parts (#18036) (7 weeks ago) -5f7a6cdc1 - fixing thin and zero width white space characters in docs (#18019) (8 weeks ago) -34a7c6769 - [DOCS] Fix nullable type linting error in OpenAPI document (#18018) (8 weeks ago) -75abd8a42 - [main] (backport #17781) Harmonize observability sre acceptance (#18001) (8 weeks ago) -bc4645cee - adding x-metatags to openapi spec (#17854) (8 weeks ago) -26027a1d8 - add pipeline id to thread context during init/start (#17944) (8 weeks ago) -50fc6458d - Bump actions/checkout in the github-actions group across 1 directory (#17996) (8 weeks ago) -7d4f476e6 - Update deploying-scaling-logstash.md (#17999) (#18002) (8 weeks ago) -4512e57e6 - Put back the full start requirement which can be managed according to scenario requirement. (#17994) (9 weeks ago) -74c7ec266 - [health-report CI] Print Logstash logs when pipeline faces an issue. (#17991) (9 weeks ago) -22f6b69f4 - Preserve coerce behavior from Ruby impl (#17992) (9 weeks ago) -332355d68 - Start all exhastive tests in parallel (#17978) (9 weeks ago) -aed28eb32 - [docs] Add `applies_to` labels for 9.1.0 (#17864) (9 weeks ago) -371d49537 - Remove release note (#17966) (#17972) (9 weeks ago) -64e64625e - Add fips config to jvm.options for observabilitySRE (#17958) (9 weeks ago) -fbb65ee34 - Include the prune filter for observabilitySRE image (#17957) (9 weeks ago) + + aabf84ba4 - Convert Ruby Integer and PositiveInteger settings classes to Java (#17460) (9 weeks ago) -5cad4367d - Replace reference to Elasticsearch Services with Elastic Cloud Hosted (#17946) (9 weeks ago) 0b4360183 - Update source/target to Java17 (#17943) (10 weeks ago) -95a0466d8 - Move logstash release information to `.ci` (#17924) (10 weeks ago) + 9b9790cd1 - pipeline logging: add cause chain when logging (#16677) (10 weeks ago) -90887906f - Update creating-logstash-pipeline.md (#17926) (#17927) (2 months ago) + 3b13df1d6 - cli: add hidden command-line flag for overriding settings (#17582) (2 months ago) -f164c2a5a - Release notes for 9.1.1 (#17914) (#17923) (2 months ago) 1d565c0af - Removal of Ruby bridge classes for Gauge and Counter (#17858) (2 months ago) -6b8d0903c - Log *what* components trigger a grype scan (#17905) (2 months ago) -97306065f - Update versions.yml (#17906) (2 months ago) -f7dd12a59 - Removed agent field from collector, because not used (#17893) (2 months ago) -a1a263bcf - Update release matrix based on 9.1.0 and 8.19.0 releases (#17868) (3 months ago) -bc7b74dd3 - [main] (backport #17875) Make sure FIPS comments belong in the platform-sre Dockerfile (#17898) (3 months ago) -b430599c5 - make sure versions.yml is only read once into gradle.ext (#17865) (3 months ago) -3565db81b - Update monkey patch for clamp to 1.3.3 (#17879) (3 months ago) -591437c94 - Standardization of FIPS Java config with ES (#17839) (3 months ago) -6d7d45ce3 - Fail pipeline when observabilitySRE fails to build (#17872) (3 months ago) -58af5e5bb - Release notes for 9.1.0 (#17822) (#17869) (3 months ago) -882021815 - use centrally maintained version variables (#17857) (3 months ago) -f1823d8fa - bump ci release version (#17844) (3 months ago) -caf91cf17 - Restore cgi pin (#17774) (3 months ago) -198adcf84 - [CI] fix benchmark docker name (#17827) (3 months ago) -f4c195ce6 - [CI] Remove UBI docker acceptance test (#17828) (3 months ago) -9583f0436 - Run tests with jruby 9.4.13.0 (#17798) (3 months ago) -207a69751 - Forwardport observability-sre internal distro support from 8.19 to main (#17785) (3 months ago) + + 521af3bd4 - Obsolete PQ setting queue.checkpoint.interval (#17759) (3 months ago) -d03f2ce33 - Release notes for 9.0.4 (#17784) (#17818) (3 months ago) -aaff3e91b - update commons-lang3 lib (#17812) (3 months ago) -95866a9fd - Pin jar-dependencies to match jruby 9.4.13.0 (#17787) (3 months ago) -9c5092d24 - update ubi9 ironbank base image to 9.6 (#17802) (3 months ago) -8e73c680f - update dockerfile template based on feedback from dockerhub (#17794) (3 months ago) + ddd519cc8 - pq: reduce read contention when caught up (#17765) (3 months ago) -95624abd9 - Fix allow_superuser comment (#17775) (3 months ago) -ba76f9414 - Bump the github-actions group across 1 directory with 3 updates (#17678) (3 months ago) -e9a3b7edb - Moved dependabot reviewers to codeowners (#17745) (3 months ago) -9721b074c - Update mergify with new 9.1 branch (#17740) (3 months ago) -d474b34e3 - mergify: remove duplicated config and use the default ones for the backport labels (#17742) (4 months ago) + ed4022057 - Implement BufferedTokenizer to return an iterable that can verify size limit for every token emitted (#17229) (4 months ago) -443e52728 - Fix deprecation warning with latest jruby/bundler (#17766) (4 months ago) -05da1bbc8 - Doc: Update getting started file to test publishing (#17762) (4 months ago) -5f54c0cc0 - Standardize image artifact name (#17727) (4 months ago) -b23882479 - Doc: Add clarification about API key format for Logstash (#17688) (4 months ago) -792382765 - Bump to 9.2.0 for main (#17739) (4 months ago) + === Logstash Plugin Release Changelogs === Computed from "git diff v9.1.5..9.2 *.release" From 92a5ed7a8d2b55fa42429ff3798692068cd45a8c Mon Sep 17 00:00:00 2001 From: donoghuc Date: Fri, 17 Oct 2025 12:22:47 -0700 Subject: [PATCH 03/11] Finalize release notes --- docs/release-notes/index.md | 53 +++++++++---------------------------- 1 file changed, 13 insertions(+), 40 deletions(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index eb7aca5f1a..bb768de561 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -25,51 +25,24 @@ To check for security updates, go to [Security announcements for the Elastic sta ### Features and enhancements [logstash-9.2.0-features-enhancements] ----------- GENERATED CONTENT STARTS HERE ------------ -=== Logstash Pull Requests with label v9.2.0 +##### Event compression for persisted queue -=== Logstash Commits between 9.2 and 9.1.5 +* Support for event compression in the persisted queue, controlled by the per-pipeline `queue.compression` setting, which defaults to `none` [#18121](https://github.com/elastic/logstash/pull/18121) +* Added metrics for pq compression [#18227](https://github.com/elastic/logstash/pull/18227) -0f8c7ba06 - Documentation for batch's event metrics (current and average) (#18017) (2 weeks ago) -425478cd6 - metrics: add gauge with compression goal if enabled (#18230) (2 weeks ago) -1b3b3eeb0 - Pq compression user metrics (#18227) (2 weeks ago) -88b853aff - (origin/mergify/bp/main/pr-18168) metric: improve accuracy of timer metric under contention (#18219) (2 weeks ago) -75eca8475 - metrics: add support for user-defined metrics (#18218) (2 weeks ago) -a1522385b - PQ: Add support for event-level compression using ZStandard (ZSTD) (#18121) (2 weeks ago) -fdeb6a0b9 - Implements current batch event count and byte size metrics (#18160) (3 weeks ago) -d29021800 - Measure average batch byte size and event count (#18000) (3 weeks ago) -f3212484f - PQ settings refactor: propagate builder upward (#18180) (4 weeks ago) +##### Batch performance metrics -6aa265dcd - Moved Ruby Password setting to Java implementation (#18183) (4 weeks ago) -cd71a4b18 - Rename Java settings classes to Setting (#18171) (4 weeks ago) -b8bc4d8de - Move Port and PortRange Ruby settings to Java (#17964) (4 weeks ago) +* Implements current batch event count and byte size metrics [#18160](https://github.com/elastic/logstash/pull/18160) +* Implements average batch event count and byte size metrics. The collection of such metric could be disabled, enabled for each batch or done on a sample of the total batches [#18000](https://github.com/elastic/logstash/pull/18000) -ba0787817 - pq: activate stringref extension for more-compact PQ representation (#17849) (6 weeks ago) - -aabf84ba4 - Convert Ruby Integer and PositiveInteger settings classes to Java (#17460) (9 weeks ago) -0b4360183 - Update source/target to Java17 (#17943) (10 weeks ago) - -9b9790cd1 - pipeline logging: add cause chain when logging (#16677) (10 weeks ago) - -3b13df1d6 - cli: add hidden command-line flag for overriding settings (#17582) (2 months ago) -1d565c0af - Removal of Ruby bridge classes for Gauge and Counter (#17858) (2 months ago) - - -521af3bd4 - Obsolete PQ setting queue.checkpoint.interval (#17759) (3 months ago) - -ddd519cc8 - pq: reduce read contention when caught up (#17765) (3 months ago) - -ed4022057 - Implement BufferedTokenizer to return an iterable that can verify size limit for every token emitted (#17229) (4 months ago) - - -=== Logstash Plugin Release Changelogs === -Computed from "git diff v9.1.5..9.2 *.release" -Changed plugin versions: -logstash-filter-elastic_integration: 9.1.1 -> 9.2.0 -logstash-filter-translate: 3.4.3 -> 3.5.0 -logstash-integration-snmp: 4.0.7 -> 4.1.0 ----------- GENERATED CONTENT ENDS HERE ------------ +* Dropped the persistent queue setting queue.checkpoint.interval [#17759](https://github.com/elastic/logstash/pull/17759) +* Reimplements BufferedTokenizer to leverage pure Java classes instead of use JRuby runtime's classes [#17229](https://github.com/elastic/logstash/pull/17229) +* Support for user defined metrics [#18218](https://github.com/elastic/logstash/pull/18218) +* Logging improvement while handling exceptions in the pipeline, ensuring that chained exceptions propagate enough information to be actionable. [#17935](https://github.com/elastic/logstash/pull/17935) +* [Support for using ES|QL queries](https://github.com/logstash-plugins/logstash-filter-elasticsearch/pull/194) in the Elasticsearch filter to add improved flexibility when ingesting data from Elasticsearch is now in Technical Preview. +* Gauge type metrics, such as current and peak connection counts of Elastic Agent, are now available in the `_node/stats` API response when the `vertices=true` parameter is included. These metrics are particularly useful for monitoring {{ls}} plugin activity on the {{ls}} Integration dashboards [#18090](https://github.com/elastic/logstash/pull/18090) +* Improve logstash release artifacts file metadata: mtime is preserved when buiilding tar archives [#18091](https://github.com/elastic/logstash/pull/18091) ### Plugins [logstash-plugin-9.2.0-changes] From 6683a5af5fea25b9d074bf75e6b5df6a926edb6e Mon Sep 17 00:00:00 2001 From: donoghuc Date: Fri, 17 Oct 2025 13:07:32 -0700 Subject: [PATCH 04/11] Restore changes for elastic_integration Add changes based on generated message with https://github.com/elastic/logstash/pull/18315 --- docs/release-notes/index.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index bb768de561..dc3548d973 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -48,7 +48,8 @@ To check for security updates, go to [Security announcements for the Elastic sta **Elastic_integration Filter - 9.2.0** -404: Not Found +* Logging compatability with Elasticsearch 9.2 [#373](https://github.com/elastic/logstash-filter-elastic_integration/pull/373) +* Utilizes Elasticsearch interfaces via Elasticsearch logstash-bridge [#336](https://github.com/elastic/logstash-filter-elastic_integration/pull/336) **Translate Filter - 3.5.0** From 1547209f7f3675550ece40124625ada2b23c6a6e Mon Sep 17 00:00:00 2001 From: Cas Donoghue Date: Fri, 17 Oct 2025 13:10:41 -0700 Subject: [PATCH 05/11] Update docs/release-notes/index.md Co-authored-by: Rob Bavey --- docs/release-notes/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index dc3548d973..8ae37e1d08 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -53,7 +53,7 @@ To check for security updates, go to [Security announcements for the Elastic sta **Translate Filter - 3.5.0** -* Introduce opt-in "yaml_load_strategy => streaming" to stream parse YAML dictionaries [#106](https://github.com/logstash-plugins/logstash-filter-translate/pull/106) +* Introduce opt-in "yaml_load_strategy => streaming" to stream parse YAML dictionaries. This can hugely reduce the memory footprint when working with large YAML dictionaries. [#106](https://github.com/logstash-plugins/logstash-filter-translate/pull/106) **Snmp Integration - 4.1.0** From 8cdeaa7f95253de4dfbac1e3f8a0f09807cff7cb Mon Sep 17 00:00:00 2001 From: Cas Donoghue Date: Fri, 17 Oct 2025 14:23:25 -0700 Subject: [PATCH 06/11] Update docs/release-notes/index.md Co-authored-by: Rye Biesemeyer --- docs/release-notes/index.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index 8ae37e1d08..f91135aea9 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -27,8 +27,9 @@ To check for security updates, go to [Security announcements for the Elastic sta ##### Event compression for persisted queue -* Support for event compression in the persisted queue, controlled by the per-pipeline `queue.compression` setting, which defaults to `none` [#18121](https://github.com/elastic/logstash/pull/18121) -* Added metrics for pq compression [#18227](https://github.com/elastic/logstash/pull/18227) +* Added support for event compression in the persisted queue, allowing you to spend CPU to reduce disk usage and IO, which can improve overall throughput with metered or throttled disks. + This feature is opt-in, and can be configured with the per-pipeline `queue.compression` setting, which accepts `speed`, `balanced`, and `size` for escalating compression ratios, and defaults to `none`. + Per-queue metrics provide visibility into the effective compression ratio and the amount of CPU being spent [#17819](https://github.com/elastic/logstash/issues/17819) ##### Batch performance metrics From d4eafc7bac7b2a29d5371479ab69f2f712ebefbd Mon Sep 17 00:00:00 2001 From: Cas Donoghue Date: Fri, 17 Oct 2025 14:51:17 -0700 Subject: [PATCH 07/11] Update docs/release-notes/index.md --- docs/release-notes/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index f91135aea9..a47e03726b 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -36,7 +36,7 @@ To check for security updates, go to [Security announcements for the Elastic sta * Implements current batch event count and byte size metrics [#18160](https://github.com/elastic/logstash/pull/18160) * Implements average batch event count and byte size metrics. The collection of such metric could be disabled, enabled for each batch or done on a sample of the total batches [#18000](https://github.com/elastic/logstash/pull/18000) - +* Persisted Queue: improved serialization to be more compact by default (note: queues containing these compact events can be processed by Logstash v8.10.0 and later) [#17849](https://github.com/elastic/logstash/pull/17849) * Dropped the persistent queue setting queue.checkpoint.interval [#17759](https://github.com/elastic/logstash/pull/17759) * Reimplements BufferedTokenizer to leverage pure Java classes instead of use JRuby runtime's classes [#17229](https://github.com/elastic/logstash/pull/17229) * Support for user defined metrics [#18218](https://github.com/elastic/logstash/pull/18218) From 6759207d9ad93be219590cb33bbee3a8210f7664 Mon Sep 17 00:00:00 2001 From: donoghuc Date: Mon, 20 Oct 2025 11:39:09 -0700 Subject: [PATCH 08/11] update release highlight for pq improvements --- docs/release-notes/index.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index a47e03726b..9196b56157 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -25,11 +25,19 @@ To check for security updates, go to [Security announcements for the Elastic sta ### Features and enhancements [logstash-9.2.0-features-enhancements] -##### Event compression for persisted queue +##### Logstash 9.2 introduces PQ compression -* Added support for event compression in the persisted queue, allowing you to spend CPU to reduce disk usage and IO, which can improve overall throughput with metered or throttled disks. - This feature is opt-in, and can be configured with the per-pipeline `queue.compression` setting, which accepts `speed`, `balanced`, and `size` for escalating compression ratios, and defaults to `none`. - Per-queue metrics provide visibility into the effective compression ratio and the amount of CPU being spent [#17819](https://github.com/elastic/logstash/issues/17819) +In Logstash 9.2, we’ve added support for compression to the [Persisted Queue](https://www.elastic.co/docs/reference/logstash/persistent-queues), allowing you to spend some CPU in exchange for reduced disk IO. This can help reduce cost and increase throughput in situations where your hardware is rate-limited or metered. + +PQ compression is implemented using the industry-standard highly-efficient ZSTD algorithm, and can be activated at one of three levels: + +* Speed: spend the least amount of CPU to get minimal compression benefit +* Balanced: spend moderate CPU to further reduce size +* Size: enable maximum compression, at significantly higher cost + +The effects of these settings will depend on the shape and size of each pipeline’s events. To help you tune your configuration to meet your own requirements, we have added [queue metrics](https://www.elastic.co/docs/api/doc/logstash/operation/operation-nodestatspipelines) exposing the effective compression ratio and the amount of CPU that is being spent to achieve it. + +PQ Compression has been introduced as an opt-in feature in 9.2 because a PQ that contains one or more compressed events cannot be read by previous versions of Logstash, making the feature a rollback-barrier. We recommend validating your pipelines with Logstash 9.2+ before enabling PQ compression, so that you have the freedom to roll back if you encounter any issues with your pipelines. ##### Batch performance metrics From 402a09bab3617509b3cf056296745af46b3c412c Mon Sep 17 00:00:00 2001 From: donoghuc Date: Tue, 21 Oct 2025 12:25:26 -0700 Subject: [PATCH 09/11] document buffered tokenizer change in behavior --- docs/release-notes/index.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index 9196b56157..d278ea3061 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -53,6 +53,10 @@ PQ Compression has been introduced as an opt-in feature in 9.2 because a PQ that * Gauge type metrics, such as current and peak connection counts of Elastic Agent, are now available in the `_node/stats` API response when the `vertices=true` parameter is included. These metrics are particularly useful for monitoring {{ls}} plugin activity on the {{ls}} Integration dashboards [#18090](https://github.com/elastic/logstash/pull/18090) * Improve logstash release artifacts file metadata: mtime is preserved when buiilding tar archives [#18091](https://github.com/elastic/logstash/pull/18091) +### Known issues + +* The `decode_size_limit_bytes` setting for plugins that use the `json_lines` codec behaves differently in 9.2.0. Specifically when the size limit exceeds the limit without a separator in the data the size will grow beyond the limit. The details for this issue and the details for the future behavior are being tracked in [#18321](https://github.com/elastic/logstash/issues/18321) + ### Plugins [logstash-plugin-9.2.0-changes] **Elastic_integration Filter - 9.2.0** From 8411ffe908b98c5d0aba95050fdc30e98565b873 Mon Sep 17 00:00:00 2001 From: Rob Bavey Date: Tue, 21 Oct 2025 16:34:45 -0400 Subject: [PATCH 10/11] Update index.md with description of batch size metrics Added batch size metrics info --- docs/release-notes/index.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index d278ea3061..ea6b39f7ec 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -39,7 +39,11 @@ The effects of these settings will depend on the shape and size of each pipeline PQ Compression has been introduced as an opt-in feature in 9.2 because a PQ that contains one or more compressed events cannot be read by previous versions of Logstash, making the feature a rollback-barrier. We recommend validating your pipelines with Logstash 9.2+ before enabling PQ compression, so that you have the freedom to roll back if you encounter any issues with your pipelines. -##### Batch performance metrics +##### Batch size metrics + +In Logstash 9.2, we have added metrics to help you track the size of batches processed by Logstash pipelines. + +The [Node API pipelines endpoint](https://www.elastic.co/docs/api/doc/logstash/operation/operation-nodestatspipelines) now shows includes information displaying the showing the average number of events processed per batch, and the average byte size of those batches for each pipeline. This information can be used to help size Logstash instances, and optimize settings for `pipeline.batch.size` for Logstash pipelines based on real observations of data. * Implements current batch event count and byte size metrics [#18160](https://github.com/elastic/logstash/pull/18160) * Implements average batch event count and byte size metrics. The collection of such metric could be disabled, enabled for each batch or done on a sample of the total batches [#18000](https://github.com/elastic/logstash/pull/18000) @@ -471,4 +475,4 @@ Check out the [security advisory](https://discuss.elastic.co/c/announcements/sec **Tcp Output - 7.0.0** -* Remove deprecated SSL settings [#58](https://github.com/logstash-plugins/logstash-output-tcp/pull/58) \ No newline at end of file +* Remove deprecated SSL settings [#58](https://github.com/logstash-plugins/logstash-output-tcp/pull/58) From 42435203201ec4c9ffa06eb340536fa17c2d77cc Mon Sep 17 00:00:00 2001 From: Rob Bavey Date: Wed, 22 Oct 2025 16:42:36 -0400 Subject: [PATCH 11/11] Updated Release Notes to add Highlights and Other Features and Enhancement Sections --- docs/release-notes/index.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/docs/release-notes/index.md b/docs/release-notes/index.md index ea6b39f7ec..3181f0aa53 100644 --- a/docs/release-notes/index.md +++ b/docs/release-notes/index.md @@ -25,6 +25,8 @@ To check for security updates, go to [Security announcements for the Elastic sta ### Features and enhancements [logstash-9.2.0-features-enhancements] +#### Highlights + ##### Logstash 9.2 introduces PQ compression In Logstash 9.2, we’ve added support for compression to the [Persisted Queue](https://www.elastic.co/docs/reference/logstash/persistent-queues), allowing you to spend some CPU in exchange for reduced disk IO. This can help reduce cost and increase throughput in situations where your hardware is rate-limited or metered. @@ -39,19 +41,26 @@ The effects of these settings will depend on the shape and size of each pipeline PQ Compression has been introduced as an opt-in feature in 9.2 because a PQ that contains one or more compressed events cannot be read by previous versions of Logstash, making the feature a rollback-barrier. We recommend validating your pipelines with Logstash 9.2+ before enabling PQ compression, so that you have the freedom to roll back if you encounter any issues with your pipelines. +Related: +* Persisted Queue: improved serialization to be more compact by default (note: queues containing these compact events can be processed by Logstash v8.10.0 and later) [#17849](https://github.com/elastic/logstash/pull/17849) +* Support for user defined metrics [#18218](https://github.com/elastic/logstash/pull/18218) +* PQ: Add support for event-level compression using ZStandard (ZSTD) [#18121](https://github.com/elastic/logstash/pull/18121) + ##### Batch size metrics In Logstash 9.2, we have added metrics to help you track the size of batches processed by Logstash pipelines. The [Node API pipelines endpoint](https://www.elastic.co/docs/api/doc/logstash/operation/operation-nodestatspipelines) now shows includes information displaying the showing the average number of events processed per batch, and the average byte size of those batches for each pipeline. This information can be used to help size Logstash instances, and optimize settings for `pipeline.batch.size` for Logstash pipelines based on real observations of data. +Related: * Implements current batch event count and byte size metrics [#18160](https://github.com/elastic/logstash/pull/18160) * Implements average batch event count and byte size metrics. The collection of such metric could be disabled, enabled for each batch or done on a sample of the total batches [#18000](https://github.com/elastic/logstash/pull/18000) -* Persisted Queue: improved serialization to be more compact by default (note: queues containing these compact events can be processed by Logstash v8.10.0 and later) [#17849](https://github.com/elastic/logstash/pull/17849) + +#### Other Features and Enhancements + * Dropped the persistent queue setting queue.checkpoint.interval [#17759](https://github.com/elastic/logstash/pull/17759) * Reimplements BufferedTokenizer to leverage pure Java classes instead of use JRuby runtime's classes [#17229](https://github.com/elastic/logstash/pull/17229) -* Support for user defined metrics [#18218](https://github.com/elastic/logstash/pull/18218) * Logging improvement while handling exceptions in the pipeline, ensuring that chained exceptions propagate enough information to be actionable. [#17935](https://github.com/elastic/logstash/pull/17935) * [Support for using ES|QL queries](https://github.com/logstash-plugins/logstash-filter-elasticsearch/pull/194) in the Elasticsearch filter to add improved flexibility when ingesting data from Elasticsearch is now in Technical Preview. * Gauge type metrics, such as current and peak connection counts of Elastic Agent, are now available in the `_node/stats` API response when the `vertices=true` parameter is included. These metrics are particularly useful for monitoring {{ls}} plugin activity on the {{ls}} Integration dashboards [#18090](https://github.com/elastic/logstash/pull/18090)