Releases: GoogleCloudPlatform/DataflowTemplates
Releases · GoogleCloudPlatform/DataflowTemplates
Dataflow Templates 2025-12-30-00_RC00
build(deps): bump org.codehaus.mojo:extra-enforcer-rules (#3165) Bumps [org.codehaus.mojo:extra-enforcer-rules](https://github.com/mojohaus/extra-enforcer-rules) from 1.10.0 to 1.11.0. - [Release notes](https://github.com/mojohaus/extra-enforcer-rules/releases) - [Commits](https://github.com/mojohaus/extra-enforcer-rules/compare/1.10.0...1.11.0) --- updated-dependencies: - dependency-name: org.codehaus.mojo:extra-enforcer-rules dependency-version: 1.11.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
SKIPPED - Dataflow Templates 2025-12-23-00_RC00
Increasing maxConnection on MySqlResourceManager For AllDataTypesIT (…
Dataflow Templates 2025-12-16-00_RC00
What's Changed
Improvements
- Add support for Postgres to Iceberg Blueprint by @tarun-google in #3019
- Fix Auto Generate script to consider option files by @tarun-google in #3088
- Update the helpText in Kafka Templates by @Dippatel98 in #3102
- chore: Switch to AvroCoder for Reverse replication template by @manitgupta in #3073
- Adding machine type validator for forward migration by @shreyakhajanchi in #3092
Bug fixes
- shuffle around mock version to centralized location to fix dependabot PR by @derrickaw in #3084
- [datastream-to-spanner] Update description of shardingContextFilePath in README by @aasthabharill in #3090
- Fix description of Pub/Sub subscription parameter by @VeronicaWasson in #2135
- Update mvn command for staging template in README to install dependent modules by @aasthabharill in #3089
- main and shadow table as a Side Input in spanner-to-sourcedb by @sm745052 in #2950
- [datastream-to-spanner] Unable to convert field timestamp to long by @aasthabharill in #3035
- fixed priority handling in assign shard id by @bharadwaj-aditya in #3110
Package Upgrades
- chore(deps): update Python dependencies by @github-actions[bot] in #3076
New Contributors
- @VeronicaWasson made their first contribution in #2135
Full Changelog: 2025-12-09-00_RC00...2025-12-16-00_RC00
Dataflow Templates 2025-12-09-00_RC00
What's Changed
Improvements
- [SpannerToSourceDb] Integration test for CDC processing algorithm for Reverse migration template by @darshan-sj in #3027
- [SourceDbToSpanner] Fixing severe error file overwrite issue in Bulk migration template by @darshan-sj in #3039
- upgrade go to 1.24 to fix cicd issue by @derrickaw in #3048
- Add auto yaml to java generation by @derrickaw in #2984
- remove old todos for 2.68, gax, httpclient, and json-smart that are not used by @derrickaw in #2887
- feat: Denylist small worker sizes in reverse replication by @manitgupta in #3044
- Add support for GCP SchemaRegistry in Kafka Templates by @Dippatel98 in #3066
Bug fixes
- When Spanner has NULL value for SQL PK column, the error is thrown by change event by @aasthabharill in #3034
Package Upgrades
New Contributors
- @NicolasRicheYopp made their first contribution in #3029
- @apanich made their first contribution in #3068
- @gpalanisamyYopp made their first contribution in #3059
Full Changelog: 2025-12-02-00_RC00...2025-12-09-00_RC00
Dataflow Templates 2025-12-02-00_RC00
What's Changed
Improvements
- Fix JSON/JSONB datatypes in postgres-to-sql template by @yair-harel in #2960
- Add IcebergResourceManager by @tarun-google in #2985
- Add support for additional fields in terraform templates for Reverse and Live migration by @aasthabharill in #3005
- [DatastreamToSpanner] Integration test for CDC processing algorithm for Live migration template by @darshan-sj in #3018
Bug fixes
- Remove preemptive cleanup by @rahulkumar0630 in #2981
- chore: Added latency metrics to Write to source DoFn by @pratickchokhani in #3009
- [spanner-to-sourcedb] Custom Transformation fails when SQL PK col is dropped in Spanner by @aasthabharill in #3003
- [spanner-to-sourcedb] DELETEs: Error fetching shard id column by @aasthabharill in #3024
- Fix dependency conflict on beam-sdks-java-core in Datastream to Firestore by @baeminbo in #3026
New Contributors
- @tarun-google made their first contribution in #2985
- @nmemond made their first contribution in #2992
- @aasthabharill made their first contribution in #3005
- @pratickchokhani made their first contribution in #3009
- @baeminbo made their first contribution in #3026
Full Changelog: 2025-11-11-00_RC00...2025-12-02-00_RC00
Dataflow Templates 2025-11-11-00_RC00
What's Changed
Package Upgrades
- chore(deps): update Python dependencies by @github-actions[bot] in #2979
Full Changelog: 2025-11-06-00_RC00...2025-11-11-00_RC00
Dataflow Templates 2025-11-06-00_RC00
What's Changed
Improvements
- Delete v2/datastream-to-postgres directory by @dhercher in #2959
- [SpannerToSourceDb] Improving error classification in Reverse replication template by @darshan-sj in #2964
- Update to beam 2.69 by @Abacn in #2966
Bug fixes
- fixing retry count in reverse replication by @shreyakhajanchi in #2949
Package Upgrades
- chore(deps): update Python dependencies by @github-actions[bot] in #2956
Full Changelog: 2025-10-28-01_RC00...2025-11-06-00_RC00
Dataflow Templates 2025-11-05-00_RC00
fix(build): update surefire report processing to handle time format (…
Dataflow Templates 2025-10-28-01_RC00
What's Changed
Improvements
- fix(kafka): handle domain-scoped project IDs in bootstrap server URL by @liferoad in #2908
- Yaml: Add yaml syntax UT by @derrickaw in #2872
- [SpannerToSourceDb] Fixing CDC events processing logic for Reverse replication template by @darshan-sj in #2919
- Apply GCP Application Default authentication when using a Managed Service for Apache Kafka cluster. by @fozzie15 in #2925
Bug fixes
- Remove special handling for only PK column tables by @manitgupta in #2929
- [yaml] - move syntax test verification to validate section by @derrickaw in #2936
Package Upgrades
- chore(deps): update Python dependencies by @github-actions[bot] in #2899
- build: update jetty dependencies to 12.0.12 to fix CVE-2024-6763 by @stankiewicz in #2921
- build: update org.apache.commons:commons-configuration to resolve CVE-2024-29131 by @stankiewicz in #2922
New Contributors
Full Changelog: 2025-10-21-00_RC00...2025-10-28-01_RC00
Dataflow Templates 2025-10-28-00_RC00
bump org.apache.commons:commons-configuration to resolve CVE-2024-291…