Java 21#52
Merged
Merged
Conversation
- Upgrade Hive metastore 2.3.9 -> 4.0.1 and Apiary Extensions 6.0.2 -> 8.1.15 - Add hadoop-client-runtime:3.3.6 to provide shaded WoodStox XML parser classes - Upgrade Spring Boot 2.7.10 -> 3.2.12 (Spring Framework 6.1) - Migrate javax.annotation.PreDestroy -> jakarta.annotation.PreDestroy - Exclude DataSourceAutoConfiguration to suppress Hive transitive JDBC drivers - Upgrade Guava 27.1-jre -> 33.4.0-jre, MSK IAM Auth 1.1.9 -> 2.2.0 - Drop explicit Dropwizard/JUnit/Mockito/AssertJ/Logback/Log4j version pins (Boot BOM manages) - Set jdk.version=21 and maven.compiler.release=21 - Upgrade Spotless 2.4.1 -> 2.43.0 (Java 21 compatible, google-java-format 1.19.2) - Upgrade JaCoCo 0.8.6 -> 0.8.12 and Surefire 3.0.0-M5 -> 3.2.5 - Add .mvn/jvm.config with --add-exports for google-java-format on JDK 21 - Add Surefire --add-opens argLine for Hadoop/Hive/Mockito runtime reflection - Update Docker base image openjdk:8-jdk -> eclipse-temurin:21-jre - Add Jib container --add-opens JVM flags for Hadoop/Hive runtime - Update GitHub Actions workflows: Java 21, temurin, checkout@v4, setup-java@v4 - Fix Hive 4.x API changes: HMSHandler constructor, event constructors, JavaUtils package - Migrate commons-lang -> commons-lang3 (StringUtils) - Replace deprecated org.awaitility.Duration with java.time.Duration Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
apiary-hive-events:8.1.15 was compiled against Hive 3.x and is incompatible with hive-metastore:4.0.1 at runtime. Three layers of incompatibility are resolved via classpath shims in drone-fly-app: 1. HiveMetaStore.java: re-introduces HiveMetaStore$HMSHandler as an inner class (removed in Hive 4.x, now a standalone HMSHandler). Required by JsonMetaStoreEventSerDe$HeplerApiaryListenerEvent which references this type in its static initializer. 2. CreateTableEvent.java: adds back the Hive 3.x 3-argument constructor alongside the Hive 4.x 4-argument one. Prevents NoSuchMethodError in JsonMetaStoreEventSerDe at application startup. 3. Column stats shims (BooleanColumnStatsData, LongColumnStatsData, DoubleColumnStatsData, StringColumnStatsData, BinaryColumnStatsData, DecimalColumnStatsData, DateColumnStatsData, TimestampColumnStatsData, Decimal): Hive 4.x Table now has a colStats field, causing Jackson to introspect all column stats types. Thrift-generated Hive 4.x classes expose both setBitVectors(byte[]) and setBitVectors(ByteBuffer), triggering Jackson InvalidDefinitionException. Shims expose only the byte[] setter, resolving the conflict. Also updates DroneFlyIntegrationTestUtils.buildPartition() to explicitly set writeId=-1 and isStatsCompliant=false, matching the values produced by JSON round-trip through Kafka (new Hive 4.x Partition fields). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents all changes from the Java 8 → Java 21 migration: dependency upgrades, build/tooling changes, source code changes, and all three layers of Hive 3.x/4.x compatibility shims added to fix integration tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update GitHub Actions workflows to use corretto distribution and change the Docker base image from eclipse-temurin:21-jre to amazoncorretto:21. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restore original code style for all files that were reformatted by google-java-format but had no functional changes. Functional changes (javax->jakarta, Hive 4.x API constructors, JavaUtils package rename, commons-lang3, java.time.Duration) are preserved in their original code style. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Restore 26 Java source files to their pre-migration formatting (original indentation and import ordering preserved) - Keep only the functional changes required for Java 21 / Hive 4.x: - DroneFly: exclude DataSourceAutoConfiguration (Spring Boot 3.x) - DroneFlyRunner: javax.annotation -> jakarta.annotation - ListenerCatalog: hive.common.JavaUtils -> hive.metastore.utils.JavaUtils - HMSHandlerFactory: HiveMetaStore.HMSHandler -> HMSHandler (top-level); remove 'false' init arg - ListenerCatalogFactory: commons.lang -> commons.lang3 StringUtils - HiveEventConverterService: Hive 4.x constructor signatures (AlterPartition, CreateTable, AlterTable, DropTable, InsertEvent) - Test files: awaitility Duration deprecation fix; HMSHandler import fix; Hive 4.x constructors - DroneFlyIntegrationTestUtils: setWriteId(-1) + setIsStatsCompliant(false) for Hive 4.x - Remove Spotless plugin from pom.xml (team does not use it) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Spotless 2.4.1 (inherited from eg-oss-parent) uses google-java-format which requires access to internal JDK compiler APIs not exported in Java 21. Override the skip property to disable it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SpotBugs 4.0.4 (from eg-oss-parent) uses an ASM version that cannot read Java 21 class files (major version 65). Version 4.8.0+ includes an updated ASM that supports Java 21 bytecode. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…JavaUtils With Java 21 / Spring Boot 3.x, JavaUtils.getClassLoader() returns the JVM AppClassLoader which does not include jars from loader.path=lib/. Switching to Thread.currentThread().getContextClassLoader() ensures Spring Boot's LaunchedURLClassLoader is used, which correctly includes external listener jars placed in the lib/ directory at runtime. Fixes ClassNotFoundException for listeners like ApiaryGlueSync loaded from the lib/ directory in containerised deployments. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update base image tag from amazoncorretto:21 (AL2) to amazoncorretto:21-al2023 for Amazon Linux 2023 support. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds Commit 3 section to JAVA21_MIGRATION.md explaining why external listener JARs (added by child Dockerfiles) became invisible after the Java 21 / Hive 4.x migration: Hive 4.x JavaUtils changed getClassLoader() to return AppClassLoader instead of Spring Boot's LaunchedURLClassLoader, and Jib bakes an explicit classpath at image-build time that omits JARs added by child Dockerfile RUN steps. Fix is to override ENTRYPOINT with a wildcard /app/libs/* classpath in the child Dockerfile. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Description
🔗 Related Issues