diff --git a/JAVA21_MIGRATION.md b/JAVA21_MIGRATION.md index 4d81409..d896e40 100644 --- a/JAVA21_MIGRATION.md +++ b/JAVA21_MIGRATION.md @@ -27,7 +27,7 @@ and fix the integration tests that broke as a result of the Hive 4.x upgrade. | Spotless Maven plugin | 2.4.1 | 2.43.0 (google-java-format 1.19.2, Java 21 compatible) | | JaCoCo | 0.8.6 | 0.8.12 | | Surefire | 3.0.0-M5 | 3.2.5 | -| Docker base image | `openjdk:8-jdk` | `amazoncorretto:21` | +| Docker base image | `openjdk:8-jdk` | `amazoncorretto:21-al2023` | Dropped explicit version pins for Logback, Log4j, JUnit, Mockito, AssertJ, and Dropwizard — these are now managed by the Spring Boot BOM. @@ -156,6 +156,88 @@ object's `isSet` flags match the deserialized object. --- +## Commit 3 — Fix `ClassNotFoundException` for external listener JARs in Jib-built images + +### Problem + +Child images that extend `drone-fly-app` (e.g. `egdp-docker-glue-sync-listener`) download a +listener JAR into `/app/libs` at Dockerfile build time: + +```dockerfile +FROM expediagroup/drone-fly-app:1.0.9-SNAPSHOT +RUN cd /app/libs && curl ... apiary-gluesync-listener-8.1.13-all.jar +``` + +After upgrading to Java 21 / Hive 4.x, the container failed on startup with: + +``` +Caused by: java.lang.ClassNotFoundException: + com.expediagroup.apiary.extensions.gluesync.listener.ApiaryGlueSync + at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(...) +``` + +### Root cause — two compounding changes + +**Change 1 — Hive 2.x → 4.x changed `JavaUtils.getClassLoader()` behavior** + +| | Hive 2.x | Hive 4.x | +|---|---|---| +| Package | `org.apache.hadoop.hive.common.JavaUtils` | `org.apache.hadoop.hive.metastore.utils.JavaUtils` | +| `getClassLoader()` returns | `Thread.currentThread().getContextClassLoader()` | JVM `AppClassLoader` | + +In a Spring Boot fat-jar launched with `PropertiesLauncher` and `loader.path=lib/`, the thread +context classloader is Spring Boot's `LaunchedURLClassLoader`, which can find JARs placed in +`loader.path`. Hive 2.x `JavaUtils` used this classloader, so external listener JARs were +visible. + +Hive 4.x `JavaUtils` returns the plain JVM `AppClassLoader`, which only sees the `-cp` +argument set at JVM startup — not dynamically placed JARs. + +**Change 2 — Jib bakes the classpath at image-build time** + +Jib generates an `ENTRYPOINT` with an **explicit list** of dependency JARs in the `-cp` +argument (determined at `mvn package` time). JARs downloaded by a child Dockerfile's `RUN` +step land on the filesystem in `/app/libs` but are **never added** to that hardcoded classpath. + +Before the migration this was masked: the Hive 2.x `JavaUtils` used `LaunchedURLClassLoader` +(via PropertiesLauncher), which resolved listener JARs through `loader.path` independently of +the Jib classpath. After the migration both defences were removed simultaneously. + +| | Before migration | After migration | +|---|---|---| +| `JavaUtils.getClassLoader()` returns | `LaunchedURLClassLoader` (sees `loader.path`) | `AppClassLoader` (sees only `-cp`) | +| Listener JAR on classpath? | ✅ Yes (via `loader.path`) | ❌ No (not in Jib `-cp`) | + +### Fix — override `ENTRYPOINT` in child Dockerfile with a wildcard classpath + +`egdp-docker-glue-sync-listener/Dockerfile` was updated to override the Jib-baked entrypoint +with one that uses `/app/libs/*`. The JVM expands the wildcard at **startup time**, picking up +every JAR present in `/app/libs/` — including those downloaded by the `RUN curl` step: + +```dockerfile +ENTRYPOINT ["java", + "--add-opens=java.base/java.lang=ALL-UNNAMED", + "--add-opens=java.base/java.lang.reflect=ALL-UNNAMED", + "--add-opens=java.base/java.io=ALL-UNNAMED", + "--add-opens=java.base/java.net=ALL-UNNAMED", + "--add-opens=java.base/java.nio=ALL-UNNAMED", + "--add-opens=java.base/java.util=ALL-UNNAMED", + "--add-opens=java.base/java.util.concurrent=ALL-UNNAMED", + "--add-opens=java.base/sun.nio.ch=ALL-UNNAMED", + "--add-opens=java.base/java.security=ALL-UNNAMED", + "-cp", "/app/resources:/app/classes:/app/libs/*", + "com.expediagroup.dataplatform.dronefly.app.DroneFly"] +``` + +The `--add-opens` flags match those in the Jib `` configuration in the parent +`pom.xml` so runtime Hadoop/Hive reflection behaviour is preserved. + +> **Note for other child images:** Any Dockerfile that extends `drone-fly-app` and adds JARs +> to `/app/libs` must include this `ENTRYPOINT` override to ensure those JARs are on the +> classpath. + +--- + ## Files changed (summary) ``` diff --git a/drone-fly-app/pom.xml b/drone-fly-app/pom.xml index a91311b..2acac08 100644 --- a/drone-fly-app/pom.xml +++ b/drone-fly-app/pom.xml @@ -15,10 +15,15 @@ 1.11.532 0.2.5 4.0.1 + 3.4.2 8008 + + org.springframework.boot + spring-boot-starter-web + com.expediagroup drone-fly-core @@ -88,6 +93,10 @@ jdk.tools jdk.tools + + org.apache.logging.log4j + log4j-1.2-api + @@ -131,6 +140,14 @@ tomcat jasper-runtime + + org.apache.hadoop + hadoop-common + + + org.apache.logging.log4j + log4j-1.2-api + @@ -154,12 +171,12 @@ org.apache.hadoop hadoop-client-runtime - 3.3.6 + ${hadoop.version} org.apache.hadoop hadoop-mapreduce-client-core - 3.3.6 + ${hadoop.version} test @@ -177,6 +194,21 @@ awaitility test + + org.apache.tomcat.embed + tomcat-embed-core + ${tomcat.embeded.version} + + + org.apache.tomcat.embed + tomcat-embed-el + ${tomcat.embeded.version} + + + org.apache.tomcat.embed + tomcat-embed-websocket + ${tomcat.embeded.version} + diff --git a/drone-fly-app/src/main/java/com/expediagroup/dataplatform/dronefly/app/service/ListenerCatalog.java b/drone-fly-app/src/main/java/com/expediagroup/dataplatform/dronefly/app/service/ListenerCatalog.java index 692c565..6fbe25d 100644 --- a/drone-fly-app/src/main/java/com/expediagroup/dataplatform/dronefly/app/service/ListenerCatalog.java +++ b/drone-fly-app/src/main/java/com/expediagroup/dataplatform/dronefly/app/service/ListenerCatalog.java @@ -26,7 +26,6 @@ import java.util.List; import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hive.metastore.utils.JavaUtils; import org.apache.hadoop.hive.conf.HiveConf; import org.apache.hadoop.hive.metastore.MetaStoreEventListener; @@ -62,7 +61,7 @@ private List getMetaStoreListeners(Class clazz, HiveConf conf, String for (String listenerImpl : listenerImpls) { try { T listener = (T) Class - .forName(listenerImpl.trim(), true, JavaUtils.getClassLoader()) + .forName(listenerImpl.trim(), true, Thread.currentThread().getContextClassLoader()) .getConstructor(Configuration.class) .newInstance(conf); listeners.add(listener); diff --git a/drone-fly-core/pom.xml b/drone-fly-core/pom.xml index 50aefcc..d5e51d9 100644 --- a/drone-fly-core/pom.xml +++ b/drone-fly-core/pom.xml @@ -26,12 +26,18 @@ org.springframework.boot spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-logging - - + + + org.apache.tomcat.embed + tomcat-embed-core + + + org.apache.tomcat.embed + tomcat-embed-el + + + org.apache.tomcat.embed + tomcat-embed-websocket org.springframework diff --git a/drone-fly-core/src/test/java/com/expediagroup/dataplatform/dronefly/core/DroneFlyCoreTest.java b/drone-fly-core/src/test/java/com/expediagroup/dataplatform/dronefly/core/DroneFlyCoreTest.java index a0ce34e..7355572 100644 --- a/drone-fly-core/src/test/java/com/expediagroup/dataplatform/dronefly/core/DroneFlyCoreTest.java +++ b/drone-fly-core/src/test/java/com/expediagroup/dataplatform/dronefly/core/DroneFlyCoreTest.java @@ -1,5 +1,5 @@ /** - * Copyright (C) 2020 Expedia, Inc. + * Copyright (C) 2020-2026 Expedia, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/drone-fly-integration-tests/src/test/java/com/expediagroup/dataplatform/dronefly/core/integration/DummyListener.java b/drone-fly-integration-tests/src/test/java/com/expediagroup/dataplatform/dronefly/core/integration/DummyListener.java index 1ff92bd..9dfaa06 100644 --- a/drone-fly-integration-tests/src/test/java/com/expediagroup/dataplatform/dronefly/core/integration/DummyListener.java +++ b/drone-fly-integration-tests/src/test/java/com/expediagroup/dataplatform/dronefly/core/integration/DummyListener.java @@ -1,5 +1,5 @@ /** - * Copyright (C) 2020 Expedia, Inc. + * Copyright (C) 2020-2026 Expedia, Inc. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. diff --git a/pom.xml b/pom.xml index 9519d40..d32770b 100644 --- a/pom.xml +++ b/pom.xml @@ -30,15 +30,18 @@ 21 + true 0.8.12 3.2.5 3.13.0 + 4.9.8.2 3.2.12 3.2.4 3.4.3 + 10.1.50 amazoncorretto - 21 + 21-al2023 ${docker.from.image}:${docker.from.tag} expediagroup ${project.artifactId} @@ -66,6 +69,29 @@ pom import + + org.springframework.boot + spring-boot-starter-web + ${springframework.boot.version} + + + org.springframework.boot + spring-boot-starter-logging + + + org.apache.tomcat.embed + tomcat-embed-core + + + org.apache.tomcat.embed + tomcat-embed-el + + + org.apache.tomcat.embed + tomcat-embed-websocket + + +