During the rename from spark-rapids-jni to cudf-spark-jni, both JARs can enter the same classpath because Maven treats the artifact IDs as different dependencies. If the metadata file is also renamed, a duplicate check that scans only cudf-spark-jni-version-info.properties cannot see the legacy JAR. Spark may then load either implementation based on classpath order.
This is not 100% reproducible in a normal Maven build or ordinary unit/integration tests, because those usually contain only one JNI artifact. It occurs when both are explicitly included, introduced through direct and transitive dependencies, retained in a staging directory, or supplied through spark-submit.
Expected: duplicate-JAR detection must see both artifacts. Keep spark-rapids-jni-version-info.properties as the common metadata name during the transition, or support both filenames.
Related: PR review comment, migration issue NVIDIA/cudf-spark#15882.
During the rename from
spark-rapids-jnitocudf-spark-jni, both JARs can enter the same classpath because Maven treats the artifact IDs as different dependencies. If the metadata file is also renamed, a duplicate check that scans onlycudf-spark-jni-version-info.propertiescannot see the legacy JAR. Spark may then load either implementation based on classpath order.This is not 100% reproducible in a normal Maven build or ordinary unit/integration tests, because those usually contain only one JNI artifact. It occurs when both are explicitly included, introduced through direct and transitive dependencies, retained in a staging directory, or supplied through
spark-submit.Expected: duplicate-JAR detection must see both artifacts. Keep
spark-rapids-jni-version-info.propertiesas the common metadata name during the transition, or support both filenames.Related: PR review comment, migration issue NVIDIA/cudf-spark#15882.