[MARTIFACT-61] Multithreaded verification fails

[jira-importer]

Mateusz Gajewski opened MARTIFACT-61 and commented

I'm adding support for reproducible builds to Trino (https://github.com/trinodb/trino).) and I think to have found an error in the latest artifact plugin version.

 

Single threaded verification works fine:

[INFO] trino-product-tests-launcher ....................... SUCCESS [ 10.050 s]
[INFO] trino-server-dev ................................... SUCCESS [  1.795 s]
[INFO] trino-test-jdbc-compatibility-old-driver ........... SUCCESS [  0.778 s]
[INFO] trino-test-jdbc-compatibility-old-server ........... SUCCESS [ 33.942 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  11:54 min
[INFO] Finished at: 2024-04-10T15:43:51+02:00
[INFO] ------------------------------------------------------------------------
(base) ➜  trino git:(master) ✗ ./mvnw clean verify artifact:compare -nsu  -DskipTests -pl '!:trino-docs' -Dtest=void -e |& tee build.log 

while multithreaded invocation fails with:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-artifact-plugin:3.5.1:compare (default-cli) on project trino-test-jdbc-compatibility-old-server: Artifact io.trino:trino-faulttolerant-tests:jar:445-SNAPSHOT points to a directory: /Users/mateuszgajewski/Projects/src/github.com/trinodb/trino/testing/trino-faulttolerant-tests/target/classes. Packaging should be 'pom'? -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-artifact-plugin:3.5.1:compare (default-cli) on project trino-test-jdbc-compatibility-old-server: Artifact io.trino:trino-faulttolerant-tests:jar:445-SNAPSHOT points to a directory: /Users/mateuszgajewski/Projects/src/github.com/trinodb/trino/testing/trino-faulttolerant-tests/target/classes. Packaging should be 'pom'?
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:333)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1570)
Caused by: org.apache.maven.plugin.MojoExecutionException: Artifact io.trino:trino-faulttolerant-tests:jar:445-SNAPSHOT points to a directory: /Users/mateuszgajewski/Projects/src/github.com/trinodb/trino/testing/trino-faulttolerant-tests/target/classes. Packaging should be 'pom'?
    at org.apache.maven.plugins.artifact.buildinfo.BuildInfoWriter.printArtifact (BuildInfoWriter.java:240)
    at org.apache.maven.plugins.artifact.buildinfo.BuildInfoWriter.printArtifacts (BuildInfoWriter.java:205)
    at org.apache.maven.plugins.artifact.buildinfo.AbstractBuildinfoMojo.generateBuildinfo (AbstractBuildinfoMojo.java:256)
    at org.apache.maven.plugins.artifact.buildinfo.AbstractBuildinfoMojo.execute (AbstractBuildinfoMojo.java:156)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:328)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:316)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:174)
    at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:75)
    at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:162)
    at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:159)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:193)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call (MultiThreadedBuilder.java:180)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:572)
    at java.util.concurrent.FutureTask.run (FutureTask.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1144)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:642)
    at java.lang.Thread.run (Thread.java:1570) 

 

./mvnw clean verify artifact:compare -nsu -DskipTests -T 2C -pl '!:trino-docs' -Dtest=void -e |& tee build.log

---

Affects: 3.5.1

Issue Links:

• MARTIFACT-65 artifact:compare marked threadsafe, but it is not
  ("is a parent of")

[jira-importer]

Hervé Boutemy commented

I confirm I never tried to rebuild and compare in parallel build, as rebuild is sufficiently exceptional that performance enhancement is not a first objective

oh, I see that the goal documents the goal as thread-safe: uh, uh, I'm not surprised that this is optimistic :)

I'll create a bug issue to just remove that "threadsafe" mark as a quick improvement to show reality

but that would be a nice feature for future: if you see how to make the goal threadsafe, or even see some aspects to fix, don't hesitate to share

[jira-importer]

Hervé Boutemy commented

BTW, I don't see any RB-enabled pom.xml in trino: is there a PR or a branch somewhere?

[jira-importer]

Mateusz Gajewski commented

Hervé Boutemy actually it was merged yesterday - RB is provided by the parent POM configuration (named airbase): airlift/airbase@529508c

The current master is reproducible - I'm going to add a job to a CI that checks the build (but without using artifact:compare as it will be slower): trinodb/trino#21733