[stevens] Build from Megamodels

[grammarware]

Submitted by @PerditaStevens to https://github.com/modelsconf2018/artifact-evaluation/tree/master/stevens

Paper: http://homepages.inf.ed.ac.uk/perdita/MegamodelBuild/modelspaper.pdf

[manuelleduc]

Dear @PerditaStevens ,

Importing the megamodelbuild project in eclipse was straightforward thanks to maven, and I also easily executed the unit tests to validate that everything was working well.

However, I tried to execute the bxexample.sh script out of curiosity and obtained the following error.

./bxexample.sh 
[INFO] Scanning for projects...
Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml
Downloaded from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml (605 B at 525 B/s)
Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml
Downloaded from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml (607 B at 1.1 kB/s)
Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/build/pluto/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml
Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/build/pluto/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml
Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/build/pluto/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml
Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/build/pluto/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Building megamodelbuild 0.1
[INFO] ------------------------------------------------------------------------
Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/org/sugarj/common/1.7.0-SNAPSHOT/maven-metadata.xml
Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/common/1.7.0-SNAPSHOT/maven-metadata.xml
Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/org/sugarj/common/1.7.0-SNAPSHOT/maven-metadata.xml
Downloaded from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/common/1.7.0-SNAPSHOT/maven-metadata.xml (766 B at 2.2 kB/s)
Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/org/sugarj/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml
Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml
Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/org/sugarj/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml
Downloaded from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml (604 B at 1.6 kB/s)
Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/org/sugarj/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml
Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml
Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/org/sugarj/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml
Downloaded from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml (606 B at 1.8 kB/s)
Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/pluto/1.9.0-SNAPSHOT/maven-metadata.xml
Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/build/pluto/pluto/1.9.0-SNAPSHOT/maven-metadata.xml
Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/build/pluto/pluto/1.9.0-SNAPSHOT/maven-metadata.xml
Downloaded from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/pluto/1.9.0-SNAPSHOT/maven-metadata.xml (978 B at 1.7 kB/s)
Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/com/cedarsoftware/java-util-pluto-fixes/1.19.4-SNAPSHOT/maven-metadata.xml
Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/com/cedarsoftware/java-util-pluto-fixes/1.19.4-SNAPSHOT/maven-metadata.xml
Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/com/cedarsoftware/java-util-pluto-fixes/1.19.4-SNAPSHOT/maven-metadata.xml
Downloaded from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/com/cedarsoftware/java-util-pluto-fixes/1.19.4-SNAPSHOT/maven-metadata.xml (1.2 kB at 2.2 kB/s)
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ megamodelbuild ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @ megamodelbuild ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- exec-maven-plugin:1.5.0:java (default-cli) @ megamodelbuild ---
[WARNING] 
java.lang.ClassNotFoundException: build.pluto.cli.Main
    at java.net.URLClassLoader.findClass (URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass (ClassLoader.java:357)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:281)
    at java.lang.Thread.run (Thread.java:745)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.050 s
[INFO] Finished at: 2018-07-17T15:27:38+02:00
[INFO] Final Memory: 14M/193M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java (default-cli) on project megamodelbuild: An exception occured while executing the Java class. build.pluto.cli.Main -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

May I ask if the script is still relevant to the project?

Sincerely,

Manuel Leduc

[PerditaStevens]

Hi Manuel, No, that script is not relevant - please ignore it. Best wishes, Perdita

…

On Tue, 17 Jul 2018, 15:32 Manuel Leduc, < ***@***.***> wrote: Dear @PerditaStevens <https://github.com/PerditaStevens> , Importing the megamodelbuild project in eclipse was straightforward thanks to maven, and I also easily executed the unit tests to validate that everything was working well. However, I tried to execute the bxexample.sh script out of curiosity and obtained the following error. ./bxexample.sh [INFO] Scanning for projects... Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml Downloaded from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml (605 B at 525 B/s) Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml Downloaded from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml (607 B at 1.1 kB/s) Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/build/pluto/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/build/pluto/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/build/pluto/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/build/pluto/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building megamodelbuild 0.1 [INFO] ------------------------------------------------------------------------ Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/org/sugarj/common/1.7.0-SNAPSHOT/maven-metadata.xml Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/common/1.7.0-SNAPSHOT/maven-metadata.xml Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/org/sugarj/common/1.7.0-SNAPSHOT/maven-metadata.xml Downloaded from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/common/1.7.0-SNAPSHOT/maven-metadata.xml (766 B at 2.2 kB/s) Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/org/sugarj/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/org/sugarj/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml Downloaded from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/maven-config-java/1.1.0-SNAPSHOT/maven-metadata.xml (604 B at 1.6 kB/s) Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/org/sugarj/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/org/sugarj/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml Downloaded from sugar-lang: http://sugar-lang.github.io/mvnrepository/org/sugarj/maven-config-common/1.1.0-SNAPSHOT/maven-metadata.xml (606 B at 1.8 kB/s) Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/pluto/1.9.0-SNAPSHOT/maven-metadata.xml Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/build/pluto/pluto/1.9.0-SNAPSHOT/maven-metadata.xml Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/build/pluto/pluto/1.9.0-SNAPSHOT/maven-metadata.xml Downloaded from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/build/pluto/pluto/1.9.0-SNAPSHOT/maven-metadata.xml (978 B at 1.7 kB/s) Downloading from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/com/cedarsoftware/java-util-pluto-fixes/1.19.4-SNAPSHOT/maven-metadata.xml Downloading from metaborg-nexus-snapshots: http://artifacts.metaborg.org/content/repositories/snapshots/com/cedarsoftware/java-util-pluto-fixes/1.19.4-SNAPSHOT/maven-metadata.xml Downloading from sugar-lang: http://sugar-lang.github.io/mvnrepository/com/cedarsoftware/java-util-pluto-fixes/1.19.4-SNAPSHOT/maven-metadata.xml Downloaded from pluto-build: https://raw.githubusercontent.com/pluto-build/pluto-build.github.io/master/mvnrepository/com/cedarsoftware/java-util-pluto-fixes/1.19.4-SNAPSHOT/maven-metadata.xml (1.2 kB at 2.2 kB/s) [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ megamodelbuild --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 3 resources [INFO] Copying 0 resource [INFO] [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @ megamodelbuild --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- exec-maven-plugin:1.5.0:java (default-cli) @ megamodelbuild --- [WARNING] java.lang.ClassNotFoundException: build.pluto.cli.Main at java.net.URLClassLoader.findClass (URLClassLoader.java:381) at java.lang.ClassLoader.loadClass (ClassLoader.java:424) at java.lang.ClassLoader.loadClass (ClassLoader.java:357) at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:281) at java.lang.Thread.run (Thread.java:745) [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 6.050 s [INFO] Finished at: 2018-07-17T15:27:38+02:00 [INFO] Final Memory: 14M/193M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java (default-cli) on project megamodelbuild: An exception occured while executing the Java class. build.pluto.cli.Main -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException May I ask if the script is still relevant to the project? Sincerely, Manuel Leduc — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AISrEAU_Rh6HoROip2WP6ZkqcjObH9MHks5uHeddgaJpZM4VOISe> .

[AEReview]

** Summary
The paper presents an approach for sound, optimal and flexible build for megamodels. The work extends the work of Erdweg et talk. on Pluto.
The artifact per se, is made of the implementation of two proof of concept (toy) examples proposed by the author.

As-is, the examples are early stage and are illustrations that are backing the theoretical elements in the paper. They do not aim at implementing and validating the author’s approach on a concrete case study, or elaborate on an example of a reasonable size and complexity.

with regard to artifact evaluation criteria:
** Consistent with the paper
The proposed artifact elaborated on some elements in the paper and represent a proof of concept whose tests are executed correctly. As-is they represent a good complement to the paper.

** As complete as possible
N/A.

** Well-documented
I am missing a small introduction and guidelines towards the toy examples (what to expect apart from the logs, or how to interpret the latter).
Though rather intuitive, I would not consider the current artifact as sufficiently documented for an outsider.

** Easy to (re)use
Though the artifact is exercisable, there is little reusability of the current artifact to expect without large, more complex, and more documented examples.

[manuelleduc]

Summary
The submitted artifact is part of a paper presenting an approach to improve the compilation chain of model-driven projects by reasoning on mega-models.
It takes the form of a Java project that demonstrates the use of the Pluto build system in the context of the paper by implementing the two proposed motivational examples.

Documented
The documentation is minimalist, the only executable artifacts are the junit tests of the project.
It could be helpful for the user to have some guidelines of what we can infer from the unit test executions.

Consistent
The artifacts are relevant and models accurately the two examples depicted in the paper.

Complete
The artifacts covers the entirety of the use cases covered in the paper.

Exercisable
The project can build imported, build and tested without problems.

Availability
A snapshot of the proposed project have been submitted to ReMoDD.

Minor comments

• The documentation can be improved easily by integrating an inventory of the provided artifacts;
• I advise to remove the unused bxexample.sh script;
• Even if the code is documented, I feel like it could be more introductory by adding some overview of the relevant samples of code to look at and help the reader find the interesting parts, without having to read the whole code base, in the documentation.

[hernanponcedeleon]

Summary

This artifact accompanies a paper to improve build systems using megamodels and a notion of consistency (consistency can be used for checking which parts of the system need to be re-built).

The artefact consist of Java classes with some documentation and a couple of unit tests.

Assessment

Coverability: Met the expectation

The java classes cover the implementation of the two examples given in the paper.

Packaging: Met expectations

The both the Java classes and the unit test can easily be built using maven.

Reproducibility and consistency: Fell below expectations

I found this artifact very difficult to evaluate in terms of reproducibility or consistency w.r.t the paper.
For the java code, the author claims that the code is expected to be read rather than executed.
However despite the fact that there are classes for each of the examples, I found not way to relate the code (for example the restoreConsistency method of each class) to the paper.
Something similar happens to the unit tests. I found no way (and there is not documentation) to relate this to the paper besides the fact that there is a class with unit tests for each example.

Documentation: Fell below expectations

There is no documentation on what can be done with the artifact besides "reading the code and executing the unit tests". The reviewer has to guess what needs to be done.
Reading other reviewers comments, it seems that several of us were confused and e.g. tried to execute the bxexample.sh script. A nice documentation would have avoided this kind of confusion.
For reading the code, I would have expected some comments on how to related the code, to the paper.
For the unit tests, I would have expected some explanation of what each test is actually verifying or validating.
Also, the documentation is distributed between different README files or comments in different classes, making it very difficult to follow.
Also, it took me a while to find out that the unit test could be executed by running "mvn clean test".

[PerditaStevens]

I am away without good email access (or any chance to update the artefact) right now - but I appreciate the comments about where more documentation would be useful, and will improve matters on my return. Perdita

…

On Wed, 18 Jul 2018, 13:11 Zeta, < ***@***.***> wrote: *Summary* This artifact accompanies a paper to improve build systems using megamodels and a notion of consistency (consistency can be used for checking which parts of the system need to be re-built). The artefact consist of Java classes with some documentation and a couple of unit tests. *Assessment* *Coverability: Met the expectation* The java classes cover the implementation of the two examples given in the paper. *Packaging: Met expectations* The both the Java classes and the unit test can easily be built using maven. *Reproducibility and consistency: Fell below expectations* I found this artifact very difficult to evaluate in terms of reproducibility or consistency w.r.t the paper. For the java code, the author claims that the code is expected to be read rather than executed. However despite the fact that there are classes for each of the examples, I found not way to relate the code (for example the restoreConsistency method of each class) to the paper. Something similar happens to the unit tests. I found no way (and there is not documentation) to relate this to the paper besides the fact that there is a class with unit tests for each example. *Documentation: Fell below expectations* There is no documentation on what can be done with the artifact besides "reading the code and executing the unit tests". The reviewer has to guess what needs to be done. Reading other reviewers comments, it seems that several of us were confused and e.g. tried to execute the bxexample.sh script. A nice documentation would have avoided this kind of confusion. For reading the code, I would have expected some comments on how to related the code, to the paper. For the unit tests, I would have expected some explanation of what each test is actually verifying or validating. Also, the documentation is distributed between different README files or comments in different classes, making it very difficult to follow. Also, it took me a while to find out that the unit test could be executed by running "mvn clean test". — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AISrEKhlXNYn0nI6Xifl8YcAr_0t2Q6Lks5uHxfDgaJpZM4VOISe> .

[grammarware]

Dear @PerditaStevens,

Based on all the comments and the reviews provided by the members of the Artifact Evaluation Committee of MoDELS 2018, we have reached the conclusion that this artifact conforms to the expectations and is hereby approved. Please use the badge instructions page to add the badge of approval to your article, and add the link to the ReMoDD entry with URI http://remodd.org/node/582 to the camera ready version of the paper.

Thank you very much for putting extra effort into the preparation and finalising of the artifact. If any of the comments above are still not addressed, please try to accommodate them before the conference. In particular, documentation could be made more reader-friendly and help potential reusers to navigate through the artifact internals.