[buehlmaier]

clojask 2.1.0 no longer uses Onyx. Every compute runs on a plain thread pool in your JVM: one thread reads the input in batches, num-worker threads apply the operations and filters, and the calling thread writes the output.

Upgrading from 2.0.x

• Remove the JVM flags 2.0 required: --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED and --enable-native-access=ALL-UNNAMED.
• The clojask.zookeeper.port, clojask.aeron.port, clojask.aeron.dir properties and CLOJASK_* variables are gone, along with the .clojask/clojask.log Onyx log. Several clojask processes can share a machine without any settings, as long as each has its own working directory (.clojask/ holds temporary files).
• Output rows are now always in input order; group-by and outer joins write in group order. :order is still accepted and has no effect.
• enable-debug still turns off the preview check of new operations and disable-debug turns it back on; neither changes a log level any more.
• The API is otherwise unchanged. Errors are still ExecutionException with a stage in the message; the cause is now the original exception.
• Dependencies: the Onyx fork, timbre and clojure-heap are gone, and with them nippy, lz4, Aeron/Agrona, ZooKeeper and curator. The remaining tree is clojask-io, externalsortinginjava 0.6.6 (was 0.6.0) and Clojure 1.12.6 (was 1.11.1); the unused data.csv was dropped.

Changes since 2.0.6

• Add a thread-pool engine for compute behind a flag #88 thread-pool engine for compute, behind a flag.
• Run aggregate and group-by computes on the local engine #89 aggregate, group-by and group-aggregate on the engine.
• Bump clojask-io to 1.0.8 #90 clojask-io 1.0.7 -> 1.0.8 (drops pigpen and Hadoop 1.1 from the tree; csv-family files use the format's default separator before inference).
• Run joins on the local engine #91 all joins on the engine; join lookup without global state.
• This PR: removes Onyx and its 12 namespaces (about 1,800 lines), the engine switch, the JVM flags and port settings; the last functions that def'd globals at run time (defn-join, sort's comparator) are gone; docs updated.

Performance

Synthetic Compustat-like CSV, 1.8M rows / 160 MB, 4 workers, JDK 25, seconds. Measured during the migration (#88, #89, #91), while both engines were in the tree; outputs matched apart from row order and the last digits of group means (see below).

Compute	Onyx (2.0.6)	thread pool (2.1.0)
element-wise operation	8.4	3.7
row-wise operation	9.3	3.7
operations, filter, formatter	14.2	4.4
aggregate	6.4	3.4
group-aggregate, disk	61	46
group-aggregate, memory	16.7	5.6
inner join, disk	78.6	70.3
inner join, memory	31.5	17.0
outer join, disk	81.6	67.9
outer join, memory	31.0	12.2
rolling join, disk	121.4	85.0
rolling join, memory	35.3	15.9

With -Xmx96m the full compute finishes correctly in 9.8 s; 2.0.6 ran out of memory inside ZooKeeper and stalled. Group means can differ from 2.0.6 in the last digits (relative 4e-15): rows now reach the aggregate in input order instead of an arbitrary order.

Tests

engine-test compares 41 cases (compute, aggregate, group-by, every join kind, on disk and in memory, with and without :select) with the output recorded from Onyx before its removal (test/clojask/correct_outputs/engine.edn); the new engine wrote the same for each. regression-test gains the first test of sort, run on externalsortinginjava 0.6.0 and 0.6.6. Suite: 55 tests / 273 assertions in about 8 seconds (was about 3 minutes), green locally on JDK 25 and a JDK 27 early-access build. CI moves to actions/checkout v7, setup-java v6 and cache v6; Leiningen stays at 2.12.0, since the 2.13.0 release's bin/lein reports 2.12.1-SNAPSHOT and cannot be installed.

Not changed: concurrent computes in one working directory are still unsafe, since they share the temporary files under .clojask/.