perf(query): every group by ahead of DuckDB, next to nothing allocated once warm

README.md

@@ -1,27 +1,38 @@
 # Vorticity
 
-A **pure .NET, dependency-free** implementation of the [Vortex](https://vortex.dev) columnar
-file format (LF AI & Data).
+Vorticity is a **high-performance**, pure .NET implementation of the [Vortex](https://vortex.dev)
+columnar file format, with no dependencies: it reads and writes Vortex files faster than Vortex's
+own Rust implementation, and its group bys run faster than DuckDB's.
 
 ## Performance
 
-**Faster than Vortex's Rust implementation on most of what is measured**, process against process
-on the same machine, both sides doing the same work. A table of four columns and 1,048,576 rows,
-Native AOT on an Apple M4 Pro, Rust's time over Vorticity's:
+Each speedup below is the other side's time divided by Vorticity's, so anything above 1.00x means
+Vorticity is faster. Both sides run as separate processes on the same machine, an Apple M4 Pro with
+Native AOT, and do the same work.
 
-| scenario | one core | all 14 cores |
+Against Vortex's own Rust implementation, Vorticity is faster in every scenario, and often twice as
+fast. On a table of four columns and a million rows:
+
+| scenario | speedup, one core | speedup, 14 cores |
 |---|---:|---:|
-| read every column | 1.98x | 2.31x |
-| read one column of four | 2.23x | 2.24x |
-| filter, 1 % of the rows | 1.35x | 2.89x |
-| filter, half the rows | 2.05x | 2.31x |
-| take 1,000 scattered rows | 1.59x | 1.17x |
-| write the table back out | 2.39x | 1.37x |
-
-Per encoding, on one core: decoding faster on 50 of 57 files (median 1.27x), taking rows faster
-from all 57 (median 1.14x), writing faster on all 56 (median 3.82x). A scan allocates next to
-nothing per batch. [How it is measured](#how-performance-is-measured), and every figure on [the
-benchmark page](docs/guide/benchmarks.md).
+| read every column | 1.99x | 2.48x |
+| read one column of four | 2.31x | 2.51x |
+| filter, 1 % of the rows | 1.45x | 3.01x |
+| filter, half the rows | 2.08x | 2.37x |
+| take 1,000 scattered rows | 1.64x | 1.11x |
+| write the table back out | 2.37x | 1.35x |
+
+Against DuckDB, its group bys win all 13 queries we measured, on the same files and with the same
+number of threads:
+
+| DuckDB reading | speedup, one thread | speedup, 14 threads |
+|---|---:|---:|
+| its own table, loaded beforehand | 1.43x to 6.15x | 1.18x to 3.50x |
+| the file, through its Vortex extension | 1.58x to 5.65x | 1.39x to 4.47x |
+
+You can read [how it is measured](#how-performance-is-measured) below, and find every figure on the
+pages [against Rust](docs/guide/benchmarks.md), [against DuckDB](docs/guide/benchmarks-duckdb.md)
+and [on x64](docs/guide/benchmarks-x64.md).
 
 ## Features
 
@@ -154,7 +165,7 @@ at the same version as the core.
 | [docs/guide/](docs/guide/README.md) | **Using the library**: pages named by what you are trying to do, each one a program the samples project compiles and runs |
 | [docs/guide/benchmarks.md](docs/guide/benchmarks.md) | Every figure the bench publishes: what this costs against the Rust implementation, and each kernel against its baseline |
 | [docs/guide/benchmarks-x64.md](docs/guide/benchmarks-x64.md) | The same page from an x64 machine, a Zen 4 processor under Windows, against a Rust built there |
-| [docs/guide/benchmarks-duckdb.md](docs/guide/benchmarks-duckdb.md) | Our group bys against DuckDB's on the same files, through its Vortex reader and from its own table: where we lead and where we trail |
+| [docs/guide/benchmarks-duckdb.md](docs/guide/benchmarks-duckdb.md) | Our group bys against DuckDB's on the same files, through its Vortex reader and from its own table, query by query, on one thread and on fourteen |
 | [docs/design/](docs/design/README.md) | **Why it is shaped this way**: the design documents, from the scope to the byte layout to the public API |
 
 ## How performance is measured
@@ -167,12 +178,12 @@ and hands what it writes to a sink that keeps nothing; on one core, Rust is time
 of its two ways of splitting a scan. The speedup is Rust's time over Vorticity's: above 1.00x,
 Vorticity is faster. The table above is read from the file Vorticity writes.
 
-Per encoding, Vorticity is well ahead on text, nested and compressed integer columns (`varbin`
-5.70x, `struct` 5.10x, `map` 3.11x, `pco` 2.96x), and level or behind on 7 files by a fifth at
-most: booleans that start at a bit offset (0.80x and 0.99x), masked columns (0.93x to 0.99x),
-`onpair` (0.97x) and `decimal` (1.00x). Its file is no more than 5 % larger than Rust's, or
-smaller, on 45 of 56. Both readers warmed up in one process, Rust is faster on 2 of 19 cases: a
-prefix filter on FSST strings (0.93x) and a band over bit-packed integers (0.98x).
+Per encoding, Vorticity is well ahead on nested, text and compressed integer columns (`struct`
+6.92x, `varbin` 6.08x, `map` 3.28x, `pco` 3.06x), and level or behind on 10 files by a tenth at
+most: plain booleans (0.90x), `onpair` (0.95x), `decimal` (0.96x), masked columns (0.96x to
+1.00x), patched bit-packed integers (0.98x), `alprd` and `sequence` (1.00x). Its file is no more
+than 5 % larger than Rust's, or smaller, on 44 of 56. Both readers warmed up in one process, Rust
+is faster on 1 of 19 cases: a prefix filter on FSST strings (0.93x).
 
 Every figure is on [the benchmark page](docs/guide/benchmarks.md), each section with the machine
 and the commit it was measured on, and [its x64 twin](docs/guide/benchmarks-x64.md) holds the

bench/README.md

@@ -58,8 +58,9 @@ finds.
 | every branch a scenario takes | `bench/profile.sh trace <scenario> <file> <rows>` | 30 s | Processor Trace, for Instruments to open. An M4 or later, and Processor Trace allowed in System Settings → Privacy & Security → Developer Tools |
 | who ran, and on what, when a scenario has lanes | `bench/profile.sh cycles <scenario> <file> <rows> --threads all` | 20 s | the cycles above, on as many lanes as the report's all-core rows, plus `threads.md` from `bench/profile/threads.py`: each thread's share, how many threads ran in each millisecond, and the hottest functions of the caller's thread and of the pool's workers. The profiler records fewer of each thread's cycles as more of them run: take shares from it, and the process's processor time from the runner's `cpu_ms` |
 | where a group by spends its cycles | `bench/profile.sh cycles group-total-k6+core ~/.cache/vorticity/queries/spread-random-4000000.vortex 0 --threads 14` | 30 s | the cycles above for a group by of the high-cardinality bench (`group-<aggregates>-<key>`, the plan's switches after `+`: `GroupScenarios.cs` in the runner), over the spread files `vortex-queries --matrix small` writes; the runner prints the engine's counts, the core's with it. The bench compiles as it goes, and Instruments names no frame of a method compiled at run time |
-| a change to the engine, against the last change to the same part | `bench/runners.sh keep <commit>` once, then `bench/runners.sh alt <kept> HEAD --threads 1,14 group-<aggregates>-<key>[@file] …` | 30 s for three scenarios at one lane | the **Native AOT runner** of a commit published once and kept under `~/.cache/vorticity/runners/<sha>`, and two kept runners run in turns, a process each, three pairs by default, the median of the rounds after the first: the ratio of the two binaries, which a loaded machine moves less than it moves a number of milliseconds taken another day. Without a commit, `keep` publishes the working tree's tracked changes. The table prints the machine's load beside each scenario: replay a degree-14 ratio taken under a load above the cores before believing it |
-| our group bys against DuckDB's | `bench/duckdb.sh [--set hc\|db\|all] [--rows N] [--threads 1,14]` | 4 min for both sets, at 1 and 14 threads, its files written | the same files, the bench's own at edition core2025.10, read by the Native AOT runner, by DuckDB's vortex extension and from a DuckDB table in memory, at the same threads: the median of the last 7 of 10 runs, each answer held to the others. `hc` is the high-cardinality table (keys of 10³ to 10⁷ values), `db` db-benchmark's group by, the queries our native aggregates cover. Needs `duckdb` and its vortex extension; informs, gates nothing |
+| a change to the engine, against the last change to the same part | `bench/runners.sh keep <commit>` once, then `bench/runners.sh alt <kept> HEAD --threads 1,14 [--warm] group-<aggregates>-<key>[@file] …` | 1 min for three scenarios at one lane | the **Native AOT runner** of a commit published once and kept under `~/.cache/vorticity/runners/<sha>`, and two kept runners run in turns, a process each, in pairs whose order alternates: three pairs, then more while the 95 % interval of the median ratio is wider than ±3 % (eight at most), the median of the rounds after the first: the ratio of the two binaries, which a loaded machine moves less than it moves a number of milliseconds taken another day. Two canary runs before and after each scenario, against the session's first, catch what other processes did meanwhile; a scenario framed by a canary 5 % off, or during which swap grew, runs again once. The table prints the interval, the pairs, the ratio of the fastest rounds (a gap of 10 % from the median marks the scenario noisy), the ratio of the rounds' cycles, the load and the processes that were busy. `--warm` keeps each side's file mapped (`VORTICITY_RUNNER_WARM=1`). Without a commit, `keep` publishes the working tree's tracked changes. Every run is kept with its time of day in `bench/.ab/runners-<date>/runs.tsv` |
+| our group bys against DuckDB's | `bench/duckdb.sh [--set hc\|db\|all] [--only <rows>] [--rows N] [--threads 1,14] [--bases both\|warm\|cold]`, then `bench/duckdb.sh --publish <session> <session>` | 15 to 25 min a session for both sets at 1 and 14 threads, its files written; 30 s for `--only db-q4,db-q10 --threads 14` | the same files, the bench's own at edition core2025.10, read once before anything is timed, in **alternated pairs**: one DuckDB process a file kept open and driven through a FIFO, its table loaded once, and the Native AOT runner, a block of each in turn, ABBA, three pairs and then more while the 95 % interval of the median speedup is wider than ±5 % (eight at most); 10 rounds a block, 3 dropped, or 5 and 2 past 100 ms. Two bases: our runner with its file kept mapped against DuckDB's table in memory (warm), mapping it anew against DuckDB's vortex reader (cold). DuckDB's time is its JSON profile's latency under 20 ms (to the nanosecond), its timer's past that (the profile cost 2 % on q10); the CPU of both sides beside it. The canaries, controls and replays of `bench/runners.sh`; each answer held to the others. Everything is kept under `bench/.runs/duckdb-<date>/`; `--publish` builds the page's table from two sessions taken at different hours, a row published when they agree within 10 %. `hc` is the high-cardinality table (keys of 10³ to 10⁷ values), `db` db-benchmark's group by, the queries our native aggregates cover; `--only db-q4,total-k7` keeps the rows named by the runner's scenario without `group-`. Needs `duckdb` and its vortex extension; informs, gates nothing |
+| the runner, its file kept mapped, and what a round cost | `VORTICITY_RUNNER_WARM=1 Vorticity.Benchmarks.Runner --scenario … --repeat N` | — | by default each round maps its file anew, as the reference maps it at each call; `VORTICITY_RUNNER_WARM=1` keeps the session's default of 64 mappings, the file already in memory, which is what a comparison against a table loaded beforehand needs. Each round line carries `cpu_us`, `instructions` and `cycles`, the process's on every thread, read with `proc_pid_rusage` as a user (macOS; zero elsewhere): a time that moves while the cycles do not was moved by the machine |
 | a ratchet | `dotnet test Vorticity.slnx -c Release` | ~1 min | the suite plus the ten allocation, count, budget and work ratchets |
 | a change, and only the tests it can affect | `dotnet run -c Release --project tools/testimpact -- run` (after `-- map` once) | seconds | the test classes that ran a method the working tree changed against `HEAD` (`--base REF` for another), from a map of what each class runs: every class in a process of its own with inlining off, the JIT's list of compiled methods recorded (35 s). A constant, a build file or the source generator runs everything; a new class runs; a comment or a `using` runs nothing |
 

bench/Vorticity.Benchmarks.Queries/FloatSumMicro.cs

@@ -0,0 +1,120 @@
+using System;
+using System.Diagnostics;
+using System.Globalization;
+using System.Runtime.InteropServices;
+using Vorticity.Aggregating;
+using Vorticity.Arrays;
+using Vorticity.Buffers;
+using Vorticity.Types;
+
+namespace Vorticity.Benchmarks.Queries;
+
+/// <summary>
+/// A grouped exact float sum alone, in memory (<c>--micro floatsum</c>): a column of doubles folded batch
+/// after batch into the <see cref="IndexedSum"/> of each row's group, by the slot a sum or a mean of
+/// doubles takes, the rows' groups given. No file, no decode, no key: what a change to the sum does, in
+/// seconds. The checksum, the bits of every group's sum folded together, tells two builds' sums apart.
+/// </summary>
+/// <remarks>
+/// The shapes are db-benchmark's: q4's mean of <c>v3</c> by a hundred groups, q3's by a hundred thousand,
+/// q10's sum where every row opens its group; and a hundred groups whose values lie in ranges of their
+/// own. Run it with <c>DOTNET_TieredCompilation=0</c>, so that every pass runs the code at its last tier.
+/// </remarks>
+internal static class FloatSumMicro
+{
+    /// <summary>The rows of a batch: a chunk of the bench's db-benchmark file.</summary>
+    private const int BatchRows = 65_536;
+
+    internal static int Run(int rounds)
+    {
+        Console.WriteLine($"{"float sum",-32} {"rows",11} {"groups",10} {"best ms",10} {"ns/row",8} {"ns/group out",13} {"checksum",18}");
+        Measure("1e2 groups, one range (q4)", 10_000_000, 100, mixed: false, rounds);
+        Measure("1e5 groups, one range (q3)", 10_000_000, 100_000, mixed: false, rounds);
+        Measure("1e2 groups, ranges of their own", 10_000_000, 100, mixed: true, rounds);
+        Measure("a group a row (q10)", 2_000_000, 0, mixed: false, rounds);
+        return 0;
+    }
+
+    /// <summary>The best of <paramref name="rounds"/> passes; <paramref name="groups"/> 0 for a group a row, each row its own in order.</summary>
+    private static void Measure(string name, int rows, int groups, bool mixed, int rounds)
+    {
+        double[] values = new double[rows];
+        int[] rowGroups = new int[rows];
+        for (int row = 0; row < rows; row++)
+        {
+            ulong mix = Mix((ulong)row);
+            int group = groups == 0 ? row : (int)((mix >> 40) % (ulong)groups);
+            rowGroups[row] = group;
+            double unit = (mix >> 11) * (1.0 / (1UL << 53));
+            values[row] = mixed ? Math.ScaleB(unit + 0.5, (group % 16 * 9) - 70) * ((mix & 1) == 0 ? 1 : -1) : unit * 100;
+        }
+
+        double best = double.MaxValue;
+        double bestOut = double.MaxValue;
+        long checksum = 0;
+        int count = groups == 0 ? rows : groups;
+        for (int round = 0; round < rounds; round++)
+        {
+            (double ms, double outMs, long sum) = Pass(values, rowGroups, count);
+            best = Math.Min(best, ms);
+            bestOut = Math.Min(bestOut, outMs);
+            checksum = sum;
+        }
+
+        Console.WriteLine(string.Create(
+            CultureInfo.InvariantCulture,
+            $"{name,-32} {rows,11:N0} {count,10:N0} {best,10:F1} {best * 1e6 / rows,8:F2} {bestOut * 1e6 / count,13:F2} {checksum,18:X16}"));
+    }
+
+    /// <summary>One pass: every batch's values a canonical node of an arena, folded into their groups' sums; then every group's sum read out.</summary>
+    private static (double Ms, double OutMs, long Checksum) Pass(double[] values, int[] rowGroups, int groups)
+    {
+        GC.Collect();
+        RetainingArena arena = new RetainingArena();
+        DType dtype = new DTypeArena().Primitive(PType.F64, Nullability.NonNullable);
+        FixedSlot<double, IndexedSum, IndexedFloatSum<double>, double> slot =
+            new FixedSlot<double, IndexedSum, IndexedFloatSum<double>, double>(StorageKind.Primitive, static s => s.Value);
+        slot.EnsureGroups(groups);
+        long start = Stopwatch.GetTimestamp();
+        long batch = 0;
+        for (int at = 0; at < values.Length; at += BatchRows)
+        {
+            int count = Math.Min(BatchRows, values.Length - at);
+            arena.ResetKeepingBlocks();
+            VortexBuffer buffer = arena.Allocate(count * sizeof(double), sizeof(double), out Span<byte> bytes);
+            values.AsSpan(at, count).CopyTo(MemoryMarshal.Cast<byte, double>(bytes));
+            int node = arena.AddPrimitive(dtype, count, Validity.NonNullable, PType.F64, buffer);
+            slot.StepRows(new BatchInput(++batch, arena, node, count, []), rowGroups.AsSpan(at, count));
+        }
+
+        double ms = Stopwatch.GetElapsedTime(start).TotalMilliseconds;
+
+        // Every group's sum read out at once, as the result's batches read them.
+        int[] order = new int[groups];
+        for (int group = 0; group < groups; group++)
+        {
+            order[group] = group;
+        }
+
+        double[] sums = new double[groups];
+        long reading = Stopwatch.GetTimestamp();
+        slot.Results(order, sums);
+        double outMs = Stopwatch.GetElapsedTime(reading).TotalMilliseconds;
+        long checksum = 0;
+        foreach (double sum in sums)
+        {
+            checksum = (checksum * 31) ^ BitConverter.DoubleToInt64Bits(sum);
+        }
+
+        arena.Reset();
+        return (ms, outMs, checksum);
+    }
+
+    private static ulong Mix(ulong x)
+    {
+        x += 0x9E3779B97F4A7C15UL;
+        x = (x ^ (x >> 30)) * 0xBF58476D1CE4E5B9UL;
+        x = (x ^ (x >> 27)) * 0x94D049BB133111EBUL;
+        return x ^ (x >> 31);
+    }
+}

bench/Vorticity.Benchmarks.Queries/KeyMicro.cs

@@ -23,6 +23,8 @@ internal static class KeyMicro
     internal static int Run(int rounds)
     {
         Console.WriteLine($"{"keys",-28} {"rows",11} {"groups",10} {"bounds ms",10} {"hashed ms",10} {"ratio",7}");
+        Measure("random 1e2", 20_000_000, row => (int)((Mix(Mix((ulong)row)) >> 32) % 100), rounds);
+        Measure("random 1e3", 20_000_000, row => (int)((Mix(Mix((ulong)row)) >> 32) % 1_000), rounds);
         Measure("random 1e6", 20_000_000, row => (int)((Mix(Mix((ulong)row)) >> 32) % 1_000_000), rounds);
         Measure("random 1e7", 20_000_000, row => (int)(Mix(Mix(Mix((ulong)row))) % 10_000_000), rounds);
         Measure("tenfold", 20_000_000, row => (int)((Mix(Mix(Mix((ulong)row))) >> 32) % 2_000_000), rounds);

bench/Vorticity.Benchmarks.Queries/Program.cs

@@ -53,6 +53,18 @@
     return KeyMicro.Run(rounds);
 }
 
+// --micro words: a short text key's groups found alone, in memory (WordMicro.cs).
+if (Text(args, "--micro") == "words")
+{
+    return WordMicro.Run(rounds);
+}
+
+// --micro floatsum: a grouped exact float sum alone, in memory (FloatSumMicro.cs).
+if (Text(args, "--micro") == "floatsum")
+{
+    return FloatSumMicro.Run(rounds);
+}
+
 // Each file is written the first time a query asks for it, and kept.
 Dictionary<string, Func<ValueTask<string>>> fixtures = new Dictionary<string, Func<ValueTask<string>>>(StringComparer.Ordinal)
 {