diff --git a/README.md b/README.md index 413adfe12..516b49d20 100644 --- a/README.md +++ b/README.md @@ -1,27 +1,38 @@ # Vorticity -A **pure .NET, dependency-free** implementation of the [Vortex](https://vortex.dev) columnar -file format (LF AI & Data). +Vorticity is a **high-performance**, pure .NET implementation of the [Vortex](https://vortex.dev) +columnar file format, with no dependencies: it reads and writes Vortex files faster than Vortex's +own Rust implementation, and its group bys run faster than DuckDB's. ## Performance -**Faster than Vortex's Rust implementation on most of what is measured**, process against process -on the same machine, both sides doing the same work. A table of four columns and 1,048,576 rows, -Native AOT on an Apple M4 Pro, Rust's time over Vorticity's: +Each speedup below is the other side's time divided by Vorticity's, so anything above 1.00x means +Vorticity is faster. Both sides run as separate processes on the same machine, an Apple M4 Pro with +Native AOT, and do the same work. -| scenario | one core | all 14 cores | +Against Vortex's own Rust implementation, Vorticity is faster in every scenario, and often twice as +fast. On a table of four columns and a million rows: + +| scenario | speedup, one core | speedup, 14 cores | |---|---:|---:| -| read every column | 1.98x | 2.31x | -| read one column of four | 2.23x | 2.24x | -| filter, 1 % of the rows | 1.35x | 2.89x | -| filter, half the rows | 2.05x | 2.31x | -| take 1,000 scattered rows | 1.59x | 1.17x | -| write the table back out | 2.39x | 1.37x | - -Per encoding, on one core: decoding faster on 50 of 57 files (median 1.27x), taking rows faster -from all 57 (median 1.14x), writing faster on all 56 (median 3.82x). A scan allocates next to -nothing per batch. [How it is measured](#how-performance-is-measured), and every figure on [the -benchmark page](docs/guide/benchmarks.md). +| read every column | 1.99x | 2.48x | +| read one column of four | 2.31x | 2.51x | +| filter, 1 % of the rows | 1.45x | 3.01x | +| filter, half the rows | 2.08x | 2.37x | +| take 1,000 scattered rows | 1.64x | 1.11x | +| write the table back out | 2.37x | 1.35x | + +Against DuckDB, its group bys win all 13 queries we measured, on the same files and with the same +number of threads: + +| DuckDB reading | speedup, one thread | speedup, 14 threads | +|---|---:|---:| +| its own table, loaded beforehand | 1.43x to 6.15x | 1.18x to 3.50x | +| the file, through its Vortex extension | 1.58x to 5.65x | 1.39x to 4.47x | + +You can read [how it is measured](#how-performance-is-measured) below, and find every figure on the +pages [against Rust](docs/guide/benchmarks.md), [against DuckDB](docs/guide/benchmarks-duckdb.md) +and [on x64](docs/guide/benchmarks-x64.md). ## Features @@ -154,7 +165,7 @@ at the same version as the core. | [docs/guide/](docs/guide/README.md) | **Using the library**: pages named by what you are trying to do, each one a program the samples project compiles and runs | | [docs/guide/benchmarks.md](docs/guide/benchmarks.md) | Every figure the bench publishes: what this costs against the Rust implementation, and each kernel against its baseline | | [docs/guide/benchmarks-x64.md](docs/guide/benchmarks-x64.md) | The same page from an x64 machine, a Zen 4 processor under Windows, against a Rust built there | -| [docs/guide/benchmarks-duckdb.md](docs/guide/benchmarks-duckdb.md) | Our group bys against DuckDB's on the same files, through its Vortex reader and from its own table: where we lead and where we trail | +| [docs/guide/benchmarks-duckdb.md](docs/guide/benchmarks-duckdb.md) | Our group bys against DuckDB's on the same files, through its Vortex reader and from its own table, query by query, on one thread and on fourteen | | [docs/design/](docs/design/README.md) | **Why it is shaped this way**: the design documents, from the scope to the byte layout to the public API | ## How performance is measured @@ -167,12 +178,12 @@ and hands what it writes to a sink that keeps nothing; on one core, Rust is time of its two ways of splitting a scan. The speedup is Rust's time over Vorticity's: above 1.00x, Vorticity is faster. The table above is read from the file Vorticity writes. -Per encoding, Vorticity is well ahead on text, nested and compressed integer columns (`varbin` -5.70x, `struct` 5.10x, `map` 3.11x, `pco` 2.96x), and level or behind on 7 files by a fifth at -most: booleans that start at a bit offset (0.80x and 0.99x), masked columns (0.93x to 0.99x), -`onpair` (0.97x) and `decimal` (1.00x). Its file is no more than 5 % larger than Rust's, or -smaller, on 45 of 56. Both readers warmed up in one process, Rust is faster on 2 of 19 cases: a -prefix filter on FSST strings (0.93x) and a band over bit-packed integers (0.98x). +Per encoding, Vorticity is well ahead on nested, text and compressed integer columns (`struct` +6.92x, `varbin` 6.08x, `map` 3.28x, `pco` 3.06x), and level or behind on 10 files by a tenth at +most: plain booleans (0.90x), `onpair` (0.95x), `decimal` (0.96x), masked columns (0.96x to +1.00x), patched bit-packed integers (0.98x), `alprd` and `sequence` (1.00x). Its file is no more +than 5 % larger than Rust's, or smaller, on 44 of 56. Both readers warmed up in one process, Rust +is faster on 1 of 19 cases: a prefix filter on FSST strings (0.93x). Every figure is on [the benchmark page](docs/guide/benchmarks.md), each section with the machine and the commit it was measured on, and [its x64 twin](docs/guide/benchmarks-x64.md) holds the diff --git a/bench/README.md b/bench/README.md index 4bd07282f..d484ed85d 100644 --- a/bench/README.md +++ b/bench/README.md @@ -58,8 +58,9 @@ finds. | every branch a scenario takes | `bench/profile.sh trace ` | 30 s | Processor Trace, for Instruments to open. An M4 or later, and Processor Trace allowed in System Settings → Privacy & Security → Developer Tools | | who ran, and on what, when a scenario has lanes | `bench/profile.sh cycles --threads all` | 20 s | the cycles above, on as many lanes as the report's all-core rows, plus `threads.md` from `bench/profile/threads.py`: each thread's share, how many threads ran in each millisecond, and the hottest functions of the caller's thread and of the pool's workers. The profiler records fewer of each thread's cycles as more of them run: take shares from it, and the process's processor time from the runner's `cpu_ms` | | where a group by spends its cycles | `bench/profile.sh cycles group-total-k6+core ~/.cache/vorticity/queries/spread-random-4000000.vortex 0 --threads 14` | 30 s | the cycles above for a group by of the high-cardinality bench (`group--`, the plan's switches after `+`: `GroupScenarios.cs` in the runner), over the spread files `vortex-queries --matrix small` writes; the runner prints the engine's counts, the core's with it. The bench compiles as it goes, and Instruments names no frame of a method compiled at run time | -| a change to the engine, against the last change to the same part | `bench/runners.sh keep ` once, then `bench/runners.sh alt HEAD --threads 1,14 group--[@file] …` | 30 s for three scenarios at one lane | the **Native AOT runner** of a commit published once and kept under `~/.cache/vorticity/runners/`, and two kept runners run in turns, a process each, three pairs by default, the median of the rounds after the first: the ratio of the two binaries, which a loaded machine moves less than it moves a number of milliseconds taken another day. Without a commit, `keep` publishes the working tree's tracked changes. The table prints the machine's load beside each scenario: replay a degree-14 ratio taken under a load above the cores before believing it | -| our group bys against DuckDB's | `bench/duckdb.sh [--set hc\|db\|all] [--rows N] [--threads 1,14]` | 4 min for both sets, at 1 and 14 threads, its files written | the same files, the bench's own at edition core2025.10, read by the Native AOT runner, by DuckDB's vortex extension and from a DuckDB table in memory, at the same threads: the median of the last 7 of 10 runs, each answer held to the others. `hc` is the high-cardinality table (keys of 10³ to 10⁷ values), `db` db-benchmark's group by, the queries our native aggregates cover. Needs `duckdb` and its vortex extension; informs, gates nothing | +| a change to the engine, against the last change to the same part | `bench/runners.sh keep ` once, then `bench/runners.sh alt HEAD --threads 1,14 [--warm] group--[@file] …` | 1 min for three scenarios at one lane | the **Native AOT runner** of a commit published once and kept under `~/.cache/vorticity/runners/`, and two kept runners run in turns, a process each, in pairs whose order alternates: three pairs, then more while the 95 % interval of the median ratio is wider than ±3 % (eight at most), the median of the rounds after the first: the ratio of the two binaries, which a loaded machine moves less than it moves a number of milliseconds taken another day. Two canary runs before and after each scenario, against the session's first, catch what other processes did meanwhile; a scenario framed by a canary 5 % off, or during which swap grew, runs again once. The table prints the interval, the pairs, the ratio of the fastest rounds (a gap of 10 % from the median marks the scenario noisy), the ratio of the rounds' cycles, the load and the processes that were busy. `--warm` keeps each side's file mapped (`VORTICITY_RUNNER_WARM=1`). Without a commit, `keep` publishes the working tree's tracked changes. Every run is kept with its time of day in `bench/.ab/runners-/runs.tsv` | +| our group bys against DuckDB's | `bench/duckdb.sh [--set hc\|db\|all] [--only ] [--rows N] [--threads 1,14] [--bases both\|warm\|cold]`, then `bench/duckdb.sh --publish ` | 15 to 25 min a session for both sets at 1 and 14 threads, its files written; 30 s for `--only db-q4,db-q10 --threads 14` | the same files, the bench's own at edition core2025.10, read once before anything is timed, in **alternated pairs**: one DuckDB process a file kept open and driven through a FIFO, its table loaded once, and the Native AOT runner, a block of each in turn, ABBA, three pairs and then more while the 95 % interval of the median speedup is wider than ±5 % (eight at most); 10 rounds a block, 3 dropped, or 5 and 2 past 100 ms. Two bases: our runner with its file kept mapped against DuckDB's table in memory (warm), mapping it anew against DuckDB's vortex reader (cold). DuckDB's time is its JSON profile's latency under 20 ms (to the nanosecond), its timer's past that (the profile cost 2 % on q10); the CPU of both sides beside it. The canaries, controls and replays of `bench/runners.sh`; each answer held to the others. Everything is kept under `bench/.runs/duckdb-/`; `--publish` builds the page's table from two sessions taken at different hours, a row published when they agree within 10 %. `hc` is the high-cardinality table (keys of 10³ to 10⁷ values), `db` db-benchmark's group by, the queries our native aggregates cover; `--only db-q4,total-k7` keeps the rows named by the runner's scenario without `group-`. Needs `duckdb` and its vortex extension; informs, gates nothing | +| the runner, its file kept mapped, and what a round cost | `VORTICITY_RUNNER_WARM=1 Vorticity.Benchmarks.Runner --scenario … --repeat N` | — | by default each round maps its file anew, as the reference maps it at each call; `VORTICITY_RUNNER_WARM=1` keeps the session's default of 64 mappings, the file already in memory, which is what a comparison against a table loaded beforehand needs. Each round line carries `cpu_us`, `instructions` and `cycles`, the process's on every thread, read with `proc_pid_rusage` as a user (macOS; zero elsewhere): a time that moves while the cycles do not was moved by the machine | | a ratchet | `dotnet test Vorticity.slnx -c Release` | ~1 min | the suite plus the ten allocation, count, budget and work ratchets | | a change, and only the tests it can affect | `dotnet run -c Release --project tools/testimpact -- run` (after `-- map` once) | seconds | the test classes that ran a method the working tree changed against `HEAD` (`--base REF` for another), from a map of what each class runs: every class in a process of its own with inlining off, the JIT's list of compiled methods recorded (35 s). A constant, a build file or the source generator runs everything; a new class runs; a comment or a `using` runs nothing | diff --git a/bench/Vorticity.Benchmarks.Queries/FloatSumMicro.cs b/bench/Vorticity.Benchmarks.Queries/FloatSumMicro.cs new file mode 100644 index 000000000..9316d2bee --- /dev/null +++ b/bench/Vorticity.Benchmarks.Queries/FloatSumMicro.cs @@ -0,0 +1,120 @@ +using System; +using System.Diagnostics; +using System.Globalization; +using System.Runtime.InteropServices; +using Vorticity.Aggregating; +using Vorticity.Arrays; +using Vorticity.Buffers; +using Vorticity.Types; + +namespace Vorticity.Benchmarks.Queries; + +/// +/// A grouped exact float sum alone, in memory (--micro floatsum): a column of doubles folded batch +/// after batch into the of each row's group, by the slot a sum or a mean of +/// doubles takes, the rows' groups given. No file, no decode, no key: what a change to the sum does, in +/// seconds. The checksum, the bits of every group's sum folded together, tells two builds' sums apart. +/// +/// +/// The shapes are db-benchmark's: q4's mean of v3 by a hundred groups, q3's by a hundred thousand, +/// q10's sum where every row opens its group; and a hundred groups whose values lie in ranges of their +/// own. Run it with DOTNET_TieredCompilation=0, so that every pass runs the code at its last tier. +/// +internal static class FloatSumMicro +{ + /// The rows of a batch: a chunk of the bench's db-benchmark file. + private const int BatchRows = 65_536; + + internal static int Run(int rounds) + { + Console.WriteLine($"{"float sum",-32} {"rows",11} {"groups",10} {"best ms",10} {"ns/row",8} {"ns/group out",13} {"checksum",18}"); + Measure("1e2 groups, one range (q4)", 10_000_000, 100, mixed: false, rounds); + Measure("1e5 groups, one range (q3)", 10_000_000, 100_000, mixed: false, rounds); + Measure("1e2 groups, ranges of their own", 10_000_000, 100, mixed: true, rounds); + Measure("a group a row (q10)", 2_000_000, 0, mixed: false, rounds); + return 0; + } + + /// The best of passes; 0 for a group a row, each row its own in order. + private static void Measure(string name, int rows, int groups, bool mixed, int rounds) + { + double[] values = new double[rows]; + int[] rowGroups = new int[rows]; + for (int row = 0; row < rows; row++) + { + ulong mix = Mix((ulong)row); + int group = groups == 0 ? row : (int)((mix >> 40) % (ulong)groups); + rowGroups[row] = group; + double unit = (mix >> 11) * (1.0 / (1UL << 53)); + values[row] = mixed ? Math.ScaleB(unit + 0.5, (group % 16 * 9) - 70) * ((mix & 1) == 0 ? 1 : -1) : unit * 100; + } + + double best = double.MaxValue; + double bestOut = double.MaxValue; + long checksum = 0; + int count = groups == 0 ? rows : groups; + for (int round = 0; round < rounds; round++) + { + (double ms, double outMs, long sum) = Pass(values, rowGroups, count); + best = Math.Min(best, ms); + bestOut = Math.Min(bestOut, outMs); + checksum = sum; + } + + Console.WriteLine(string.Create( + CultureInfo.InvariantCulture, + $"{name,-32} {rows,11:N0} {count,10:N0} {best,10:F1} {best * 1e6 / rows,8:F2} {bestOut * 1e6 / count,13:F2} {checksum,18:X16}")); + } + + /// One pass: every batch's values a canonical node of an arena, folded into their groups' sums; then every group's sum read out. + private static (double Ms, double OutMs, long Checksum) Pass(double[] values, int[] rowGroups, int groups) + { + GC.Collect(); + RetainingArena arena = new RetainingArena(); + DType dtype = new DTypeArena().Primitive(PType.F64, Nullability.NonNullable); + FixedSlot, double> slot = + new FixedSlot, double>(StorageKind.Primitive, static s => s.Value); + slot.EnsureGroups(groups); + long start = Stopwatch.GetTimestamp(); + long batch = 0; + for (int at = 0; at < values.Length; at += BatchRows) + { + int count = Math.Min(BatchRows, values.Length - at); + arena.ResetKeepingBlocks(); + VortexBuffer buffer = arena.Allocate(count * sizeof(double), sizeof(double), out Span bytes); + values.AsSpan(at, count).CopyTo(MemoryMarshal.Cast(bytes)); + int node = arena.AddPrimitive(dtype, count, Validity.NonNullable, PType.F64, buffer); + slot.StepRows(new BatchInput(++batch, arena, node, count, []), rowGroups.AsSpan(at, count)); + } + + double ms = Stopwatch.GetElapsedTime(start).TotalMilliseconds; + + // Every group's sum read out at once, as the result's batches read them. + int[] order = new int[groups]; + for (int group = 0; group < groups; group++) + { + order[group] = group; + } + + double[] sums = new double[groups]; + long reading = Stopwatch.GetTimestamp(); + slot.Results(order, sums); + double outMs = Stopwatch.GetElapsedTime(reading).TotalMilliseconds; + long checksum = 0; + foreach (double sum in sums) + { + checksum = (checksum * 31) ^ BitConverter.DoubleToInt64Bits(sum); + } + + arena.Reset(); + return (ms, outMs, checksum); + } + + private static ulong Mix(ulong x) + { + x += 0x9E3779B97F4A7C15UL; + x = (x ^ (x >> 30)) * 0xBF58476D1CE4E5B9UL; + x = (x ^ (x >> 27)) * 0x94D049BB133111EBUL; + return x ^ (x >> 31); + } +} diff --git a/bench/Vorticity.Benchmarks.Queries/KeyMicro.cs b/bench/Vorticity.Benchmarks.Queries/KeyMicro.cs index 003c4279f..c50143d75 100644 --- a/bench/Vorticity.Benchmarks.Queries/KeyMicro.cs +++ b/bench/Vorticity.Benchmarks.Queries/KeyMicro.cs @@ -23,6 +23,8 @@ internal static class KeyMicro internal static int Run(int rounds) { Console.WriteLine($"{"keys",-28} {"rows",11} {"groups",10} {"bounds ms",10} {"hashed ms",10} {"ratio",7}"); + Measure("random 1e2", 20_000_000, row => (int)((Mix(Mix((ulong)row)) >> 32) % 100), rounds); + Measure("random 1e3", 20_000_000, row => (int)((Mix(Mix((ulong)row)) >> 32) % 1_000), rounds); Measure("random 1e6", 20_000_000, row => (int)((Mix(Mix((ulong)row)) >> 32) % 1_000_000), rounds); Measure("random 1e7", 20_000_000, row => (int)(Mix(Mix(Mix((ulong)row))) % 10_000_000), rounds); Measure("tenfold", 20_000_000, row => (int)((Mix(Mix(Mix((ulong)row))) >> 32) % 2_000_000), rounds); diff --git a/bench/Vorticity.Benchmarks.Queries/Program.cs b/bench/Vorticity.Benchmarks.Queries/Program.cs index 5291a08e0..7afc20f44 100644 --- a/bench/Vorticity.Benchmarks.Queries/Program.cs +++ b/bench/Vorticity.Benchmarks.Queries/Program.cs @@ -53,6 +53,18 @@ return KeyMicro.Run(rounds); } +// --micro words: a short text key's groups found alone, in memory (WordMicro.cs). +if (Text(args, "--micro") == "words") +{ + return WordMicro.Run(rounds); +} + +// --micro floatsum: a grouped exact float sum alone, in memory (FloatSumMicro.cs). +if (Text(args, "--micro") == "floatsum") +{ + return FloatSumMicro.Run(rounds); +} + // Each file is written the first time a query asks for it, and kept. Dictionary>> fixtures = new Dictionary>>(StringComparer.Ordinal) { diff --git a/bench/Vorticity.Benchmarks.Queries/WordMicro.cs b/bench/Vorticity.Benchmarks.Queries/WordMicro.cs new file mode 100644 index 000000000..89f974f28 --- /dev/null +++ b/bench/Vorticity.Benchmarks.Queries/WordMicro.cs @@ -0,0 +1,115 @@ +using System; +using System.Buffers.Binary; +using System.Diagnostics; +using System.Globalization; +using System.Runtime.InteropServices; +using System.Text; +using Vorticity.Aggregating; +using Vorticity.Arrays; +using Vorticity.Buffers; +using Vorticity.Expressions; +using Vorticity.Types; + +namespace Vorticity.Benchmarks.Queries; + +/// +/// A short text key's groups found alone, in memory (--micro words): texts of 5, 8 and 12 bytes, a +/// hundred to a hundred thousand of them drawn at random, handed batch after batch to +/// .Assign in the canonical views a decoder makes of them, every view holding +/// its value whole. No file, no decode, no aggregate: what a change to the words does, in seconds. +/// +/// +/// The shapes are db-benchmark's: id1 and id2, a hundred texts of 5 bytes; id3, texts +/// of 12 bytes, a hundred thousand of them at 10⁷ rows. Run it with DOTNET_TieredCompilation=0, so +/// that every pass runs the code at its last tier. +/// +internal static class WordMicro +{ + /// The rows of a batch: a chunk of the bench's db-benchmark file. + private const int BatchRows = 65_536; + + private const int ViewSize = 16; + + internal static int Run(int rounds) + { + Console.WriteLine($"{"words",-28} {"rows",11} {"groups",10} {"best ms",10} {"ns/row",8}"); + Measure("5 bytes, 1e2 (id1)", 10_000_000, 5, 100, rounds); + Measure("8 bytes, 1e4", 10_000_000, 8, 10_000, rounds); + Measure("12 bytes, 1e2", 10_000_000, 12, 100, rounds); + Measure("12 bytes, 1e5 (id3)", 10_000_000, 12, 100_000, rounds); + return 0; + } + + /// The best of passes over rows of texts of bytes. + private static void Measure(string name, int rows, int length, int values, int rounds) + { + byte[] views = Views(rows, length, values); + double best = double.MaxValue; + int groups = 0; + for (int round = 0; round < rounds; round++) + { + (double ms, int count) = Pass(views, rows); + best = Math.Min(best, ms); + groups = count; + } + + Console.WriteLine(string.Create( + CultureInfo.InvariantCulture, + $"{name,-28} {rows,11:N0} {groups,10:N0} {best,10:F1} {best * 1e6 / rows,8:F2}")); + } + + /// + /// Every row's view: its length, then its text, id and a number of − 2 + /// digits drawn among , zero past it, as a decoder writes an inline view. + /// + private static byte[] Views(int rows, int length, int values) + { + byte[] views = new byte[rows * ViewSize]; + string format = "D" + (length - 2).ToString(CultureInfo.InvariantCulture); + for (int row = 0; row < rows; row++) + { + int value = (int)(Mix((ulong)row) % (ulong)values); + Span view = views.AsSpan(row * ViewSize, ViewSize); + BinaryPrimitives.WriteUInt32LittleEndian(view, (uint)length); + Encoding.ASCII.GetBytes("id" + value.ToString(format, CultureInfo.InvariantCulture), view[4..]); + } + + return views; + } + + /// One pass: every batch's views a canonical node of an arena, assigned to groups. + private static (double Ms, int Groups) Pass(byte[] views, int rows) + { + GC.Collect(); + RetainingArena arena = new RetainingArena(); + DType dtype = new DTypeArena().Utf8(Nullability.NonNullable); + ColumnShape shape = new ColumnShape(new ColumnSym(Expr.Field("k"), VortexType.Utf8, null, null, -1, [])); + ShortTextKeys groups = new ShortTextKeys(shape, sorted: false); + ulong[] selection = []; + int[] rowGroups = new int[BatchRows]; + GroupRanges ranges = new GroupRanges(); + long start = Stopwatch.GetTimestamp(); + for (int at = 0; at < rows; at += BatchRows) + { + int count = Math.Min(BatchRows, rows - at); + arena.ResetKeepingBlocks(); + VortexBuffer buffer = arena.Allocate(count * ViewSize, ViewSize, out Span bytes); + views.AsSpan(at * ViewSize, count * ViewSize).CopyTo(bytes); + int node = arena.AddVarBinView(dtype, count, Validity.NonNullable, buffer, []); + ranges.Clear(); + groups.Assign(arena, [node], count, selection, rowGroups, ranges); + } + + double ms = Stopwatch.GetElapsedTime(start).TotalMilliseconds; + arena.Reset(); + return (ms, groups.Count); + } + + private static ulong Mix(ulong x) + { + x += 0x9E3779B97F4A7C15UL; + x = (x ^ (x >> 30)) * 0xBF58476D1CE4E5B9UL; + x = (x ^ (x >> 27)) * 0x94D049BB133111EBUL; + return x ^ (x >> 31); + } +} diff --git a/bench/Vorticity.Benchmarks.Runner/GroupScenarios.cs b/bench/Vorticity.Benchmarks.Runner/GroupScenarios.cs index 12717011f..f83f6cdd1 100644 --- a/bench/Vorticity.Benchmarks.Runner/GroupScenarios.cs +++ b/bench/Vorticity.Benchmarks.Runner/GroupScenarios.cs @@ -38,7 +38,8 @@ namespace Vorticity.Bench.Runner; /// streams at every degree), blocking (no stream), core (the core at every degree), /// whole (the core's groups delivered whole), tables (the lanes' tables, the core taken /// neither from the first batch nor from the first rows), nofan (the lanes all start at the head -/// of the queue), noretire (lanes under pressure write their tables rather than retire into +/// of the queue), numbers (a key of five to eight columns as the numbers of its columns rather +/// than its tuples), noretire (lanes under pressure write their tables rather than retire into /// one), budget=N (the session's memory budget, in MiB, under which the runner prints its peak), /// capacity=N, alpha=N, floor=N, table=N, batch=N, window=N, probe=N, bypass=N /// (the core's ε, in percent). The degree is the runner's --threads. @@ -196,6 +197,7 @@ private static Action Switch(string option) plan.CoreOnNew = false; }, "nofan" => static plan => plan.Fan = false, + "numbers" => static plan => plan.Tuples = false, "capacity" => plan => plan.CoreCapacity = Valued(value, name), "alpha" => plan => plan.CoreAlpha = Valued(value, name), "floor" => plan => plan.CoreFloor = Valued(value, name), @@ -206,7 +208,7 @@ private static Action Switch(string option) "bypass" => plan => plan.CoreBypass = Valued(value, name) / 100.0, "noretire" => static plan => plan.LanesRetire = false, "budget" => Budget(Valued(value, name)), - _ => throw new ArgumentException($"No switch named '{name}': zones, blocking, core, whole, tables, nofan, noretire, budget=N (MiB), capacity=N, alpha=N, floor=N, table=N, batch=N, window=N, probe=N, bypass=N (percent).", nameof(option)), + _ => throw new ArgumentException($"No switch named '{name}': zones, blocking, core, whole, tables, nofan, numbers, noretire, budget=N (MiB), capacity=N, alpha=N, floor=N, table=N, batch=N, window=N, probe=N, bypass=N (percent).", nameof(option)), }; } diff --git a/bench/Vorticity.Benchmarks.Runner/Program.cs b/bench/Vorticity.Benchmarks.Runner/Program.cs index c2f3b895a..b1de5ff6a 100644 --- a/bench/Vorticity.Benchmarks.Runner/Program.cs +++ b/bench/Vorticity.Benchmarks.Runner/Program.cs @@ -71,11 +71,14 @@ private static async Task Main(string[] args) // A round maps its file anew, as the reference maps it at each call: the default session // would hand round two the mapping round one left, pages already mapped. The bench host // sets the same session in `Scenarios.OpenFilesCold`. A group by takes its lanes from the - // session, as the queries bench gives them. + // session, as the queries bench gives them. VORTICITY_RUNNER_WARM=1 keeps the session's + // default instead, 64 files mapped once and taken over by the next open: the file already in + // memory, the base a comparison against a table loaded beforehand takes (bench/duckdb.sh). QueryMemoryBudget? budget = grouped && GroupScenarios.BudgetBytes is long bytes ? new QueryMemoryBudget(bytes) : null; + bool warm = Environment.GetEnvironmentVariable("VORTICITY_RUNNER_WARM") == "1"; ScenarioSet.Session = VortexSession.Create(options => { - options.MappedFileCacheCount = 0; + options.MappedFileCacheCount = warm ? 64 : 0; if (grouped) { options.MaxDegreeOfParallelism = threads; @@ -88,15 +91,32 @@ private static async Task Main(string[] args) long[] delivered = new long[repeat]; long[] workMicros = new long[repeat]; long[] allocated = new long[repeat]; + ProcessCounters[] spent = new ProcessCounters[repeat]; + + // The collections of each generation a round called, and the time they held every thread: the + // stops a lane cannot work through. + int[] collections = new int[repeat * 3]; + long[] pausedMicros = new long[repeat]; long before = AllocatedSoFar(); for (int round = 0; round < repeat; round++) { // The action is timed from inside the process, once it is up: what the report compares // is the work, and the process start is a property of the build that the parent times - // apart. + // apart. The processor's counters are read outside the timed span, on every thread of the + // process: what the round cost the machine beside how long it took. + int gen0 = GC.CollectionCount(0); + int gen1 = GC.CollectionCount(1); + int gen2 = GC.CollectionCount(2); + TimeSpan paused = GC.GetTotalPauseDuration(); + ProcessCounters counted = ProcessCost.ReadCounters(); long started = Stopwatch.GetTimestamp(); delivered[round] = await scenario(args[2]).ConfigureAwait(false); workMicros[round] = (long)(Stopwatch.GetElapsedTime(started).TotalMilliseconds * 1000); + spent[round] = ProcessCost.ReadCounters().Since(counted); + pausedMicros[round] = (long)(GC.GetTotalPauseDuration() - paused).TotalMicroseconds; + collections[(3 * round) + 0] = GC.CollectionCount(0) - gen0; + collections[(3 * round) + 1] = GC.CollectionCount(1) - gen1; + collections[(3 * round) + 2] = GC.CollectionCount(2) - gen2; long after = AllocatedSoFar(); allocated[round] = after - before; before = after; @@ -108,7 +128,7 @@ private static async Task Main(string[] args) { Console.WriteLine(string.Create( CultureInfo.InvariantCulture, - $"round={round} rows={delivered[round]} work_us={workMicros[round]} allocated_bytes={allocated[round]}")); + $"round={round} rows={delivered[round]} work_us={workMicros[round]} allocated_bytes={allocated[round]} cpu_us={spent[round].CpuNanoseconds / 1000} instructions={spent[round].Instructions} cycles={spent[round].Cycles} gc={collections[3 * round]}/{collections[(3 * round) + 1]}/{collections[(3 * round) + 2]} gc_paused_us={pausedMicros[round]}")); } } diff --git a/bench/Vorticity.Benchmarks.Scenarios/ProcessCost.cs b/bench/Vorticity.Benchmarks.Scenarios/ProcessCost.cs index f714aa3db..042010a36 100644 --- a/bench/Vorticity.Benchmarks.Scenarios/ProcessCost.cs +++ b/bench/Vorticity.Benchmarks.Scenarios/ProcessCost.cs @@ -38,6 +38,70 @@ public static unsafe (long CpuMs, long RssBytes) Read() RuntimeInformation.IsOSPlatform(OSPlatform.OSX) ? maxrss : maxrss * 1024); } + /// + /// What the process has spent so far, to be read before and after a round: its processor time in + /// nanoseconds, user and system, and on macOS the instructions it retired and the cycles it ran, + /// the counters /usr/bin/time -l prints, which a user reads without privileges. Zero where + /// the platform has no such counters. A round whose time moves while its cycles do not was slowed by + /// the machine, not by its code. + /// + /// + /// proc_pid_rusage with RUSAGE_INFO_V4: the times at bytes 16 and 24 in Mach absolute + /// units, the instructions at 248 and the cycles at 256 (sys/resource.h, checked against + /// offsetof on 2026-10-09). No allocation: a round reads it twice. + /// + public static unsafe ProcessCounters ReadCounters() + { + if (!RuntimeInformation.IsOSPlatform(OSPlatform.OSX)) + { + (long cpuMs, _) = Read(); + return new ProcessCounters(cpuMs * 1_000_000, 0, 0); + } + + byte* info = stackalloc byte[RusageInfoV4Bytes]; + if (proc_pid_rusage(Environment.ProcessId, RusageInfoV4, info) != 0) + { + return default; + } + + ulong ticks = *(ulong*)(info + 16) + *(ulong*)(info + 24); + return new ProcessCounters(TicksToNanoseconds(ticks), (long)*(ulong*)(info + 248), (long)*(ulong*)(info + 256)); + } + + private const int RusageInfoV4 = 4; + + /// The bytes of struct rusage_info_v4, 296 on macOS 26, with room to spare. + private const int RusageInfoV4Bytes = 512; + + private static readonly (uint Numer, uint Denom) Timebase = ReadTimebase(); + + private static long TicksToNanoseconds(ulong ticks) => (long)(ticks * Timebase.Numer / Timebase.Denom); + + private static unsafe (uint, uint) ReadTimebase() + { + if (!RuntimeInformation.IsOSPlatform(OSPlatform.OSX)) + { + return (1, 1); + } + + uint* timebase = stackalloc uint[2]; + return mach_timebase_info(timebase) == 0 && timebase[1] != 0 ? (timebase[0], timebase[1]) : (1, 1); + } + [DllImport("libc", EntryPoint = "getrusage")] private static extern unsafe int getrusage(int who, byte* usage); + + [DllImport("libc", EntryPoint = "proc_pid_rusage")] + private static extern unsafe int proc_pid_rusage(int pid, int flavor, byte* buffer); + + [DllImport("libc", EntryPoint = "mach_timebase_info")] + private static extern unsafe int mach_timebase_info(uint* timebase); +} + +/// A process's processor time in nanoseconds, and the instructions and cycles it ran, so far. +public readonly record struct ProcessCounters(long CpuNanoseconds, long Instructions, long Cycles) +{ + /// What was spent between and this reading. + public ProcessCounters Since(ProcessCounters before) => + new ProcessCounters(CpuNanoseconds - before.CpuNanoseconds, Instructions - before.Instructions, Cycles - before.Cycles); } diff --git a/bench/duckdb.sh b/bench/duckdb.sh index 6981629d4..9f1336123 100755 --- a/bench/duckdb.sh +++ b/bench/duckdb.sh @@ -1,19 +1,55 @@ #!/usr/bin/env bash -# Our group bys against DuckDB's, on the same files: DuckDB through its vortex extension and on a native -# table already in memory, our side the Native AOT runner, at the same threads. +# Our group bys against DuckDB's, on the same files, in alternated pairs: DuckDB through its vortex +# extension and on a native table already in memory, our side the Native AOT runner, at the same threads. # -# bench/duckdb.sh [--set hc|db|all] [--rows N] [--db-rows N] [--threads N[,M…]] [--runner |] -# [--rounds N] [--rest S] +# bench/duckdb.sh [--set hc|db|all] [--only ] [--rows N] [--db-rows N] [--threads N[,M…]] +# [--runner |] [--bases both|warm|cold] [--target P] [--most N] [--rest S] +# [--no-canary] +# bench/duckdb.sh --publish [] # # bench/duckdb.sh PR #43's table: 40M rows, 1 and 14 threads # bench/duckdb.sh --set db --db-rows 10000000 db-benchmark's group by at 10^7 rows -# bench/duckdb.sh --rows 4000000 --threads 1 a quick look +# bench/duckdb.sh --set all --only db-q4,total-k7 --threads 14 +# the rows a change touches, and no others: the +# runner's scenario names without "group-" (total-k3, +# range-k6, strided-k7, db-q10…) +# bench/duckdb.sh --publish bench/.runs/duckdb-20261010-0900 bench/.runs/duckdb-20261010-1500 +# the page's table, from two sessions at different hours # # THE SAME WORK ON BOTH SIDES. Each query is a group by whose result is aggregated once more: DuckDB's outer # query sums every column the group by aggregates, and the runner sums the same columns as it reads the # groups (GroupScenarios.Checksum). The two sums are the same answer, and the table says when they are -# not. Each figure is the median of the last 7 of 10 runs (--rounds), each side's process started once a -# query; DuckDB's timer counts whole milliseconds. The native table's time excludes its loading. +# not. +# +# TWO BASES. Warm: our runner with its file kept mapped (VORTICITY_RUNNER_WARM=1) against DuckDB's native +# table, loaded before anything is timed. Cold: our runner mapping its file anew each round against +# DuckDB's vortex reader. A comparison of a table in memory with a file mapped anew measured page faults, +# not engines. +# +# PAIRS, NOT ONE SIDE THEN THE OTHER. One DuckDB process a file, kept open and driven through a FIFO, its +# table loaded once. For each row, degree and base, a block of DuckDB's and a process of our runner in +# turn, ABBA (DuckDB then us, us then DuckDB…), which cancels a drift of the machine: run all of one side +# then all of the other, a perturbation biased one side unseen (the published q10 at fourteen threads, +# 257 ms for DuckDB where it took 102 the same day). A block is 10 rounds, the first 3 dropped, or 5, the +# first 2 dropped, once a row's first block passes 100 ms. A pair's speedup is DuckDB's median over ours; +# a row's, the median of its pairs', 3 pairs and then one more while the 95 % interval of that median (a +# bootstrap of the pairs) is wider than ±5 % (--target), 8 at most (--most). DuckDB's time is its JSON +# profile's latency, to the nanosecond, a profile a round (its timer counts whole milliseconds); its CPU, +# the profile's cpu_time. Ours, the runner's work_us and cpu_us. +# +# THE MACHINE, as bench/runners.sh watches it, all of it as a user: before and after each row and base, +# two canaries of our runner with their files kept mapped (a scan of one lane, a count and sum of 10^3 +# keys on 14 lanes), the first values of the session their standard; the busiest other process, which +# holds a row back while it takes more than half a core (30 s at most); the swap, the free memory and the +# power source. A row framed by a canary more than 5 % off whose pairs did not hold the target, or during +# which the swap grew, is run again once. THE FILES are read once before anything is timed, so that both +# engines find them in the page cache. +# +# A SESSION keeps everything it ran under bench/.runs/duckdb-/ (ignored by git): runs.tsv, each +# block, canary and control with its time of day; session.tsv, each row's result; DuckDB's output and +# profiles. --publish takes two sessions run at different hours and prints the page's table: a row is +# published when the two agree within 10 %, a third session deciding otherwise, beside its pairs' range +# and count, then the sessions' canaries and controls. # # THE FILES are the bench's own, written by its generators (bench/Vorticity.Benchmarks.Queries --fixture), # canonical, at edition core2025.10, the one the extension reads (it does not know vortex.zoned); both @@ -28,40 +64,141 @@ # DuckDB is a workstation dependency, as cargo is for the comparison with Rust: duckdb on the PATH (or # DUCKDB), its vortex extension installed (INSTALL vortex). Never in CI. These figures inform; they gate # nothing (the plan's steps are judged on their ratchets). A figure for publication is taken on a machine -# at rest: docs/guide/benchmarks-duckdb.md. +# at rest, twice: docs/guide/benchmarks-duckdb.md. set -uo pipefail root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" duckdb="${DUCKDB:-$(command -v duckdb || echo /opt/homebrew/bin/duckdb)}" +corpus="${VORTICITY_QUERIES_CORPUS:-$HOME/.cache/vorticity/queries}" set_name="hc" +only="" rows=40000000 db_rows=10000000 threads="1,14" runner="HEAD" -rounds=10 +bases="both" +target=5 +most=8 rest=0 +canary=1 +publish=() usage() { - echo "usage: bench/duckdb.sh [--set hc|db|all] [--rows N] [--db-rows N] [--threads N[,M…]] [--runner |] [--rounds N] [--rest S]" >&2 + echo "usage: bench/duckdb.sh [--set hc|db|all] [--only ] [--rows N] [--db-rows N] [--threads N[,M…]] [--runner |] [--bases both|warm|cold] [--target P] [--most N] [--rest S] [--no-canary]" >&2 + echo " bench/duckdb.sh --publish []" >&2 exit 2 } while [ $# -gt 0 ]; do case "$1" in --set) set_name="$2"; shift 2 ;; + --only) only=",$2,"; shift 2 ;; --rows) rows="$2"; shift 2 ;; --db-rows) db_rows="$2"; shift 2 ;; --threads) threads="$2"; shift 2 ;; --runner) runner="$2"; shift 2 ;; - --rounds) rounds="$2"; shift 2 ;; + --bases) bases="$2"; shift 2 ;; + --target) target="$2"; shift 2 ;; + --most) most="$2"; shift 2 ;; --rest) rest="$2"; shift 2 ;; + --no-canary) canary=0; shift ;; + --publish) shift; while [ $# -gt 0 ]; do publish+=("$1"); shift; done ;; *) usage ;; esac done +# The awk functions the decisions and the tables share: a median, and the 95 % interval of a median by a +# bootstrap of 2 000 draws, seeded so that the same ratios give the same interval (as bench/runners.sh). +stats=' + function sortn(a, n, gap, i, j, t) { + for (gap = int(n / 2); gap > 0; gap = int(gap / 2)) + for (i = gap; i < n; i++) { t = a[i]; for (j = i; j >= gap && a[j - gap] + 0 > t + 0; j -= gap) a[j] = a[j - gap]; a[j] = t } + } + function median(a, n, c, i) { + for (i = 0; i < n; i++) c[i] = a[i] + sortn(c, n) + return n % 2 ? c[int(n / 2)] : (c[n / 2 - 1] + c[n / 2]) / 2 + } + function interval(r, n, b, i, s, m) { + srand(7) + for (b = 0; b < 2000; b++) { for (i = 0; i < n; i++) s[i] = r[int(rand() * n)]; m[b] = median(s, n) } + sortn(m, 2000) + low = m[49]; high = m[1950] + }' + +# --publish: the page's table from two or three sessions' session.tsv. A row of a base is published when +# two sessions agree within 10 %: the mean of their speedups, the widest of their 95 % intervals, their +# pairs summed; none agreeing, it is marked and its speedups printed. +if [ ${#publish[@]} -gt 0 ]; then + [ ${#publish[@]} -ge 2 ] || usage + files=() + for session in "${publish[@]}"; do + [ -f "$session/session.tsv" ] || { echo "no session.tsv in $session" >&2; exit 2; } + files+=("$session/session.tsv") + done + + echo "# Our group bys against DuckDB's" + echo + for session in "${publish[@]}"; do + echo "- session $(basename "$session"): $(head -1 "$session/about.txt" 2> /dev/null)" + done + + echo "- a speedup is DuckDB's time over ours, above 1.00x we are faster: the mean of the two sessions that agreed within 10 %, in brackets the widest of their 95 % intervals and their pairs" + echo + echo "| query | rows | threads | ours warm (ms) | DuckDB native table (ms) | speedup | ours cold (ms) | DuckDB Vortex reader (ms) | speedup | answer |" + echo "|---|---:|---:|---:|---:|---:|---:|---:|---:|---|" + awk -F '\t' ' + FNR == 1 { s++ } + { + key = $1 SUBSEP $2 SUBSEP $3 + if (!(key in seen)) { seen[key] = 1; order[++n] = key; label[key] = $1; rows[key] = $2; degree[key] = $3 } + k = key SUBSEP $4; c = count[k]++ + ours[k, c] = $5; duck[k, c] = $6; sp[k, c] = $7; lo[k, c] = $8; hi[k, c] = $9; pairs[k, c] = $10 + if ($12 != "same") answer[key] = "DIFFERENT" + } + function cell(k, i, j, best, bi, bj, d, m) { + best = -1 + for (i = 0; i < count[k]; i++) + for (j = i + 1; j < count[k]; j++) { + d = sp[k, i] / sp[k, j] - 1; if (d < 0) d = -d + if (d <= 0.10 && (best < 0 || d < best)) { best = d; bi = i; bj = j } + } + if (count[k] == 0) return "| | |" + if (best < 0) { + m = "" + for (i = 0; i < count[k]; i++) m = m sprintf("%s%.2fx", (i ? ", " : ""), sp[k, i]) + return sprintf("%.1f | %.1f | unsettled: %s", ours[k, 0], duck[k, 0], m) + } + return sprintf("%.1f | %.1f | %.2fx [%.2f–%.2f, %d pairs]", (ours[k, bi] + ours[k, bj]) / 2, (duck[k, bi] + duck[k, bj]) / 2, + (sp[k, bi] + sp[k, bj]) / 2, (lo[k, bi] < lo[k, bj] ? lo[k, bi] : lo[k, bj]), (hi[k, bi] > hi[k, bj] ? hi[k, bi] : hi[k, bj]), pairs[k, bi] + pairs[k, bj]) + } + END { + for (i = 1; i <= n; i++) { + key = order[i] + printf "| %s | %s | %s | %s | %s | %s |\n", label[key], rows[key], degree[key], cell(key SUBSEP "warm"), cell(key SUBSEP "cold"), (key in answer) ? answer[key] : "same" + } + }' "${files[@]}" + + echo + echo "The machine during each session (its canaries' spread around their standard, the controls' worst):" + echo + for session in "${publish[@]}"; do + awk -F '\t' -v name="$(basename "$session")" ' + $1 == "canary" { scan = $7 + 0; lanes = $8 + 0; split($9, st, "/") + d = scan / st[1] - 1; if (d < 0) d = -d; if (d > worst) worst = d + d = lanes / st[2] - 1; if (d < 0) d = -d; if (d > worst) worst = d; canaries++ } + $1 == "control" { if ($8 + 0 > swap) swap = $8 + 0; if ($5 == "before") waited += $6; if ($10 != "ac") battery = 1 } + $1 == "replay" { replays++ } + END { printf "- %s: %d canaries, at most %.1f %% off their standard; %d rows replayed; %d s waited on another process; swap at most %s MB; %s\n", + name, canaries, 100 * worst, replays, waited, swap, battery ? "on battery at times" : "on mains power" }' "$session/runs.tsv" + done + + exit 0 +fi + [ -x "$duckdb" ] || { echo "no duckdb at '$duckdb': install it, or set DUCKDB" >&2; exit 2; } "$duckdb" -c "LOAD vortex;" > /dev/null 2>&1 || { echo "duckdb cannot load its vortex extension: run INSTALL vortex in it" >&2; exit 2; } -[ "$rounds" -ge 7 ] || { echo "--rounds must be 7 or more: the median is of the last 7" >&2; exit 2; } +case "$bases" in both) base_list="warm cold" ;; warm|cold) base_list="$bases" ;; *) usage ;; esac # The runner: a directory, or a commit's, kept by bench/runners.sh. if [ -d "$runner" ]; then @@ -77,9 +214,6 @@ fixture() { dotnet run -c Release --project "$root/bench/Vorticity.Benchmarks.Queries" -- --fixture "$1" 2> /dev/null | tail -1 } -# The median of the last 7 of the numbers on stdin, one a line. -median7() { tail -7 | sort -n | sed -n 4p; } - # The rows of the comparison: a label, the runner's scenario, the fixture, and DuckDB's query over `x`. queries=() if [ "$set_name" = "hc" ] || [ "$set_name" = "all" ]; then @@ -105,6 +239,17 @@ if [ "$set_name" = "db" ] || [ "$set_name" = "all" ]; then queries+=("db q10: sum v3, count by id1:id6|group-db-q10|$db|SELECT sum(v3) + sum(c) FROM (SELECT Id1, Id2, Id3, Id4, Id5, Id6, sum(V3) AS v3, count(*) AS c FROM x GROUP BY Id1, Id2, Id3, Id4, Id5, Id6)") fi +# --only keeps the rows it names, by their scenario without "group-". +if [ -n "$only" ]; then + kept_queries=() + for entry in "${queries[@]}"; do + IFS='|' read -r _ scenario _ _ <<< "$entry" + case "$only" in *",${scenario#group-},"*) kept_queries+=("$entry") ;; esac + done + + queries=(${kept_queries[@]+"${kept_queries[@]}"}) +fi + [ ${#queries[@]} -gt 0 ] || usage IFS=',' read -r -a degrees <<< "$threads" @@ -129,93 +274,340 @@ for entry in "${queries[@]}"; do paths+=("$path") done -# DuckDB's side, a process a file: its view over the extension and its native table, then each query at -# each degree, `rounds` times on each, each block marked. Prints "