Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions docs/format-selection.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,13 @@ attempts. They remain research leads, not production recommendations. Their
exact commits, failures, and retry conditions are recorded in the
[negative-evidence records](../research/formats/).

FastLanes also has no general selection yet. Its corrected pipe-delimited
numeric case round-tripped, but partial-vector string cases failed, malformed
input terminated the process, and the pinned Python binding hit a reproducible
macOS build blocker. Keep it as experimental evidence until those retry
conditions are resolved.
FastLanes also has no general selection yet. Results are host- and
workload-sensitive: an earlier pipe-delimited numeric case round-tripped, but
the pinned Linux retry crashed in the one-million-row numeric case and all
string-boundary cases, while the mixed 13-column case raised an internal
`UNREACHABLE` error. The pinned macOS Python binding also has a reproducible
build blocker. Keep it as experimental evidence until these retry conditions
are resolved.

DuckDB is a query engine and can be useful with several of these formats. It is
not a competing file-format row in this guide.
Expand Down
2 changes: 1 addition & 1 deletion docs/research-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ The investigation then asked whether vendor and project claims were plausible, a

[FastLanes](https://github.com/cwida/FastLanes), [Nimble](https://github.com/facebookincubator/nimble), and [AnyBlox](https://github.com/AnyBlox/vldb-2025) were attempted from pinned source commits. Mixed-column conversion, official build, or toolchain reproduction did not complete. Their failures are versioned evidence with retry conditions, not blank table cells.

Correction: the FastLanes numeric failure was caused by the lab writing comma-separated input to a reader configured for a pipe delimiter. With `|`, one million rows by eight numeric columns round-trip. Separate string tests still show a partial-vector boundary: 1,024 and 2,048 rows succeed while 1,023, 1,025, and 2,049 fail dictionary lookup. Invalid comma input can also terminate the process. The robustness lane records these as distinct harness, conformance, and crash-resistance observations rather than one format verdict.
Correction: the first FastLanes numeric failure was caused by the lab writing comma-separated input to a reader configured for a pipe delimiter. A later Linux retry used `|` and the pinned official binding, but one million numeric rows and all five string-boundary cases still exited `SIGSEGV`; the 13-column case raised `RuntimeError: UNREACHABLE`, and malformed comma input raised a normal `RuntimeError`. The earlier pipe-delimited numeric success is therefore host- or workload-specific, not a general claim. The [Linux workflow run](https://github.com/Anionix/data-format-lab/actions/runs/29520329351) retains per-case artifacts and hashes. The robustness lane records these as distinct harness, conformance, and crash-resistance observations rather than one format verdict.

Follow-up on 2026-07-17: the pinned FastLanes Python binding was rebuilt from the official source before attempting the mixed 13-column table. On macOS ARM64 with the lab's Python 3.12 and Clang 21 environment, compilation stopped at `TypedStats<std::string>` with `-Werror,-Wnonnull`. This is a reproducible build blocker, not evidence that the mixed-schema writer itself failed; the machine-readable attempt and exact pinned commit remain in `research/formats/fastlanes.json`.

Expand Down
21 changes: 19 additions & 2 deletions research/formats/fastlanes.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
"attempted_on": "2026-07-14",
"comparability": "PARTIAL",
"state": "FAILED",
"claim_summary": "numeric data round-tripped; partial-vector string dictionary lookup failed; Python binding rebuild blocked before mixed-schema test",
"correction": "the earlier numeric -11 result used comma-separated input against the official pipe-delimited reader and is superseded",
"claim_summary": "results are host- and workload-sensitive: the latest Linux retry crashed in numeric and string cases and failed the mixed-schema harness",
"correction": "the earlier numeric -11 result used comma-separated input against the official pipe-delimited reader; a separate Linux retry with the corrected delimiter still observed SIGSEGV, so the earlier numeric success is not a general claim",
"attempts": [
{
"entrypoint": "official pipe-delimited numeric conversion",
Expand All @@ -26,6 +26,23 @@
"workload": "malformed delimiter input",
"result": "process exited -11; retained as robustness evidence"
},
{
"entrypoint": "official pyfastlanes binding on Linux x86_64",
"attempted_on": "2026-07-17",
"workflow_run": "https://github.com/Anionix/data-format-lab/actions/runs/29520329351",
"workload": "1,000,000 numeric rows, five string-boundary cases, one 13-column mixed case, and malformed comma input",
"environment": {
"os": "Linux x86_64",
"runner": "ubuntu-24.04",
"python": "3.12.13",
"clang": "21.1.8",
"cmake": "4.1.2",
"ninja": "1.13.2",
Comment on lines +39 to +40

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Record the build-tool versions installed by the workflow

The inspected Linux workflow installs cmake==4.4.0 and ninja==1.13.0 into the FastLanes virtual environment immediately before the editable build (.github/workflows/benchmark-fastlanes.yml:74-83), while this evidence lists only the Nix-shell versions 4.1.2 and 1.13.2. This leaves the recorded build environment inconsistent with the workflow; distinguish the shell versions from the virtual-environment build dependencies and record both.

Useful? React with 👍 / 👎.

"package": "pyfastlanes==0.1.3.post9",
"runtime": "0.1.4"
},
"result": "numeric and all five string cases exited SIGSEGV; the mixed 13-column case raised RuntimeError UNREACHABLE; malformed comma input raised RuntimeError. The workflow preserved per-case inputs, artifacts, logs, and SHA-256 hashes."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist the advertised hashes beyond artifact expiry

The only referenced copies of the per-case files and SHA-256 values are uploaded by .github/workflows/benchmark-fastlanes.yml:292-301 with retention-days: 14; none of the hashes or an archive digest is included in this record. After that window, the workflow link can no longer substantiate this newly recorded result, so persist the hashes here or publish the archive at a durable location.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include numeric and mixed cases in the retry criteria

This attempt newly reports failures for both the numeric control and mixed-schema case, but retry_when at line 92 still requires only a successful build, partial-vector strings, and malformed-input handling. A retry could therefore satisfy every documented condition while still reproducing these two fatal failures; extend the criteria to require the numeric and mixed round trips as well.

Useful? React with 👍 / 👎.

},
{
"entrypoint": "official pyfastlanes editable build with scikit-build-core",
"attempted_on": "2026-07-17",
Expand Down
Loading