Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/research-log.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ The investigation then asked whether vendor and project claims were plausible, a

[FastLanes](https://github.com/cwida/FastLanes), [Nimble](https://github.com/facebookincubator/nimble), and [AnyBlox](https://github.com/AnyBlox/vldb-2025) were attempted from pinned source commits. Mixed-column conversion, official build, or toolchain reproduction did not complete. Their failures are versioned evidence with retry conditions, not blank table cells.

Correction: the first FastLanes numeric failure was caused by the lab writing comma-separated input to a reader configured for a pipe delimiter. A later Linux retry used `|` and the pinned official binding, but one million numeric rows and all five string-boundary cases still exited `SIGSEGV`; the 13-column case raised `RuntimeError: UNREACHABLE`, and malformed comma input raised a normal `RuntimeError`. The earlier pipe-delimited numeric success is therefore host- or workload-specific, not a general claim. The [Linux workflow run](https://github.com/Anionix/data-format-lab/actions/runs/29520329351) retains per-case artifacts and hashes. The robustness lane records these as distinct harness, conformance, and crash-resistance observations rather than one format verdict.
Correction: the first FastLanes numeric failure was caused by the lab writing comma-separated input to a reader configured for a pipe delimiter. A later Linux retry used `|` and the pinned official binding, but one million numeric rows and all five string-boundary cases still exited `SIGSEGV`; the 13-column case raised `RuntimeError: UNREACHABLE`, and malformed comma input raised a normal `RuntimeError`. The earlier pipe-delimited numeric success is therefore host- or workload-specific, not a general claim. The [Linux workflow run](https://github.com/Anionix/data-format-lab/actions/runs/29520329351) retains per-case artifacts and hashes, and the raw archive is preserved in the [v0.1.0 Release](https://github.com/Anionix/data-format-lab/releases/download/v0.1.0/data-format-lab-fastlanes-linux-x86_64-29520329351.tar.zst) with a [checksum sidecar](https://github.com/Anionix/data-format-lab/releases/download/v0.1.0/data-format-lab-fastlanes-linux-x86_64-29520329351.tar.zst.sha256). The robustness lane records these as distinct harness, conformance, and crash-resistance observations rather than one format verdict.

Follow-up on 2026-07-17: the pinned FastLanes Python binding was rebuilt from the official source before attempting the mixed 13-column table. On macOS ARM64 with the lab's Python 3.12 and Clang 21 environment, compilation stopped at `TypedStats<std::string>` with `-Werror,-Wnonnull`. This is a reproducible build blocker, not evidence that the mixed-schema writer itself failed; the machine-readable attempt and exact pinned commit remain in `research/formats/fastlanes.json`.

Expand Down
30 changes: 25 additions & 5 deletions research/formats/fastlanes.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"source_commits": {"core": "f0edc1020a538f1f8098640fce8347c9ac247a0d"},
"primary_source": "https://fastlanes.io/",
"claim": "nested encodings designed for parallel SIMD-friendly decoding",
"attempted_on": "2026-07-14",
"attempted_on": "2026-07-17",
"comparability": "PARTIAL",
"state": "FAILED",
"claim_summary": "results are host- and workload-sensitive: the latest Linux retry crashed in numeric and string cases and failed the mixed-schema harness",
Expand All @@ -29,19 +29,39 @@
{
"entrypoint": "official pyfastlanes binding on Linux x86_64",
"attempted_on": "2026-07-17",
"workflow_run": "https://github.com/Anionix/data-format-lab/actions/runs/29520329351",
"workload": "1,000,000 numeric rows, five string-boundary cases, one 13-column mixed case, and malformed comma input",
"environment": {
"os": "Linux x86_64",
"os": "Ubuntu 24.04 GitHub-hosted runner",
"kernel": "6.17.0-1020-azure",
"architecture": "x86_64",
"runner": "ubuntu-24.04",
"host_python": "3.12.3",
"python": "3.12.13",
"clang": "21.1.8",
"cmake": "4.1.2",
"ninja": "1.13.2",
"scikit_build_core": "0.11.6",
"pybind11": "2.12.1",
"setuptools_scm": "8.3.1",
"build_type": "Release",
"parallelism": 4,
"timeout_seconds": 120,
"package": "pyfastlanes==0.1.3.post9",
"runtime": "0.1.4"
},
"result": "numeric and all five string cases exited SIGSEGV; the mixed 13-column case raised RuntimeError UNREACHABLE; malformed comma input raised RuntimeError. The workflow preserved per-case inputs, artifacts, logs, and SHA-256 hashes."
"build_command": "nix develop --command python -m venv native/fastlanes/.venv; nix develop --command native/fastlanes/.venv/bin/python -m pip install scikit-build-core==0.11.6 cmake==4.4.0 ninja==1.13.0 pybind11==2.12.1 setuptools-scm==8.3.1; CMAKE_BUILD_PARALLEL_LEVEL=4 nix develop --command native/fastlanes/.venv/bin/python -m pip install --no-build-isolation -e native/fastlanes",
"result": "numeric and all five string cases exited SIGSEGV; the mixed 13-column case raised RuntimeError UNREACHABLE; malformed comma input raised RuntimeError. The workflow preserved per-case inputs, artifacts, logs, and SHA-256 hashes.",
"workflow_run": {
"run_id": "29520329351",
"url": "https://github.com/Anionix/data-format-lab/actions/runs/29520329351",
"artifact_name": "data-format-lab-fastlanes-linux-x86_64-29520329351",
"results_path": "runs/fastlanes-linux-29520329351/results.json",
"archive_path": "outputs/release/data-format-lab-fastlanes-linux-x86_64-29520329351.tar.zst",
"archive_sha256": "b9270a10d9cc0f5d81a81e67b8e22af591a06018e767720b9d9c6e4c3689227a",
"release_tag": "v0.1.0",
"release_asset_url": "https://github.com/Anionix/data-format-lab/releases/download/v0.1.0/data-format-lab-fastlanes-linux-x86_64-29520329351.tar.zst",
"release_checksum_url": "https://github.com/Anionix/data-format-lab/releases/download/v0.1.0/data-format-lab-fastlanes-linux-x86_64-29520329351.tar.zst.sha256"
}
},
{
"entrypoint": "official pyfastlanes editable build with scikit-build-core",
Expand Down Expand Up @@ -89,5 +109,5 @@
"result": "target built after warning overrides; all 10 CTest cases exited SIGTRAP"
}
],
"retry_when": "the official Python extension builds in a reproducible toolchain, round-trips partial-vector string dictionaries, and rejects malformed input without terminating the process"
"retry_when": "a pinned source and toolchain complete the 13-column mixed-schema round trip without numeric or string SIGSEGV or UNREACHABLE, with the same result independently reproduced on macOS and Linux"
}
11 changes: 11 additions & 0 deletions tests/test_research.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,17 @@ def test_negative_research_records_are_pinned_and_unranked() -> None:
assert "checksum-complete dataset" in records["anyblox"]["retry_when"]
assert ".anyblox bundle" in records["anyblox"]["retry_when"]

fastlanes_attempt = next(
attempt
for attempt in records["fastlanes"]["attempts"]
if "Linux x86_64" in attempt["entrypoint"]
)
assert fastlanes_attempt["environment"]["architecture"] == "x86_64"
assert fastlanes_attempt["result"].startswith("numeric and all five")
assert fastlanes_attempt["workflow_run"]["run_id"] == "29520329351"
assert fastlanes_attempt["workflow_run"]["archive_sha256"].startswith("b9270a10")
assert fastlanes_attempt["workflow_run"]["release_tag"] == "v0.1.0"

nimble_json = json.dumps(records["nimble"])
assert "/nix/store/" not in nimble_json
assert "/opt/homebrew/" not in nimble_json
Expand Down
Loading