Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions docs/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,14 +213,16 @@ For unsupported extension ids the inspector falls back to a placeholder cell
correctly via the primitive accessors, callers just have to format the value
themselves.

## S3 Fixture Status (v0.72.0)
## S3 Fixture Status (v0.75.0)

> **Note:** the fixture matrix below is locked to `v0.72.0/`. The Rust reference is
> now at `v0.74.0`; re-run the integration suite against `v0.74.0/arrays/` once
> upstream publishes the corresponding fixture set, and refresh this section.
> **Note:** the oracle round-trip suite is pinned to `v0.75.0/` (current Rust
> release). The bucket reuses identical fixture file names across versions but
> rewrites the bytes, so the `/tmp/pco-fixtures` cache is version-keyed; bump the
> `FIXTURE_VERSION` constant in the integration tests and refresh this section
> when a newer set is published.

Cross-language round-trips tested against Rust-written fixture files hosted at
`s3://vortex-compat-fixtures/v0.72.0/arrays/`.
`s3://vortex-compat-fixtures/v0.75.0/arrays/`.

| Fixture | Status |
|-------------------------------------|--------|
Expand Down Expand Up @@ -259,6 +261,6 @@ Cross-language round-trips tested against Rust-written fixture files hosted at
| `pco.vortex` | ✅ |
| `clickbench_hits_5k.compact.vortex` | ✅ |
| `clickbench_hits_5k.regular.vortex` | ✅ |
| `masked.vortex` | ❓ | No fixture in v0.72.0 |
| `patched.vortex` | ❓ | No fixture in v0.72.0 |
| `variant.vortex` | ❓ | No fixture in v0.72.0 |
| `masked.vortex` | ❓ | No fixture through v0.75.0 |
| `patched.vortex` | ❓ | No fixture through v0.75.0 |
| `variant.vortex` | ❓ | No fixture through v0.75.0 |
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,9 @@
/// implement first).
class PcoFixtureInspectionIntegrationTest {

private static final String BASE = "https://vortex-compat-fixtures.s3.amazonaws.com/v0.72.0/arrays/";
private static final String FIXTURE_VERSION = "v0.75.0";
private static final String BASE =
"https://vortex-compat-fixtures.s3.amazonaws.com/" + FIXTURE_VERSION + "/arrays/";
private static final String[] FIXTURES = {
"pco.vortex",
"tpch_lineitem.compact.vortex",
Expand Down Expand Up @@ -303,8 +305,11 @@ private static String formatDType(DType d) {
}

private static Path downloadIfMissing(Path tmp, String name) throws Exception {
// Reuse /tmp/pco-fixtures cache if present.
Path cached = Path.of("/tmp/pco-fixtures", name);
// Reuse the version-keyed /tmp/pco-fixtures cache if present. The version
// segment matters: the Rust reference rewrites identical file names with
// different bytes across versions, so a version-less cache would serve
// stale bytes after a bump.
Path cached = Path.of("/tmp/pco-fixtures", FIXTURE_VERSION, name);
if (Files.exists(cached)) {
return cached;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
class RustJavaReaderComparisonIntegrationTest {

private static final URI BASE =
URI.create("https://vortex-compat-fixtures.s3.amazonaws.com/v0.72.0/arrays/");
URI.create("https://vortex-compat-fixtures.s3.amazonaws.com/v0.75.0/arrays/");

private static final Session SESSION = Session.create();
private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@
/// Cross-compatibility: Rust (JNI) writer → Java reader.
class RustWritesJavaReadsIntegrationTest {

private static final String S3_BASE = "https://vortex-compat-fixtures.s3.amazonaws.com/v0.72.0/arrays/";
private static final String FIXTURE_VERSION = "v0.75.0";
private static final String S3_BASE =
"https://vortex-compat-fixtures.s3.amazonaws.com/" + FIXTURE_VERSION + "/arrays/";

private static final Session SESSION = Session.create();
private static final BufferAllocator ALLOCATOR = ArrowAllocation.rootAllocator();
Expand Down Expand Up @@ -218,7 +220,10 @@ private static long[] readJavaLongColumn(Path file, String column) throws IOExce
}

private static Path downloadIfMissing(Path tmp, String name) throws Exception {
Path cached = Path.of("/tmp/pco-fixtures", name);
// Cache is keyed by fixture version: the Rust reference rewrites the same
// file names with different bytes across versions, so a version-less cache
// would silently serve stale bytes after a version bump.
Path cached = Path.of("/tmp/pco-fixtures", FIXTURE_VERSION, name);
if (Files.exists(cached)) {
return cached;
}
Expand Down
Loading