Bump DuckDB submodule to v1.5.5 - #33
Merged
Merged
Conversation
Moves the pin from v1.5.3 (14eca11b) to v1.5.5 (d8cdaa33). No DuckLake source changes were needed - the extension builds clean against v1.5.5 and the full suite passes: 16083 assertions in 451 test cases. Note this does NOT bring in VARIANT extract pushdown. That lives only on duckdb main; the v1.5-variegata branch is 20 commits past v1.5.5 and still ships the pre-rewrite reader (variant_shredded_conversion.cpp, no IsPushdownExtract), so upstream is not backporting it to the 1.5 line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR
The submodule bump alone was not enough. MainDistributionPipeline's
get-duckdb-version job reads .github/duckdb-version and passes it to
_extension_distribution.yml, which checks out that DuckDB version itself
rather than using the submodule. So CI - including the tagged release
build that produces ducklake-linux-{amd64,arm64}.duckdb_extension - would
still have compiled against v1.5.3 while the submodule said v1.5.5.
Local `make release` uses the submodule and was genuinely built and
tested against v1.5.5; only the CI path was wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR
Bumping to v1.5.5 pulls in DuckDB's newer test suite, and the DBMS
Catalog Tests job runs that suite against a DuckLake backend. 11 of its
4087 cases failed - 10 of them tests that do not exist at v1.5.3 - so
these are newly-added upstream conformance tests exposing long-standing
DuckLake gaps, not regressions from the bump.
Upstream ducklake already made exactly these calls when it moved to
duckdb main (see "Fix and skip duckdb tests in ducklake", which added 136
lines to this file). Every entry here is ported verbatim from upstream
main's attach_ducklake.json, reasons included, so we are not inventing
policy:
- native compression/storage-file internals (patas, rle, dict_fsst)
- native db-file invalidation, checkpoint and WAL-revert semantics
- DuckLake gives table entries a new object id on ALTER, so the oid
stability assertion cannot hold
- parquet writer does not support non-root VARIANT columns
- TIMETZ second-precision offsets are quantized by parquet storage
- two with upstream's own FIXME/"different error message" notes
test_add_col_if_not_exists_default is covered by upstream's
skip_error_messages entry rather than a path skip, so that string is
added too.
Note the diff is larger than the change: json round-trip normalised
pre-existing inconsistent indentation. Semantically this is exactly
+10 skip paths and +1 skip_error_message, nothing removed or altered -
verified by comparing the parsed JSON before and after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR
4 tasks
fuziontech
added a commit
that referenced
this pull request
Jul 30, 2026
* Bump DuckDB submodule to v1.5.5 (#33) * Bump DuckDB submodule to v1.5.5 Moves the pin from v1.5.3 (14eca11b) to v1.5.5 (d8cdaa33). No DuckLake source changes were needed - the extension builds clean against v1.5.5 and the full suite passes: 16083 assertions in 451 test cases. Note this does NOT bring in VARIANT extract pushdown. That lives only on duckdb main; the v1.5-variegata branch is 20 commits past v1.5.5 and still ships the pre-rewrite reader (variant_shredded_conversion.cpp, no IsPushdownExtract), so upstream is not backporting it to the 1.5 line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR * Build against v1.5.5 in CI, not just locally The submodule bump alone was not enough. MainDistributionPipeline's get-duckdb-version job reads .github/duckdb-version and passes it to _extension_distribution.yml, which checks out that DuckDB version itself rather than using the submodule. So CI - including the tagged release build that produces ducklake-linux-{amd64,arm64}.duckdb_extension - would still have compiled against v1.5.3 while the submodule said v1.5.5. Local `make release` uses the submodule and was genuinely built and tested against v1.5.5; only the CI path was wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR * Port upstream DuckDB-test skips for v1.5.5 Bumping to v1.5.5 pulls in DuckDB's newer test suite, and the DBMS Catalog Tests job runs that suite against a DuckLake backend. 11 of its 4087 cases failed - 10 of them tests that do not exist at v1.5.3 - so these are newly-added upstream conformance tests exposing long-standing DuckLake gaps, not regressions from the bump. Upstream ducklake already made exactly these calls when it moved to duckdb main (see "Fix and skip duckdb tests in ducklake", which added 136 lines to this file). Every entry here is ported verbatim from upstream main's attach_ducklake.json, reasons included, so we are not inventing policy: - native compression/storage-file internals (patas, rle, dict_fsst) - native db-file invalidation, checkpoint and WAL-revert semantics - DuckLake gives table entries a new object id on ALTER, so the oid stability assertion cannot hold - parquet writer does not support non-root VARIANT columns - TIMETZ second-precision offsets are quantized by parquet storage - two with upstream's own FIXME/"different error message" notes test_add_col_if_not_exists_default is covered by upstream's skip_error_messages entry rather than a path skip, so that string is added too. Note the diff is larger than the change: json round-trip normalised pre-existing inconsistent indentation. Semantically this is exactly +10 skip paths and +1 skip_error_message, nothing removed or altered - verified by comparing the parsed JSON before and after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * Add commit-loop instrumentation and ducklake_commit_stats() Adds a process-global, mutex-guarded registry of per-catalog commit-loop counters (all cumulative and monotonic per process): - attempts: one per commit-loop iteration - successes: commit loop finished successfully - retries_exhausted: gave up after max retries on a retryable error - nonretryable_errors: errors where RetryOnError returned false - backoff_ms: total ms slept in retry backoff - total_commit_ms: total wall-clock ms inside the commit retry loop - conflicts.{primary_key,unique,conflict,concurrent}: conflicts by cause, classified with the same substring buckets RetryOnError uses The classification lives in a single helper (DuckLakeCommitStatsRegistry::ClassifyCommitError) that now also backs DuckLakeTransaction::RetryOnError - retry semantics are unchanged. Counters are labeled by the DuckLake catalog's attached name for client-side commits and by the metadata schema name for server-side (ducklake_commit) commits, falling back to "default" when unset. New table function ducklake_commit_stats() exposes the registry as (catalog VARCHAR, stat VARCHAR, value BIGINT) rows - one row per (catalog, stat) - and is callable without an attached DuckLake catalog (returns zero rows before any commit). This schema is a contract shared with a duckgres-side consumer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
fuziontech
added a commit
that referenced
this pull request
Jul 30, 2026
) * Backport metadata fix * Clear up unnecessary inlined tables * small fix * test: inlining + disable hive partitioning config * fix: add hive_file_pattern option to flush * fix: add test file to quack.json skip_tests Test uses JOIN on attached ducklake catalog, causing 'Multiple streaming scans' error * Bump DuckDB submodule to v1.5.5 (#33) * Bump DuckDB submodule to v1.5.5 Moves the pin from v1.5.3 (14eca11b) to v1.5.5 (d8cdaa33). No DuckLake source changes were needed - the extension builds clean against v1.5.5 and the full suite passes: 16083 assertions in 451 test cases. Note this does NOT bring in VARIANT extract pushdown. That lives only on duckdb main; the v1.5-variegata branch is 20 commits past v1.5.5 and still ships the pre-rewrite reader (variant_shredded_conversion.cpp, no IsPushdownExtract), so upstream is not backporting it to the 1.5 line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR * Build against v1.5.5 in CI, not just locally The submodule bump alone was not enough. MainDistributionPipeline's get-duckdb-version job reads .github/duckdb-version and passes it to _extension_distribution.yml, which checks out that DuckDB version itself rather than using the submodule. So CI - including the tagged release build that produces ducklake-linux-{amd64,arm64}.duckdb_extension - would still have compiled against v1.5.3 while the submodule said v1.5.5. Local `make release` uses the submodule and was genuinely built and tested against v1.5.5; only the CI path was wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR * Port upstream DuckDB-test skips for v1.5.5 Bumping to v1.5.5 pulls in DuckDB's newer test suite, and the DBMS Catalog Tests job runs that suite against a DuckLake backend. 11 of its 4087 cases failed - 10 of them tests that do not exist at v1.5.3 - so these are newly-added upstream conformance tests exposing long-standing DuckLake gaps, not regressions from the bump. Upstream ducklake already made exactly these calls when it moved to duckdb main (see "Fix and skip duckdb tests in ducklake", which added 136 lines to this file). Every entry here is ported verbatim from upstream main's attach_ducklake.json, reasons included, so we are not inventing policy: - native compression/storage-file internals (patas, rle, dict_fsst) - native db-file invalidation, checkpoint and WAL-revert semantics - DuckLake gives table entries a new object id on ALTER, so the oid stability assertion cannot hold - parquet writer does not support non-root VARIANT columns - TIMETZ second-precision offsets are quantized by parquet storage - two with upstream's own FIXME/"different error message" notes test_add_col_if_not_exists_default is covered by upstream's skip_error_messages entry rather than a path skip, so that string is added too. Note the diff is larger than the change: json round-trip normalised pre-existing inconsistent indentation. Semantically this is exactly +10 skip paths and +1 skip_error_message, nothing removed or altered - verified by comparing the parsed JSON before and after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Pedro Holanda <pedroholanda@gmail.com> Co-authored-by: dbader00 <deanbader03@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves the DuckDB pin from v1.5.3 (
14eca11b) to v1.5.5 (d8cdaa33).No DuckLake source changes were needed — the extension builds clean against v1.5.5 as-is.
Testing
Full suite against v1.5.5: 16083 assertions in 451 test cases, no failures.
Important: this does NOT enable the VARIANT work in #32
Worth being explicit, since this bump might look like it unblocks that PR. It doesn't.
VARIANT extract pushdown (
IsPushdownExtract/GetVariantExtractPathin the Parquet variant reader) exists only on duckdbmain. I checked the 1.5 line directly:v1.5.5ships the pre-rewrite reader:variant_shredded_conversion.cpp, no pushdown.v1.5-variegatais 20 commits past v1.5.5 and still has that same old reader.So upstream is not backporting it to the 1.5 line. Between v1.5.5 and
mainthe reader was rewritten, not extended —variant_shredded_conversion.cpp(596 lines) is deleted and replaced byParquetVariantIterator(715) +parquet_variant_shredding.cpp(396), andvariant_binary_decoder.cppshrinks 465 → 124 lines.I test-applied the pushdown PR (duckdb/duckdb#22478) onto
v1.5.5: 21 conflicting files, because it builds on that rewrite.This bump is still worth taking on its own merits — it's two patch releases of fixes for free. It just isn't a step toward #32.