Build httpfs against DuckDB v1.5.5 - #3
Merged
Conversation
Rebuilds the cred-refresh-write-retry patch set against v1.5.5 so a 1.5.5 worker row can be added to duckgres. Without a 1.5.5 build of this extension the fleet cannot move off 1.5.3: httpfs carries mid-statement S3 credential recovery on ExpiredToken, which is not optional in prod. MainDistributionPipeline reads .github/duckdb-version and passes it to the distribution workflow, so this one line is what selects the DuckDB the extension is compiled against. No source changes are expected - DuckLake made the identical 1.5.3 -> 1.5.5 jump with zero source changes (clean build, 16083 assertions) - but CI on this PR is what actually proves it for httpfs. Motivation for 1.5.5 specifically: it is the first release containing eaf8af373d, which fixes swapped min/max for multi-row-group 128-bit decimals in RETURN_STATS. DuckLake feeds RETURN_STATS straight into ducklake_file_column_stats, and those stats drive file pruning, so on <=1.5.4 a DECIMAL(p>=19) column in a multi-row-group file can prune away files that do contain matches. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Rebuilds the
cred-refresh-write-retrypatch set against v1.5.5 so a 1.5.5 worker row can be added to duckgres.Without a 1.5.5 build of this extension the fleet cannot move off 1.5.3 — httpfs carries mid-statement S3 credential recovery on
ExpiredToken, which is not optional in prod.What
One line:
.github/duckdb-versionv1.5.3→v1.5.5.MainDistributionPipeline.yml'sget-duckdb-versionjob reads that file and passes it to_extension_distribution.yml, so this is what selects the DuckDB the extension compiles against.No source changes are expected — DuckLake made the identical 1.5.3 → 1.5.5 jump with zero source changes (clean build, 16083 assertions in 451 test cases). CI on this PR is what actually proves it for httpfs. If it goes red, the patch set needs forward-porting and that becomes the real work.
Why 1.5.5 specifically
v1.5.5 is the first release containing
eaf8af373d, "Fix swapped min/max for multi-row-group 128-bit decimals in RETURN_STATS" — not in v1.5.3 or v1.5.4.That matters beyond httpfs: DuckLake requests
WRITTEN_FILE_STATISTICSand parsesmin/maxstraight intoducklake_file_column_stats, and those stats drive file pruning. On ≤1.5.4, aDECIMAL(p>=19)column in a multi-row-group file can therefore have wrong recorded bounds and prune away files that do contain matches — surfacing as silently missing rows rather than an error.Next steps once this is green
v1.5.5-cred-refresh-write-retry→ publisheshttpfs-linux-{amd64,arm64}.duckdb_extension.posthog/v1.5.5→ publishes the matching DuckLake build.1.5.5row (default: false) to both matrices in duckgrescontainer-image-worker-cd.yml, canary, then flip the default.🤖 Generated with Claude Code
https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR