Skip to content

Build httpfs against DuckDB v1.5.5 - #3

Merged
fuziontech merged 1 commit into
posthog/v1.5.3from
posthog/v1.5.5
Jul 29, 2026
Merged

Build httpfs against DuckDB v1.5.5#3
fuziontech merged 1 commit into
posthog/v1.5.3from
posthog/v1.5.5

Conversation

@fuziontech

Copy link
Copy Markdown
Member

Why

Rebuilds the cred-refresh-write-retry patch set against v1.5.5 so a 1.5.5 worker row can be added to duckgres.

Without a 1.5.5 build of this extension the fleet cannot move off 1.5.3 — httpfs carries mid-statement S3 credential recovery on ExpiredToken, which is not optional in prod.

What

One line: .github/duckdb-version v1.5.3v1.5.5.

MainDistributionPipeline.yml's get-duckdb-version job reads that file and passes it to _extension_distribution.yml, so this is what selects the DuckDB the extension compiles against.

No source changes are expected — DuckLake made the identical 1.5.3 → 1.5.5 jump with zero source changes (clean build, 16083 assertions in 451 test cases). CI on this PR is what actually proves it for httpfs. If it goes red, the patch set needs forward-porting and that becomes the real work.

Why 1.5.5 specifically

v1.5.5 is the first release containing eaf8af373d, "Fix swapped min/max for multi-row-group 128-bit decimals in RETURN_STATS" — not in v1.5.3 or v1.5.4.

That matters beyond httpfs: DuckLake requests WRITTEN_FILE_STATISTICS and parses min/max straight into ducklake_file_column_stats, and those stats drive file pruning. On ≤1.5.4, a DECIMAL(p>=19) column in a multi-row-group file can therefore have wrong recorded bounds and prune away files that do contain matches — surfacing as silently missing rows rather than an error.

Next steps once this is green

  1. Tag this branch v1.5.5-cred-refresh-write-retry → publishes httpfs-linux-{amd64,arm64}.duckdb_extension.
  2. Merge Bump DuckDB submodule to v1.5.5 ducklake#33 and tag posthog/v1.5.5 → publishes the matching DuckLake build.
  3. Add a 1.5.5 row (default: false) to both matrices in duckgres container-image-worker-cd.yml, canary, then flip the default.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR

Rebuilds the cred-refresh-write-retry patch set against v1.5.5 so a 1.5.5
worker row can be added to duckgres. Without a 1.5.5 build of this
extension the fleet cannot move off 1.5.3: httpfs carries mid-statement
S3 credential recovery on ExpiredToken, which is not optional in prod.

MainDistributionPipeline reads .github/duckdb-version and passes it to
the distribution workflow, so this one line is what selects the DuckDB
the extension is compiled against. No source changes are expected -
DuckLake made the identical 1.5.3 -> 1.5.5 jump with zero source changes
(clean build, 16083 assertions) - but CI on this PR is what actually
proves it for httpfs.

Motivation for 1.5.5 specifically: it is the first release containing
eaf8af373d, which fixes swapped min/max for multi-row-group 128-bit
decimals in RETURN_STATS. DuckLake feeds RETURN_STATS straight into
ducklake_file_column_stats, and those stats drive file pruning, so on
<=1.5.4 a DECIMAL(p>=19) column in a multi-row-group file can prune away
files that do contain matches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mntu6YhMuqNBFD9snXqvBR
@fuziontech
fuziontech merged commit 575da0b into posthog/v1.5.3 Jul 29, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant