Add commit-loop instrumentation and ducklake_commit_stats()

[fuziontech]

Summary

Metrics-only instrumentation of the commit retry loop — the first step of the collision-observability plan. Production context: megaduck's metadata store shows a 12.4% transaction rollback rate while every existing metric reads ~0, because the extension's internal retry loop absorbs collisions invisibly (duckgres only sees conflicts that escape the loop, and millpond exports nothing). This PR makes the loop observable from inside.

No behavior changes: RetryOnError now delegates to the new ClassifyCommitError helper using the exact same substring checks in the same order (verified by the passing concurrent/transaction suites — 537 assertions); all counters are additive.

What's added

• DuckLakeCommitStatsRegistry — process-global, per-catalog counters (std::atomic, mutex-guarded map): attempts, successes, retries_exhausted, nonretryable_errors, backoff_ms, total_commit_ms (RAII timer, all exit paths), and per-cause conflicts.
• ducklake_commit_stats() table function → (catalog VARCHAR, stat VARCHAR, value BIGINT), one row per (catalog, stat). Stat names: attempts, successes, retries_exhausted, nonretryable_errors, backoff_ms, total_commit_ms, conflicts.{primary_key,unique,conflict,concurrent}. This schema is a contract consumed by feat(duckdbservice): export ducklake commit-loop stats from ducklake_commit_stats() duckgres#1012 (worker-side Prometheus re-export, already merged-ready and dormant until an extension binary ships this function).
• Catalog labels: client-side commits use the attached catalog name; server-side ducklake_commit() uses the metadata schema name; fallback default.

Known edge case

An error that is retryable-by-cause but aborts because the conflict-check phase failed (can_retry == false before max retries) counts toward attempts and its conflict bucket but neither retries_exhausted nor nonretryable_errors — it fits neither definition. Documented rather than invented a fifth counter; can revisit if it shows up in practice.

Test plan

• GEN=ninja make release clean
• New sqllogictest test/sql/functions/ducklake_commit_stats.test (12 assertions): counters after attach/create/insert, zero rows with no ducklake attached
• Regression: ducklake_basic (39), functions/* (96), concurrent/* (277), transaction/* (260, 1 skip needing httpfs) — all pass
• clang-format 11.0.1 (the pinned version) on touched files

Stacked on #34 (branch includes the consolidation merge; diff vs posthog/v1.5.5 shows only this PR's changes). Follow-ups: millpond poller (separate repo — the writer where megaduck's numbers will actually appear), then tag v1.0-posthog.7 + duckgres DUCKLAKE_EXTENSION_TAG bump to ship it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LUnQHp46rmLKdupmDUr7kG

[fuziontech]

Adversarial self-review (requested by James). Verified against the full diff and a local build of the branch.

Verified correct:

• RetryOnError semantics are provably unchanged: ClassifyCommitError uses the identical lowercased substring checks in the identical order (primary key → unique → conflict → concurrent), and IsRetryableCause accepts exactly the same set. The split of the original combined primary key || unique check into two ordered buckets cannot change the boolean outcome.
• Counter placement: attempts per iteration; backoff_ms incremented immediately before the actual sleep_for inside the same #ifndef DUCKDB_NO_THREADS guard (no phantom backoff when threads are disabled); total_commit_ms via RAII covers every exit path including throws; RecordConflict(NONE) is a no-op so non-retryable errors don't pollute the conflict buckets.
• Table function paginates via offset + STANDARD_VECTOR_SIZE, so >2048 rows (200+ catalogs) works; returns zero rows with nothing attached.
• Registry: unique_ptr map values keep handed-out references stable across rehashes; atomics make the hot path lock-free (lock only on first-touch of a catalog and in Snapshot()).

Notes for reviewers (known, accepted):

1. Uncounted abort bucket: an error that is retryable-by-cause but aborts because can_retry == false (conflict-check-phase failure) before max retries increments attempts + a conflict bucket but neither retries_exhausted nor nonretryable_errors. Consequence: failed commits ≠ exhausted + nonretryable. If reconciliation against xact_rollback needs exactness, a fifth stat (aborted_conflict_check) can be added later without breaking the contract — the duckgres consumer (PostHog/duckgres#1012) deliberately passes unknown conflicts.* causes through and drops unknown non-conflict stats, so it's forward-compatible.
2. Snapshot() "consistent" is per-counter, not cross-counter: it holds the registry lock but concurrent commits can advance counters between individual load()s, so a snapshot can momentarily show e.g. attempts incremented without its successes. Harmless for monotonic counters + rate() math; just don't build invariant checks on exact cross-stat equality at a single instant (the sqllogictest correctly uses >=).
3. Label collision: two DuckDB instances in one process attaching different lakes under the same alias merge into one label (the registry is process-global by design). Not a concern for duckgres/millpond topologies (one lake per process today).

CI note: the heavy suites were also run locally pre-push (basic 39, functions 96, concurrent 277, transaction 260) — all pass.

[fuziontech]

CI caught a real config-coverage gap in the test (not in the instrumentation): the "Postgres/SQLite" lane also runs the quack RPC server config, where data-only commits execute their commit loop in the server process — so the client-side registry legitimately doesn't count the INSERT, and attempts >= 2 failed. That's the server-side commit path working exactly as documented (server-side commits are labelled by metadata schema name in the server process's own registry).

Fixed in 87f29f2 by making the test config-agnostic: thresholds anchor on DDL commits (always client-side, even on quack) with a NOTE comment explaining why, and the monotonicity probe now uses a second CREATE TABLE instead of an INSERT. Verified locally on the duckdb config; CI re-running.