fix(data-imports): map DeltaLake object-store access errors to an actionable message - #73490
Conversation
|
Hey @Gilbert09! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
There was a problem hiding this comment.
Small, contained fix adding one error-string mapping plus a test, authored by the owning team; no risky territory touched.
- Author wrote 0% of the modified lines and has 28 merged PRs in these paths (familiarity MODERATE).
- 👍 on the PR from hex-security-app[bot].
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 3L, 1F substantive, 20L/2F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1a-trivial (20L, 2F, single-area, fix) |
| stamphog 2.0.0b3 | .stamphog/policy.yml @ 591ed2d · reviewed head 4348ef2 |
|
😎 Merged successfully - details. |
|
/trunk merge |
|
/trunk merge |
1 similar comment
|
/trunk merge |
…ionable message Error tracking surfaced a generic "Could not read the files from your storage bucket" exception during a MySQL sync's schema-validation step, with an underlying DeltaLake-kernel object-store access error that `ExtractErrors` had no entry for. `get_columns()`'s fallback error handler (`_safe_expose_ch_error`) matches known ClickHouse error substrings against `ExtractErrors` to give a specific, actionable message, falling back to a generic one otherwise. `ExtractErrors` only covered ClickHouse's native S3 client error vocabulary, not the different wording used by ClickHouse's DeltaLake kernel (the `object_store` Rust crate) — the format used by every warehouse_sources synced table. Added a matching entry so this error class surfaces the same actionable message as the equivalent native-S3 case. Generated-By: PostHog Code Task-Id: 5407d15d-430b-4648-b6ee-483880dcbfae
0f412be to
00a4c16
Compare
New commits pushed (delta classified non_linear_history) — stamphog approval dismissed; re-review running automatically.
|
/trunk merge |
There was a problem hiding this comment.
Trivial, well-tested error-message mapping fix by an owning-team author; no risky territory touched.
- Author wrote 0% of the modified lines and has 31 merged PRs in these paths (familiarity MODERATE).
- 👍 on the PR from hex-security-app[bot].
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 3L, 1F substantive, 20L/2F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1a-trivial (20L, 2F, single-area, fix) |
| stamphog 2.0.0b3 | .stamphog/policy.yml @ 8bba179 · reviewed head 00a4c16 |
Problem
Error tracking surfaced a generic exception during a MySQL sync's schema-validation step: issue.
The call path:
import_data_activity_sync->pipeline.run->validate_schema_and_update_table->DataWarehouseTable.get_columns()(products/warehouse_sources/backend/models/table.py), which issues a ClickHouseDESCRIBE TABLEagainst the synced Delta table. That query failed with a genuine object-store access error from ClickHouse's DeltaLake kernel:get_columns()'s fallback error handler (_safe_expose_ch_error) matches known ClickHouse error substrings inExtractErrorsto surface a specific, actionable message, falling back to a generic one otherwise.ExtractErrorsalready has an entry for this exact situation from ClickHouse's native S3 client ("Access Denied: while reading key:"), but every warehouse_sources synced table reads its schema through ClickHouse's DeltaLake kernel instead (format="DeltaS3Wrapper"), which uses different wording from the underlying Rustobject_storecrate. That vocabulary wasn't represented inExtractErrorsat all, so this whole error class — not just this one occurrence — fell through to the generic fallback regardless of the actual cause.This occurred 3 times in a ~36 second window during a single sync attempt and never recurred, consistent with a transient storage blip rather than a persistent credential problem — retries (5 attempts with backoff) are already correct behavior here and I didn't change the retry policy. The gap is purely in the message a user sees when this class of error does surface.
Changes
ExtractErrorsentry matching the DeltaLake-kernelobject_storecrate's permission-error wording ("The operation lacked the necessary privileges to complete"), mapping it to the same actionable message already used for the equivalent native-ClickHouse-S3 case.How did you test this code?
Added
test_delta_kernel_permission_error_gets_actionable_messageto the existingTestSafeExposeChErrorclass intest_table.py, using the real captured error text (with identifiers redacted) — this is exactly the shape of the bug this PR fixes: without theExtractErrorsentry, this case falls through to the generic message like every other unmatched error.Ran:
uv run pytest products/warehouse_sources/backend/tests/test_table.py -k SafeExposeChError— 3 passeduv run mypy --cache-fine-grained products/warehouse_sources/backend/models/table.py products/warehouse_sources/backend/tests/test_table.py— clean./bin/hogli ci:preflight --fix— 0 failuresAutomatic notifications
Docs update
N/A — internal error-message mapping, no user-facing API/config change.
🤖 Agent context
Autonomy: Fully autonomous
Triaged directly from the linked error-tracking issue via PostHog's error-tracking MCP tools (
query-error-tracking-issue,query-error-tracking-issue-events), which surfaced the full stack/exception-chain and confirmed the failure originates in shared schema-introspection code (table.py), not the MySQL source connector. Traced the ClickHouse error-wrapping path (posthog/errors.py'swrap_clickhouse_query_error, dynamically-builtCHQueryErrorDeltaKernelErrorfor ClickHouse error code 742) to confirm the DeltaLake-kernel error vocabulary is structurally different from the native-S3 vocabularyExtractErrorsalready covers.Checked for duplicate open PRs by exception type, message phrase, and module path, and scanned the maintainer's own open PR queue. Found #73454 ("recognize DeltaError as a transient object-store error"), which fixes a related-but-distinct gap in
delta_table_helper.py's best-effort pre-write maintenance classifier (a different code path, using the Pythondeltalakepackage rather than ClickHouse's SQL-level DeltaLake kernel) — it doesn't touch theExtractErrors/get_columnsgap this PR closes.Invoked
/writing-testsbefore adding the regression test.Created with PostHog Code