Fixes #31790: [1.13] upgrade collate-sqllineage and require Python 3.10 - #31896
Conversation
2.1.4 produces no column lineage for UPDATE and MERGE. Table lineage still resolves, so the gap is quiet: the graph looks populated while every column edge on those statements is missing. 2.1.7 also clears one of the sqlparse>=0.6.0 ceilings tracked in the issue. Update the lineage tests to assert the correct lineage instead of the empty result that encoded the old limitation. Two shapes are still unresolved upstream and stay marked xfail with the correct lineage asserted.
…vention Greptile flagged that whole-test xfails also cover the table-lineage and parser checks, so an unrelated regression reports as XFAIL instead of failing. Use the per-parser flags the rest of the file already uses. merge_06 asserts the one edge all three parsers resolve, with a comment on why the two window ORDER BY edges are missing. merge_10 goes back to an empty expectation with SqlGlot and SqlParse disabled and their specific errors named, plus skip_graph_check on the table assertion since SqlGlot and SqlFluff build different internal shapes for the recursion.
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
github-actions
Bot
added
Ingestion
safe to test
Code Review ✅ ApprovedBackports the sqlparse CVE remediation by upgrading collate-sqllineage to 2.1.7, requiring Python 3.10+, and aligning window lineage expectations. No issues found. OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
|
The Python checkstyle failed. Please run You can install the pre-commit hooks with |
3 participants
Fixes #31790.\n\nBackports the sqlparse CVE remediation from #31885 to 1.13 by upgrading collate-sqllineage from 2.1.4 to 2.1.7 and removing the temporary Docker-level sqlparse override. The resolver now installs sqlparse 0.6.0 through the normal dependency graph.\n\nThis raises the ingestion package's supported Python floor from 3.9 to 3.10 because both collate-sqllineage 2.1.7 and sqlparse 0.6.0 require Python 3.10 or newer. It also aligns the LAG/LEAD lineage expectation with the behavior already accepted on newer branches.