fix(security): airflow 3.3.1 (CVE-2026-67587 / CVE-2026-54183) and expat 2.8.3 (CVE-2026-72522) - #31890
Merged
Merged
Conversation
The image reports expat 2.8.2 for CVE-2026-72522 (DSA-6446-1: out-of-bounds read and resultant infinite loop in libexpat's *_toUtf16 surrogate handling); trixie-security has 2.8.3-1~deb13u1. expat is not in the python:3.12-slim-trixie base at all -- it comes in transitively from the top-of-file apt install -- and that layer is exactly why the image is stuck at 2.8.2: its cache key never changes, so it keeps resolving against the Debian index that was current when it was first built. Adding expat there would freeze the fix the same way. It goes in the late root layer instead, below the COPY and pip layers, where the index is re-read on every build -- the same reasoning that put util-linux there. expat rides in the existing dpkg source-package query rather than a second RUN, so one apt call covers both source packages and the next OS CVE is a one-line edit. The package set stays computed from dpkg rather than hand-listed for the reason already documented there: scanners report each binary of a source package separately, so a hand-written list silently leaves one behind. util-linux keeps its non-empty assert and expat deliberately does not get one. util-linux is Essential, so an empty query there means dpkg-query misbehaved and the build must stop; expat is transitive, so an empty query is a legitimate image with nothing to patch and failing on it would break the build for no reason. Asserting one of the two still covers the hazard the check exists for: `apt-get install --only-upgrade` with no package arguments exits 0, so an all-empty query would give a green build that shipped the vulnerable packages. Verified in a real build layer on python:3.12-slim-trixie: libexpat1 2.7.1-2 -> 2.8.3-1~deb13u1 and all nine util-linux binaries -> 2.41.5-0+deb13u1, exit 0. The expat-absent path was exercised on the bare base and also exits 0, upgrading util-linux alone. ingestion/Dockerfile gets no equivalent change because there is nothing to upgrade to, not because it is clean: apache/airflow:3.3.0-python3.12 is bookworm, Debian still marks bookworm and bookworm-security vulnerable, and the image already carries 2.5.0-1+deb12u2 -- the newest build either suite offers. It needs revisiting when Debian ships a bookworm fix.
…CVE-2026-54183 CVE-2026-67587 (High) -- the Task SDK rebuilt a Serde `Callback` by re-running its constructor, which imports the module named by the stored callback path. `SyncCallback` is an Airflow class, so it passes the default `allowed_deserialization_classes` allow-list and tightening that setting does not help. A Dag author controls a task instance's `next_kwargs`, so they can get an arbitrary module imported inside the scheduler process when the `awaiting_input` timeout sweep deserializes that value. No non-default configuration is required. CVE-2026-54183 (Medium) -- the secrets masker's recursion-depth limit did not descend into values nested inside a list, tuple or set, so a Variable holding a deeply nested sensitive value rendered unmasked in the Variables UI. Anyone who can see the Variable in the UI can already read it through the Variables REST API, so this is a shoulder-surfing defense rather than a disclosure boundary -- hence Medium. Both are fixed in 3.3.1 and nothing in the 3.3.0 line is clean. The pin, both base image tags, the vendored constraints file and the integration-test image move together, as they did for 3.2.2 -> 3.3.0 in #31338. The constraints file is a straight re-download this time. It carried two hand-patches marked "keep on regeneration" -- impyla and thrift at 0.24.0 for CVE-2026-66053 / CVE-2026-41608 / CVE-2026-48586, against upstream constraints-3.3.0 shipping impyla==0.22.0 and thrift==0.16.0 -- and upstream constraints-3.3.1 now ships both at 0.24.0 natively. Diffing the vendored 3.3.0 file against upstream constraints-3.3.0 confirms those two blocks were the only divergence, and the new file is byte-identical to upstream constraints-3.3.1 outside comments. The comments are kept, reworded to say the pin is now upstream-native, so a future regeneration off a branch that regressed either version still gets caught. Two comments that name a constraint were checked rather than blind-renumbered: - tests/integration/airflow/Dockerfile explains the constraints are deliberately not applied because they pin chardet against openmetadata-ingestion's chardet==4.0.0. Still true; the version moved 6.0.0.post1 -> 7.5.1, so the number is updated. - Dockerfile.ci's universal-pathlib workaround cites Airflow's >=0.3.8 floor. apache-airflow-core 3.3.1 still declares `universal-pathlib>=0.3.8` and constraints still pin 0.3.10, so only the Airflow version in the prose changes. Resolution verified with `uv pip compile --python-version 3.12 --extra airflow`: apache-airflow==3.3.1 co-installs with every transitive floor pin in the airflow extra (providers-http, -opensearch, -elasticsearch, tornado, Werkzeug, starlette, python-multipart) with no conflict. sqlparse resolves to 0.5.4 there via collate-sqllineage, unchanged by this PR -- the release images force 0.6.0 in ingestion/operators/docker/Dockerfile, which does not consume this constraints file. Folded in: drop the stale croniter<3 ceiling from the dagster extra `uv pip install ingestion[all]` + `[test]` in the CI test environment fails to resolve against apache-airflow 3.3.1: Because apache-airflow-core>=3.3.1 depends on croniter>=6.2.2 [...] and because openmetadata-ingestion[test]==2.0.0.0.dev0 depends on apache-airflow==3.3.1 and croniter<3, we can conclude that [...] requirements are unsatisfiable. apache-airflow-core raised its croniter floor from >=2.0.2 to >=6.2.2 between 3.3.0 and 3.3.1, so the ceiling that used to co-exist with airflow now excludes it. The ceiling is dead weight and predates this conflict -- it has been in the dagster extra since #6416 and survived the #15679 dependency cleanup: - dagster 1.13.18 (what `dagster_graphql>=1.8.0` resolves to) declares no croniter dependency at all; resolving dagster_graphql alone produces no croniter. - Nothing under ingestion/ imports croniter. The only importer in the repo is openmetadata-airflow-apis, a separate distribution that does not declare it and picks it up transitively from airflow. - That importer is already running croniter 6.2.x: constraints-3.3.0 pins croniter==6.2.2 and the airflow images install under it, so 6.x compatibility for `croniter.is_valid` / `get_prev` is established by the shipped image, not assumed here. The call site is also behind an `is_airflow_3_or_higher()` guard and unreachable on Airflow 3. Only the CI test environment resolved croniter 2.x, because it installs from the extras without the constraints file. Removing the ceiling makes it match the images. Verified with `uv pip compile --extra all --extra test` on both 3.10 and 3.12: resolves to apache-airflow==3.3.1, croniter==6.2.4, dagster==1.13.18 with no conflict. This was the only pin in the tree that collided with the airflow bump.
Contributor
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
This was referenced Aug 21, 2026
Contributor
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
Contributor
✅ Playwright Results — workflow succeededValidated commit ✅ 615 passed · ❌ 0 failed · 🟡 1 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky PerformanceBlocking targets: ✅ met · Optimization targets: 🟡 in progress Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting. 🕒 Full workflow signal wall (to summary) 50m 48s ⏱️ Max setup 4m 52s · max shard execution 15m 42s · max shard-job elapsed before upload 21m 19s · reporting 8s 🌐 221.47 requests/attempt · 2.75 app boots/UI scenario · 18.22% common-shard skew Optimization targets still in progress:
🟡 1 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
Khairajani
added
safe to test
mohittilala
previously approved these changes
Aug 21, 2026
Khairajani
enabled auto-merge
August 21, 2026 12:55
|
IceS2
requested review from
a team,
akash-jain-10,
harshach and
tutte
as code owners
August 21, 2026 17:02
Code Review ✅ ApprovedBumps Apache Airflow to 3.3.1, pulls expat 2.8.3 into ingestion base images, and migrates the NumPy and Pydantic dependency stack to resolve multiple critical security vulnerabilities. No issues found. OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
Contributor
|
Unit tests failing due to stale spacey model on the workflow on main. This PR updates it. You can check the workflow against this branch update here: https://github.com/open-metadata/OpenMetadata/actions/runs/32522341633 |
pmbrull
approved these changes
Aug 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two OS/runtime security fixes for the ingestion images, one commit each:
fix(security): pull expat 2.8.3-1~deb13u1 into the ingestion-base image2.8.2->2.8.3-1~deb13u1iningestion/operators/docker/Dockerfile{,.ci}fix(security): bump apache-airflow 3.3.0 -> 3.3.1These were previously #31757 and #31861. They are combined here because both are security fixes for the same image family with disjoint file sets — the expat commit touches only
ingestion/operators/docker/, the airflow commit onlyingestion/Dockerfile{,.ci},setup.py, the constraints file and the integration-test harness. Nothing overlaps, so the two commits are reviewable independently. Both superseded PRs were green.Commit 1 — expat CVE-2026-72522
The image reports expat 2.8.2 for CVE-2026-72522 (DSA-6446-1 — out-of-bounds read and resultant infinite loop in libexpat's
*_toUtf16surrogate handling). trixie-security has2.8.3-1~deb13u1.expat is not in the
python:3.12-slim-trixiebase at all — it arrives transitively from the top-of-fileapt-get install. That layer is exactly why the image is stuck at 2.8.2: its cache key never changes, so it keeps resolving against the Debian index that was current when it was first built. Adding expat there would freeze the fix the same way. It goes in the late root layer instead, below the COPY and pip layers, where the index is re-read on every build — the same reasoning that put util-linux there.expat rides in the existing dpkg source-package query rather than a second
RUN, so one apt call covers both source packages and the next OS CVE is a one-line edit:The package set stays computed from dpkg rather than hand-listed, for the reason already documented there: scanners report each binary of a source package separately, so a hand-written list silently leaves one behind.
util-linux keeps its non-empty assert and expat deliberately does not get one. util-linux is Essential, so an empty query there means
dpkg-querymisbehaved and the build must stop. expat is transitive, so an empty query is a legitimate image with nothing to patch — asserting it would break the build for no reason. Asserting one of the two still covers the hazard the check exists for:apt-get install --only-upgradewith no package arguments exits 0, so an all-empty query would give a green build that shipped the vulnerable packages.Dockerfile.cicarries its own copy of this block and gets the same change — it is the base that Collate's Snyk scan builds from.Verification. Ran the
RUNblock verbatim in a real build layer onpython:3.12-slim-trixie:libexpat12.7.1-22.8.3-1~deb13u12.41-52.41.5-0+deb13u1Exit 0. The expat-absent path was exercised separately on the bare base and also exits 0, upgrading util-linux alone — that path is real, since the base ships no expat.
Out of scope, deliberately.
ingestion/Dockerfilegets no equivalent expat change because there is nothing to upgrade to, not because it is clean. Its base is bookworm; Debian's tracker still marks bookworm and bookworm-security vulnerable, and the image already carries2.5.0-1+deb12u2— the newest build either suite offers. Needs revisiting when Debian ships a bookworm fix. Re-checked against the tag this PR moves to:apache/airflow3.3.1 is stilldebian:bookworm-slim, byte-identical to 3.3.0 on every base-image line and with no trixie reference anywhere, so the airflow bump in commit 2 does not change this conclusion.Commit 2 — airflow CVE-2026-67587 / CVE-2026-54183
CVE-2026-67587 (High) — the Task SDK rebuilt a Serde
Callbackby re-running its constructor, which imports the module named by the stored callback path.SyncCallbackis an Airflow class, so it passes the defaultallowed_deserialization_classesallow-list and tightening that setting does not help. A Dag author controls a task instance'snext_kwargs, so they can get an arbitrary module imported inside the scheduler process when theawaiting_inputtimeout sweep deserializes that value. No non-default configuration is required.CVE-2026-54183 (Medium) — the secrets masker's recursion-depth limit did not descend into values nested inside a list, tuple or set, so a Variable holding a deeply nested sensitive value rendered unmasked in the Variables UI. Anyone who can see the Variable in the UI can already read it through the Variables REST API, so this is a shoulder-surfing defense rather than a disclosure boundary — hence Medium.
Both are fixed in 3.3.1 and nothing in the 3.3.0 line is clean. The pin, both base image tags, the vendored constraints file and the integration-test image move together, as they did for 3.2.2 -> 3.3.0 in #31338.
The constraints file is a straight re-download this time. It carried two hand-patches marked "keep on regeneration" — impyla and thrift at 0.24.0 for CVE-2026-66053 / CVE-2026-41608 / CVE-2026-48586, against upstream constraints-3.3.0 shipping
impyla==0.22.0andthrift==0.16.0— and upstream constraints-3.3.1 now ships both at 0.24.0 natively. Diffing the vendored 3.3.0 file against upstream constraints-3.3.0 confirms those two blocks were the only divergence, and the new file is byte-identical to upstream constraints-3.3.1 outside comments. The comments are kept, reworded to say the pin is now upstream-native, so a future regeneration off a branch that regressed either version still gets caught.Two comments that name a constraint were checked rather than blind-renumbered:
tests/integration/airflow/Dockerfileexplains the constraints are deliberately not applied because they pin chardet against openmetadata-ingestion'schardet==4.0.0. Still true; the version moved6.0.0.post1->7.5.1, so the number is updated.Dockerfile.ci's universal-pathlib workaround cites Airflow's>=0.3.8floor. apache-airflow-core 3.3.1 still declaresuniversal-pathlib>=0.3.8and constraints still pin0.3.10, so only the Airflow version in the prose changes.This commit also folds in the
croniter<3removal that was a second commit on #31861 — it is a prerequisite of the bump, not an unrelated change, which is whysetup.pyhas a hunk in the dagster extra. apache-airflow-core raised its croniter floor from>=2.0.2to>=6.2.2between 3.3.0 and 3.3.1, souv pip install ingestion[all]+[test]becomes unsatisfiable against the old ceiling. The ceiling was already dead weight: dagster 1.13.18 declares no croniter dependency at all, nothing underingestion/imports croniter, and the one importer in the repo (openmetadata-airflow-apis) already runs croniter 6.2.x via the constraints file. Only the CI test environment resolved croniter 2.x, because it installs from the extras without constraints. Full rationale is in the commit message.Verification.
uv pip compile --python-version 3.12 --extra airflow:apache-airflow==3.3.1co-installs with every transitive floor pin in the airflow extra (providers-http, -opensearch, -elasticsearch, tornado, Werkzeug, starlette, python-multipart) with no conflict.uv pip compile --extra all --extra teston both 3.10 and 3.12: resolves toapache-airflow==3.3.1,croniter==6.2.4,dagster==1.13.18, no conflict. sqlparse resolves to 0.5.4 via collate-sqllineage, unchanged by this PR — the release images force 0.6.0 iningestion/operators/docker/Dockerfile, which does not consume this constraints file.Related
🤖 Generated with Claude Code
Greptile Summary
The PR updates Airflow and Debian expat remediation paths while also migrating the ingestion dependency stack and associated Pydantic, NumPy, spaCy, PII, and test code.
Confidence Score: 5/5
The PR appears safe to merge because no blocking failure remains within the eligible follow-up review scope.
No blocking failure remains.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR Setup[setup.py dependency ranges] --> Images[Ingestion Docker images] Constraints[Airflow 3.3.1 constraints] --> Images Airflow[Airflow 3.3.1 base] --> Images Expat[Late apt source-package upgrade] --> OperatorImages[Operator images] Setup --> Runtime[Ingestion runtime] Runtime --> Models[Pydantic models] Runtime --> PII[PII and profiler paths] ModelWheel[spaCy model 3.8.0] --> CI[Python CI tests]Reviews (6): Last reviewed commit: "fix: filter invalid date recognizer resu..." | Re-trigger Greptile