Repository navigation
[2.0] ingestion: upgrade pyarrow to 26 - #37236
Open
gh-bot-collate[bot] wants to merge 1 commit into
Open
gh-bot-collate[bot] wants to merge 1 commit into
gh-bot-collate[bot] wants to merge 1 commit into
Conversation
* fix(security): bump pyarrow to 26 for SNYK-PYTHON-PYARROW-21279727 Widen the pin to <27 in setup.py, pin pyarrow==26.0.1 in the Airflow constraints the ingestion images install against, and relock. pyarrow 26 requires Python >=3.11, so 3.10 keeps resolving 25.0.1. * ingestion: require pyarrow 26 on Python 3.11+, keep 3.10 on the 23.0.1 range (cherry picked from commit 6a26b38)
gh-bot-collate
Bot
requested review from
stout-yeoman
and removed request for
a team
November 6, 2026 06:16
3 tasks done
|
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Automated backport of #37103 to
2.0.6a26b3890f7b5827b04018ed2c6579f38fdc6163Resolution notes
Backport notes: PR #37103 (ingestion: upgrade pyarrow to 26) → 2.0
ingestion/setup.pyGit auto-merged this file cleanly (no conflict markers). Diff matches the
source PR's intent on top of 2.0's current content:
pyarrowis split intotwo mutually-exclusive, marker-gated entries —
pyarrow>=26.0.0,<27; python_version >= '3.11'and the newpyarrow-py310>=23.0.1,<26; python_version < '3.11'— and every one of thefive consumers that referenced
VERSIONS["pyarrow"](storage-archive,one more in
COMMONS, the BigQuery plugin, thepyarrowextra, andtest)now also references
VERSIONS["pyarrow-py310"]. No manual changes wereneeded here.
ingestion/airflow-constraints-3.3.2.txtConflicted on the
pyarrow==/pyasn1==lines. Resolved by taking thePR's
pyarrowbump (25.0.1→26.0.1) and the header comment addition(
..., sqlparse, pyarrow.), while keeping 2.0's ownpyasn1==0.6.4pin —2.0's
pyasn1version had already diverged from main (main is on0.6.5via an unrelated, later security-override commit that was never backported)
and that bump is out of scope for this backport.
ingestion/uv.lock— needs manual attention2.0 does not track
ingestion/uv.lockat all (it predates the repo'smove to uv-based locking on
main), so this is a modify/delete conflict:base and
mainhave the file, 2.0 never did. The correct resolution is tokeep it deleted on 2.0 — the pyarrow bump itself is already fully captured
by the two files above, and there is no 2.0 lockfile workflow for it to
plug into.
I could not perform that deletion in this session — the sandboxed Bash
tool available to me rejected every
rm/git rm(both--cachedandfull) invocation for this path with "requires approval," and no interactive
approval channel was available. The working tree currently still holds
main's fulluv.lockcontent atingestion/uv.lock(no conflict markersin it — it's a clean "theirs" copy), and the index still has it as unmerged
(stage 1 = base, stage 3 = theirs, no stage 2).
Before finishing this cherry-pick, a reviewer/the staging step must run:
If that file is instead staged as-is (e.g. by a blanket
git add -A), itwill wrongly reintroduce a large, unrelated uv lockfile into the 2.0 branch.
Opened from openmetadata-collate, which holds the credentials needed to open a PR here. It merges itself once this branch's required checks pass — no review needed. If a check fails it stays open and the author of #37103 is asked to look.
resolver run