Skip to content

Fix ObjectStoragePath to exclude conn_id from storage options passed to fsspec#62701

Merged
kaxil merged 4 commits intoapache:mainfrom
jason810496:fix/fix-object-storage-path-constructing-issue
Mar 3, 2026
Merged

Fix ObjectStoragePath to exclude conn_id from storage options passed to fsspec#62701
kaxil merged 4 commits intoapache:mainfrom
jason810496:fix/fix-object-storage-path-constructing-issue

Conversation

@jason810496
Copy link
Copy Markdown
Member

@jason810496 jason810496 commented Mar 2, 2026

closes: #62691
related: #60519

Why

The fsspec doesn't understand conn_id, so we need to exclude from the storage_options.

Follow-up

Add an integration test to cover XCom ObjectStore and prevent regressions.

@jason810496 jason810496 self-assigned this Mar 3, 2026
Copy link
Copy Markdown
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one qn

@kaxil kaxil merged commit d42623e into apache:main Mar 3, 2026
98 of 99 checks passed
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
…sed to fsspec (apache#62701)

* Fix ObjectStoragePath to exclude conn_id from storage options passed to fsspec

* Add conn_id property to avoid re-initialization

* Fix div/ joinpath operations by overriding _from_upath method

* Fix mypy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Task failing with AioSession.__init__() got an unexpected keyword argument 'conn_id' with xcombackend

4 participants