GH-48314: [Python] Compat with pandas 3.0 changed default datetime unit #48319
Conversation
jorisvandenbossche
requested review from
AlenkaF,
raulcd and
rok
as code owners
|
|
github-actions
bot
added
Component: Python
awaiting committer review
|
@github-actions crossbow submit test-conda-python-3.13-pandas-nightly-numpy-nightly test-conda-python-3.13-pandas-upstream_devel-numpy-nightly |
python/pyarrow/tests/test_pandas.py
Outdated
| [(1, 'a', 2.0), (2, 'b', 3.0), (3, 'c', 4.0)], | ||
| columns=pd.date_range( | ||
| start='2017-01-01', periods=3, tz='America/New_York' | ||
| start='2017-01-01', periods=3, tz='America/New_York', unit='us' |
There was a problem hiding this comment.
Do we want to parametrize this test on datetime unit or is it superfluous?
|
Revision: f0afc77 Submitted crossbow builds: ursacomputing/crossbow @ actions-bfea497ea7
|
|
Unrelated, but do we want to remove compatibility with very old Pandas versions at some point? Pandas 1.3.4 was released in 2021 and only provides wheels for Python 3.10 and earlier. |
|
@AlenkaF @pitrou @jorisvandenbossche do we want to fix this for the release? I am happy to follow this up if @jorisvandenbossche does not have the time. We have several jobs failing on CI due to this issue even though none of them seem release blocking as jobs required to release (wheels, etc) are successful. |
Yes, this sounds desirable. |
|
This seems to be a blocker now as it is failing on the free-threaded Windows wheels: I'll try to fix it. |
raulcd
force-pushed
the
pyarrow-compat-pandas-3-unit
branch
from
f0afc77 to
af64016
Compare
|
@github-actions crossbow submit -g python -g wheel |
|
Revision: 868e6af Submitted crossbow builds: ursacomputing/crossbow @ actions-c5e90a9ecd
|
I've created the following issue to discuss bumping pandas support here: |
|
I've fixed the tests so they are compatible with Pandas 1.3.4 and have added the fixture for time unit as requested on the review. Current CI failures have been fixed. |
3 participants
Rationale for this change
pandas 3.0 changes the default datetime/timedelta resolution from nanoseconds to microseconds. We already had mostly accounted for that in previous PRs, but pandas made this change the last days in a few additional places (eg
pd.date_range), uncovering some more issuesWhat changes are included in this PR?
Are these changes tested?
Yes
Are there any user-facing changes?
No