Repository navigation
Add support for python-polars #1868
Description
Besides the use of pandas, it would be great to add support for the polars Python interface: https://github.com/pola-rs/polars.
Polars has very similar functionalities to Pandas, but uses a Rust backend that significantly speeds up operations compared to Pandas.
Reacted by Franky1, Lyubomir Georgiev, Gabor Schulz, 1 Line at a time, Daniel, Francesco Bruzzesi, baggiponte, impervius, Cameron Elliott, Konilo Zio and 6 more
Activity
ValueRaider commentedon Feb 24, 2024
I like what I hear about Polars, but great care will be needed to avoid upsetting user base.
Is there another popular package that migrated that we can learn from?
edoaltamura commentedon Feb 29, 2024
I can't think of projects that started in Pandas and then transitioned, and I do agree that care is needed when migrating foundational dependencies. This Medium article can give a flavor of what a 'translation' might look like.
In my view, simply translating can give some speed up over Pandas, but some more thinking is required to make the most of the lazy execution in Polars, which would give an extra boost.
gaborschulz commentedon Oct 16, 2024
Also agree with being careful about the migration. In my opinion, the most efficient way to do so would be to transition to using
narwhals(https://github.com/narwhals-dev/narwhals) as this is becoming the de-facto standard for supporting multiple dataframe libraries.FBruzzesi commentedon Mar 25, 2025
Hey everyone! I am one of developers contributing to Narwhals!
yfinanceis a very interesting project, thanks for mentioning Narwhals here @gaborschulz@edoaltamura this might have been true at the time, but in the last 6-8 months a few (big?) projects transitioned to a dataframe agnostic approach using Narwhals (see ecosystem).
I am not sure that Narwhals has all the functionality to let yfinance make such transition as well, but our approach has been very pragmatic in adding features whenever a use case presents itself, and we currently have a weekly release cycle 😁
If there is still interest in this, I can start taking a look at the internals in the next future 😇
gaborschulz commentedon Apr 2, 2025
I‘m happy to join and look into it together, @FBruzzesi
FBruzzesi commentedon Apr 3, 2025
Hi @gaborschulz - the main/only real blocker I see is that we don't support the equivalent of
polars.DataFrame.group_by_dynamic- which is the pandas equivalent ofresample.Aside that, from a design point of view, my understand is that everything revolves around the
Tickerclass (from a user point of view), and we could either:return_typein the initialization method, defaulting to pandas (we did something similar in plotly.express.datasets)ValueRaider commentedon Apr 3, 2025
Preferred option. There's plans to do config properly, but until then you can shove something into
data.py.FBruzzesi commentedon May 12, 2025
Tiny update on:
after the development of
dt.truncateand allowing expressions as group by keys, it should be enough to support the operations required in yfinance via:( frame .group_by(nw.col('date').dt.truncate('1mo')) .agg(nw.col('price').mean()) )(From narwhals#2532 comment)
Add opt-in polars output via pandas pipeline + boundary adapter