Repository navigation
[Enh]: support Map datatype #3525
Description
We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?
I would really like to to natively handle Map columns in my Narwhals code in Metaxy.
Right now I had to go through a few hops [1, 2] to get it working for Polars, PyArrow and Ibis backed frames.
Please describe the purpose of the new feature or describe the problem to solve.
Currently, Narwhals does not support the Arrow Map datatype, and for a good reason: neither does Polars.
However, all the other tools built around Arrow do support it: Ibis, DuckDB, ClickHouse, and storage formats such as Parquet, DeltaLake, and Apache Iceberg.
Therefore, this is a core datatype in the Arrow ecosystem, and we should support it as well - at least in some (experimental) form.
Suggest a solution if possible.
Initially we can make a separate narwhals-map plugin and document it as experimental.
For Polars we can either use https://github.com/hafaio/polars-map, or fail at runtime (I would prefer the former).
For all the other datatypes we can use their native Map support.
If you have tried alternatives, please describe them below.
No response
Activity
MarcoGorelli commentedon Mar 27, 2026
thanks for the issue
yup, on board with this (not yet sure how the details would work, but open to exploring it)
danielgafni commentedon Mar 27, 2026
Alright, I can start working on it in my own repo? Or will you create it?
MarcoGorelli commentedon Mar 27, 2026
sure, feel free to do so! we can then move it over to the narwhals org if/when ready
danielgafni commentedon Apr 2, 2026
I have a draft at https://github.com/danielgafni/narwhals-map and it even works in my Metaxy project!
Very minimal for now, supports Arrow, Ibis, Polars (via
polars-map, fyi @erikbrinkman) and only one operation:.get:)@MarcoGorelli can we add some minimal hooks to Narwhals in order to expose the
.mapnamespace andMaptype?MarcoGorelli commentedon Apr 3, 2026
thanks!
discussed today, and yup, open to it! the
Maptype could even be fully upstreamedFBruzzesi commentedon Nov 4, 2026
Closing in favor of #4128 which is tracking each namespace method