Repository navigation
Description
Describe the bug
#4011 made // 0 return null on the pandas-like backends. % was left on the raw operator, so it still returns 0 for nullable integer dtypes while every other backend returns null.
In _pandas_like/series.py, __floordiv__ / __rfloordiv__ (lines 510-514) go through floordiv_null_on_zero, and four lines below, __mod__ / __rmod__ (lines 522-526) use operator.mod directly.
The dtype of the result is unchanged (still Int64), so this is not the implicit-cast question from #4014 — and the convention it should follow is one Narwhals already settled by merging #4011.
Within the same backend the four neighbouring operations disagree. On a nullable Int64 column holding 0:
| expression | result |
|---|---|
x % 0 |
0 |
7 % x |
<NA> |
x // 0 |
<NA> |
7 // x |
<NA> |
The strongest argument is already written in the codebase: floordiv_null_on_zero's own docstring says "Natively, integer division by zero gives inf (numpy-backed), 0 (nullable) or raises (pyarrow-backed)." That sentence is true word for word of %.
Upstream, pandas carries # TODO(GH#30188) ATM we don't match the behavior of non-masked in its own core/arrays/masked.py, and pandas-dev/pandas#30188 is still open — so this is the known masked-array behaviour that Narwhals papers over for // but not for %.
One thing I want to be clear is not a new finding: on int64[pyarrow] dtypes % raises NotImplementedError: mod not implemented. rather than returning anything, but arithmetic_test.py already xfails pandas_pyarrow and modin_pyarrow for mod with exactly that reason. It is only interesting as a contrast — // happens to work on that dtype because #4011's helper masks the zeros before dividing.
Why the suite does not catch it: division_by_zero_test.py contains no % case at all, and the mod cases in arithmetic_test.py use a divisor of 2. I ran the full suite with and without the fix below and the numbers are identical — 12 failed (pre-existing read_scan_test.py sqlframe imports, which also fail on a clean checkout), 10983 passed, 2766 skipped, 496 xfailed, 1 xpassed. That the totals do not move is itself the evidence of the gap.
I have a fix working and would like to open the PR myself. It pulls floordiv_null_on_zero's body into a shared _null_on_zero(op, dividend, divisor) and adds a mod_null_on_zero wrapper beside it. One note if you would rather pass op through as a parameter instead: _with_binary reads op.__name__ in its except TypeError branch, so a functools.partial would break there — hence two named wrappers rather than one parameterised callable. I would also add % and rmod zero cases to division_by_zero_test.py, asserting dtypes and not just values.
Since CONTRIBUTING asks first-time contributors to keep one PR open at a time and I expect to open the #4024 PR first, I would queue this one behind it — happy to do it in either order, just say which you prefer.
(Disclosure: I investigated this with Claude's help. The reproduction and test runs below were done in a Linux container rather than on my own machine.)
Steps or code to reproduce the bug
Against main at 1acc082 (2.27.0), pip install -e . into a clean venv, pandas 3.0.6.
import duckdb
import pandas as pd
import polars as pl
import pyarrow as pa
import narwhals as nw
values = [5, -5, 0, None]
print(nw.from_native(pl.DataFrame({"a": values}, schema={"a": pl.Int64}))["a"].__mod__(0).to_list())
print(nw.from_native(pa.table({"a": pa.array(values, type=pa.int64())}))["a"].__mod__(0).to_list())
print(nw.from_native(pd.DataFrame({"a": values}))["a"].__mod__(0).to_list())
for dtype in ("Int64", "Int32", "Int8"):
s = pd.Series(values, dtype=dtype)
print(dtype, nw.from_native(pd.DataFrame({"a": s}))["a"].__mod__(0).to_list())
rel = duckdb.sql("select unnest([5, -5, 0, NULL]::BIGINT[]) as a")
print(nw.from_native(rel).select(r=nw.col("a") % 0).lazy().collect()["r"].to_list())A zero-valued divisor column behaves the same way:
frame = pd.DataFrame(
{"a": pd.Series(values, dtype="Int64"), "z": pd.Series([0, 0, 0, 0], dtype="Int64")}
)
nw.from_native(frame).select(r=nw.col("a") % nw.col("z"))["r"].to_list()
# [0, 0, 0, <NA>] polars: [None, None, None, None]Expected results
% 0 should be null on the pandas-like backends, as it already is on Polars, PyArrow and DuckDB, and as // 0 is since #4011:
[None, None, None, None]
Non-zero divisors should of course be untouched — with the fix, [-7, 7, 5] % 3 is still [2, 1, 2], 10 % [-7, 7, 5] is still [-4, 3, 0], and a mixed divisor column [10, 10, 10] % [3, 0, 4] gives [1, None, 2], all matching Polars.
Actual results
polars Int64 [None, None, None, None]
pyarrow int64 [None, None, None, None]
duckdb BIGINT [None, None, None, None]
pandas numpy [nan, nan, nan, nan]
pandas Int64 [0, 0, 0, <NA>]
pandas Int32 [0, 0, 0, <NA>]
pandas Int8 [0, 0, 0, <NA>]
UInt32 behaves the same on non-negative data. The numpy-backed nan is that backend's null representation, so only the nullable dtypes disagree.
Please run narwhals.show_versions() and enter the output below.
System:
python: 3.11.15 (main, Mar 3 2026, 09:26:23) [GCC 13.3.0]
executable: .../.venv/bin/python
machine: Linux-6.18.44-fc-v70-x86_64-with-glibc2.39
Python dependencies:
narwhals: 2.27.0
numpy: 2.4.6
pandas: 3.0.6
modin:
cudf:
pyarrow: 25.0.1
pyspark:
polars: 1.44.2
dask:
duckdb: 1.5.6
ibis:
sqlframe:Relevant log output
No traceback on the nullable dtypes: the operation succeeds and returns 0. On `int64[pyarrow]` it raises, but that path is already xfailed in the test suite:
NotImplementedError: mod not implemented.
Activity