Skip to content

[Bug]: % 0 returns 0 on pandas nullable dtypes, where // 0 returns null after #4011 #4036

Open

Description

@lucaluo925

Describe the bug

#4011 made // 0 return null on the pandas-like backends. % was left on the raw operator, so it still returns 0 for nullable integer dtypes while every other backend returns null.

In _pandas_like/series.py, __floordiv__ / __rfloordiv__ (lines 510-514) go through floordiv_null_on_zero, and four lines below, __mod__ / __rmod__ (lines 522-526) use operator.mod directly.

The dtype of the result is unchanged (still Int64), so this is not the implicit-cast question from #4014 — and the convention it should follow is one Narwhals already settled by merging #4011.

Within the same backend the four neighbouring operations disagree. On a nullable Int64 column holding 0:

expression result
x % 0 0
7 % x <NA>
x // 0 <NA>
7 // x <NA>

The strongest argument is already written in the codebase: floordiv_null_on_zero's own docstring says "Natively, integer division by zero gives inf (numpy-backed), 0 (nullable) or raises (pyarrow-backed)." That sentence is true word for word of %.

Upstream, pandas carries # TODO(GH#30188) ATM we don't match the behavior of non-masked in its own core/arrays/masked.py, and pandas-dev/pandas#30188 is still open — so this is the known masked-array behaviour that Narwhals papers over for // but not for %.

One thing I want to be clear is not a new finding: on int64[pyarrow] dtypes % raises NotImplementedError: mod not implemented. rather than returning anything, but arithmetic_test.py already xfails pandas_pyarrow and modin_pyarrow for mod with exactly that reason. It is only interesting as a contrast — // happens to work on that dtype because #4011's helper masks the zeros before dividing.

Why the suite does not catch it: division_by_zero_test.py contains no % case at all, and the mod cases in arithmetic_test.py use a divisor of 2. I ran the full suite with and without the fix below and the numbers are identical — 12 failed (pre-existing read_scan_test.py sqlframe imports, which also fail on a clean checkout), 10983 passed, 2766 skipped, 496 xfailed, 1 xpassed. That the totals do not move is itself the evidence of the gap.

I have a fix working and would like to open the PR myself. It pulls floordiv_null_on_zero's body into a shared _null_on_zero(op, dividend, divisor) and adds a mod_null_on_zero wrapper beside it. One note if you would rather pass op through as a parameter instead: _with_binary reads op.__name__ in its except TypeError branch, so a functools.partial would break there — hence two named wrappers rather than one parameterised callable. I would also add % and rmod zero cases to division_by_zero_test.py, asserting dtypes and not just values.

Since CONTRIBUTING asks first-time contributors to keep one PR open at a time and I expect to open the #4024 PR first, I would queue this one behind it — happy to do it in either order, just say which you prefer.

(Disclosure: I investigated this with Claude's help. The reproduction and test runs below were done in a Linux container rather than on my own machine.)

Steps or code to reproduce the bug

Against main at 1acc082 (2.27.0), pip install -e . into a clean venv, pandas 3.0.6.

import duckdb
import pandas as pd
import polars as pl
import pyarrow as pa

import narwhals as nw

values = [5, -5, 0, None]

print(nw.from_native(pl.DataFrame({"a": values}, schema={"a": pl.Int64}))["a"].__mod__(0).to_list())
print(nw.from_native(pa.table({"a": pa.array(values, type=pa.int64())}))["a"].__mod__(0).to_list())
print(nw.from_native(pd.DataFrame({"a": values}))["a"].__mod__(0).to_list())
for dtype in ("Int64", "Int32", "Int8"):
    s = pd.Series(values, dtype=dtype)
    print(dtype, nw.from_native(pd.DataFrame({"a": s}))["a"].__mod__(0).to_list())

rel = duckdb.sql("select unnest([5, -5, 0, NULL]::BIGINT[]) as a")
print(nw.from_native(rel).select(r=nw.col("a") % 0).lazy().collect()["r"].to_list())

A zero-valued divisor column behaves the same way:

frame = pd.DataFrame(
    {"a": pd.Series(values, dtype="Int64"), "z": pd.Series([0, 0, 0, 0], dtype="Int64")}
)
nw.from_native(frame).select(r=nw.col("a") % nw.col("z"))["r"].to_list()
# [0, 0, 0, <NA>]      polars: [None, None, None, None]

Expected results

% 0 should be null on the pandas-like backends, as it already is on Polars, PyArrow and DuckDB, and as // 0 is since #4011:

[None, None, None, None]

Non-zero divisors should of course be untouched — with the fix, [-7, 7, 5] % 3 is still [2, 1, 2], 10 % [-7, 7, 5] is still [-4, 3, 0], and a mixed divisor column [10, 10, 10] % [3, 0, 4] gives [1, None, 2], all matching Polars.

Actual results

polars Int64      [None, None, None, None]
pyarrow int64     [None, None, None, None]
duckdb BIGINT     [None, None, None, None]
pandas numpy      [nan, nan, nan, nan]
pandas Int64      [0, 0, 0, <NA>]
pandas Int32      [0, 0, 0, <NA>]
pandas Int8       [0, 0, 0, <NA>]

UInt32 behaves the same on non-negative data. The numpy-backed nan is that backend's null representation, so only the nullable dtypes disagree.

Please run narwhals.show_versions() and enter the output below.

System:
    python: 3.11.15 (main, Mar  3 2026, 09:26:23) [GCC 13.3.0]
executable: .../.venv/bin/python
   machine: Linux-6.18.44-fc-v70-x86_64-with-glibc2.39

Python dependencies:
     narwhals: 2.27.0
        numpy: 2.4.6
       pandas: 3.0.6
        modin:
         cudf:
      pyarrow: 25.0.1
      pyspark:
       polars: 1.44.2
         dask:
       duckdb: 1.5.6
         ibis:
     sqlframe:

Relevant log output

No traceback on the nullable dtypes: the operation succeeds and returns 0. On `int64[pyarrow]` it raises, but that path is already xfailed in the test suite:

NotImplementedError: mod not implemented.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions