Repository navigation
BUG: Series.str.get_dummies adds "NaN" / "" column for categorical / Arrow-backed strings #71887
Description
Ref: #70812
s = pd.Series(["a|b", "a", None, ""], dtype=object)
print(s.str.get_dummies("|"))
# a b
# 0 1 1
# 1 1 0
# 2 0 0
# 3 0 0
print(s.astype("category").str.get_dummies("|"))
# NaN a b
# 0 0 1 1
# 1 0 1 0
# 2 1 0 0
# 3 0 0 0
print(pd.Series(["a|b", "a", None, ""], dtype="str").str.get_dummies("|"))
# a b
# 0 0 1 1
# 1 0 1 0
# 2 0 0 0
# 3 1 0 0Every dtype should give the same result as object dtype: NA values and empty strings get a row of zeros, with no "NaN" or "" column:
a b
0 1 1
1 1 0
2 0 0
3 0 0
Activity
AntsiferovBogdan commentedon Nov 4, 2026
/take
[-]BBUG: Series.str.get_dummies adds "NaN" / "" column for categorical / Arrow-backed strings[/-][+]BUG: Series.str.get_dummies adds "NaN" / "" column for categorical / Arrow-backed strings[/+]jorisvandenbossche commentedon Nov 4, 2026
This is a regression from 2.x for strings (due to implicit object -> str dtype change), so marking for 3.1.x
jorisvandenbossche commentedon Nov 4, 2026
Although for
categoryit is long standing behaviour, so there we should maybe not fix it in a patch releaseAntsiferovBogdan commentedon Nov 4, 2026
Thanks for clarifying the release scope. I can split this into two PRs: an Arrow-backed string fix suitable for backporting to 3.1.x, and a separate categorical fix.
The Arrow PR would exclude the empty-string dummy column and preserve the row count when no labels remain, without changing categorical behavior.
Would that split work for you? I would use Refs #71887 for the Arrow PR so that the issue remains open for the categorical part.