BUG: Logical OR does not obey Kleene three-valued logic #71478

Activity

1. warwickmm

   added

   Bug

   Needs TriageIssue that has not been reviewed by a pandas team memberIssue that has not been reviewed by a pandas team member

   on Oct 22, 2026
2. mohamd764 commented on Oct 22, 2026

   

   mohamd764

   on Oct 22, 2026

   More actions

   Confirmed on pandas 3.1.0 (Python 3.13, Linux).

   Same-column Kleene OR is fine (x | x, and Series |), but when the frames need a column union the missing side is filled with np.nan, which promotes the new column to float64:

   left, right = x._align_for_op(y, 1, flex=True, level=None)
   # left.dtypes:  a boolean, b float64
   # right.dtypes: a float64, b boolean

   BooleanArray._logical_method then does np.asarray(other, dtype="bool"). bool(np.nan) is True, so every missing cell becomes True before kleene_or runs — and NA | True / False | True collapse to True. That is why AND looks fine on the same inputs: True is the AND identity, so the poisoned fill does not show up.

   A couple of nearby notes:

   1. _should_reindex_frame_op returns False when the column intersection is empty, so this case goes through _align_for_op + float NaN fill instead of the shared-column reindex path.
   2. Partial overlap still ends up with NaN in the non-shared boolean columns after result.reindex(..., axis=1) (those cells become float NaN rather than boolean <NA>).

   Possible directions: keep boolean dtype on align/reindex for logical ops (fill with pd.NA), and/or treat NaNs as masked/pd.NA in BooleanArray._logical_method instead of casting them with dtype="bool".
3. jorisvandenbossche commented on Oct 22, 2026

   

   jorisvandenbossche

   on Oct 22, 2026

   Last edited by jorisvandenbossche

   Member

   More actions

   @warwickmm thanks for the report!

   @mohamd764's agent's analysis is correct. It is indeed due to the alignment introducing all-NaN columns (although I had expected them to be object dtype instead of float), and then the logical OR operation between a boolean array and an all-NaN float array behaves as this:

   >>> arr = pd.array([True, pd.NA], dtype="boolean")
   >>> arr | np.array([np.nan, np.nan])
   <BooleanArray>
   [True, True]
   Length: 2, dtype: boolean

   This is indeed because the np.array([np.nan, np.nan]) gets converted to a boolean array of [True, True]. However, the typical constructor for boolean array would actually preserve the missing values:

   >>> pd.array(np.array([np.nan, np.nan]), dtype="boolean")
   <BooleanArray>
   [<NA>, <NA>]
   Length: 2, dtype: boolean

   And I think we should do that under the hood in the BooleanArray._logical_method as well.
4. jorisvandenbossche

   added

   Numeric OperationsArithmetic, Comparison, and Logical operationsArithmetic, Comparison, and Logical operations

   NA - MaskedArraysRelated to pd.NA and nullable extension arraysRelated to pd.NA and nullable extension arrays

   and removed

   Needs TriageIssue that has not been reviewed by a pandas team memberIssue that has not been reviewed by a pandas team member

   on Oct 22, 2026
5. swethakommuri commented on Oct 23, 2026

   

   swethakommuri

   on Oct 23, 2026

   More actions

   take

   I'd like to work on this. Following @jorisvandenbossche's suggestion, I plan to convert a float/object other containing NaN into a masked BooleanArray (as pd.array(..., dtype="boolean") does) inside BooleanArray._logical_method, rather than np.asarray(other, dtype="bool").

   One thing I noticed while reproducing on 3.1.0: the array-level bug isn't limited to |. & and ^ are also wrong:

   >>> arr = pd.array([True, False, pd.NA], dtype="boolean")
   >>> nan = np.array([np.nan] * 3)
   >>> arr | nan   # [True, True, True]   expected [True, <NA>, <NA>]
   >>> arr & nan   # [True, False, <NA>]  expected [<NA>, False, <NA>]
   >>> arr ^ nan   # [False, True, <NA>]  expected [<NA>, <NA>, <NA>]

   The DataFrame x & y / x ^ y results happen to come out right, but the underlying array op isn't, so I'll add tests covering all three operators at both the array and DataFrame level.

   The partial-overlap case (x | z leaving the non-shared column as float NaN instead of boolean <NA>) looks like a separate alignment/reindex concern. I'd keep it out of this PR unless you'd prefer both together.
6. ryan1234814 commented on Oct 23, 2026

   

   ryan1234814

   on Oct 23, 2026

   Contributor

   More actions

   /take

   👍React with 👍1Reacted by github-actions
7. github-actions

   assigned

   ryan1234814

   on Oct 23, 2026
8. swethakommuri commented on Oct 23, 2026

   

   swethakommuri

   on Oct 23, 2026

   More actions

   Hi @ryan1234814 - I’d claimed this just above with a plan and repro for |, & and ^ at the array level (I think the bot missed my take because the comment had more text after it). I already have the fix in BooleanArray._logical_method plus tests underway. Have you started on it? If not, would you mind if I continue? Happy to coordinate either way.

   @jorisvandenbossche - I’m keeping the partial-overlap reindex case (x | z) out of scope for this one and can open it as a separate issue.
9. ryan1234814 commented on Oct 25, 2026

   

   ryan1234814

   on Oct 25, 2026

   Contributor

   More actions

   @swethakommuri You can raise the pr in that case
10. ryan1234814 commented on Oct 25, 2026

    

    ryan1234814

    on Oct 25, 2026

    Contributor

    More actions

    /untake

    👍React with 👍2Reacted by github-actions and swethakommuri
11. github-actions

    unassigned

    ryan1234814

    on Oct 25, 2026
12. swethakommuri commented on Oct 26, 2026

    

    swethakommuri

    on Oct 26, 2026

    More actions

    /take

    👍React with 👍1Reacted by github-actions
13. github-actions

    assigned

    swethakommuri

    on Oct 26, 2026