ENH: DataFrame.filter and Series.filter to take a boolean masks

@@ -949,6 +949,49 @@ and :ref:`Advanced Indexing <advanced>` you may select along more than one axis        df.iloc[s.values, 1] +.. _indexing.boolean.filter:++Filtering with ``DataFrame.filter``+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~++.. versionadded:: 3.1.0++:meth:`DataFrame.filter` and :meth:`Series.filter` also accept a boolean mask+through the ``cond`` keyword, keeping the rows where the mask is True. A+:class:`Series` mask is aligned on the index; any other array-like must have+the same length as the index.++.. ipython:: python++   df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]}, index=list('abc'))+   df.filter(cond=df['A'] > 1)++``cond`` may also be a callable that returns a boolean mask, or an expression+created with :func:`pandas.col`. These two forms may be passed positionally,+which is convenient in a method chain where the intermediate object has no+name:++.. ipython:: python++   df.filter(pd.col('A') > 1)+   df.assign(C=pd.col('A') + pd.col('B')).filter(lambda df: df['C'] > 5)++Pass ``axis=1`` to filter columns instead of rows:++.. ipython:: python++   df.filter(cond=df.columns.str.startswith('A'), axis=1)++Missing values in the mask are treated as False by default, matching+``df[mask]`` for a mask with nullable boolean dtype. Pass ``na=True`` to keep+the corresponding rows instead, or ``na="raise"`` to raise an error:++.. ipython:: python++   mask = pd.array([True, None, False], dtype="boolean")+   df.filter(cond=mask)+   df.filter(cond=mask, na=True)+ .. _indexing.basics.indexing_isin:  Indexing with isin

@@ -84,7 +84,7 @@ User-Defined Functions can be applied across various pandas methods: +-------------------------------+------------------------+--------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ | :ref:`udf.pipe`               | Series or DataFrame    | Series or DataFrame      | Chain functions together to apply to Series or DataFrame                                                                                     | +-------------------------------+------------------------+--------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+-| :ref:`udf.filter`             | Series or DataFrame    | Boolean                  | Only accepts UDFs in group by. Function is called for each group, and the group is removed from the result if the function returns ``False`` |+| :ref:`udf.filter`             | Series or DataFrame    | Boolean mask             | Function is called with the object and returns a boolean mask; rows where the mask is ``False`` are removed from the result                  | +-------------------------------+------------------------+--------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ | :ref:`udf.agg`                | Series or DataFrame    | Scalar or Series         | Aggregate and summarize values, e.g., sum or custom reducer                                                                                  | +-------------------------------+------------------------+--------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+@@ -267,9 +267,19 @@ When to use: Use :meth:`pipe` when you need to create a pipeline of operations a ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  The ``filter`` method is used to select a subset of rows that match certain criteria.-:meth:`Series.filter` and :meth:`DataFrame.filter` do not support user defined functions,-but :meth:`SeriesGroupBy.filter` and :meth:`DataFrameGroupBy.filter` do. You can read more-about ``filter`` in groupby operations in :ref:`groupby.filter`.+:meth:`Series.filter` and :meth:`DataFrame.filter` accept a callable that is passed the+object and returns a boolean mask; the rows where the mask is True are kept. This is+convenient in a method chain where the intermediate object has no name.++.. ipython:: python++   df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})+   df.assign(c=df["a"] + df["b"]).filter(lambda df: df["c"] > 5)++You can read more about filtering with a boolean mask in :ref:`indexing.boolean.filter`.+:meth:`SeriesGroupBy.filter` and :meth:`DataFrameGroupBy.filter` instead call the function+once per group; you can read more about ``filter`` in groupby operations in+:ref:`groupby.filter`.  .. _udf.agg: 

@@ -56,6 +56,64 @@ earlier in the same call    1      Alice         264      7920    2        Bob         420     12600 +.. _whatsnew_310.enhancements.filter_mask:++Filtering with a boolean mask in ``DataFrame.filter`` and ``Series.filter``+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^++:meth:`DataFrame.filter` and :meth:`Series.filter` now accept a boolean mask through the new+``cond`` keyword, keeping the rows (or columns with ``axis=1``) where the mask is True.+``cond`` may also be a callable returning a boolean mask or an expression created with+:func:`pandas.col`; these two forms may also be passed positionally, which is convenient in+method chains (:issue:`61317`).++.. code-block:: ipython++    In [1]: df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})++    In [2]: df.filter(cond=df["a"] > 1)+    Out[2]:+       a  b+    1  2  5+    2  3  6++    In [3]: df.filter(pd.col("a") > 1)+    Out[3]:+       a  b+    1  2  5+    2  3  6++    In [4]: df.filter(lambda df: df["b"] < 6)+    Out[4]:+       a  b+    0  1  4+    1  2  5++A :class:`Series` mask is aligned with the filtered axis; any other mask must have the same+length as that axis. Missing values in the mask are treated as False by default, matching+``obj[mask]`` for a mask with nullable boolean dtype. Pass ``na=True`` to keep the+corresponding entries instead, or ``na="raise"`` to raise.++.. code-block:: ipython++    In [5]: mask = pd.array([True, None, False], dtype="boolean")++    In [6]: df.filter(cond=mask)+    Out[6]:+       a  b+    0  1  4++    In [7]: df.filter(cond=mask, na=True)+    Out[7]:+       a  b+    0  1  4+    1  2  5++Any other value passed positionally continues to select labels, as before. Since a list-like+of booleans passed this way is likely intended as a mask, it now issues a ``UserWarning``;+pass ``items`` to select labels or ``cond`` to filter with a mask. Selecting labels with+``items``, ``like``, or ``regex`` will be deprecated in a future version.+ .. _whatsnew_310.enhancements.enhancement2:  enhancement2

@@ -108,6 +108,7 @@ from pandas.core.array_algos.replace import should_use_regex from pandas.core.arrays import ExtensionArray from pandas.core.base import PandasObject+from pandas.core.col import Expression from pandas.core.construction import extract_array from pandas.core.flags import Flags from pandas.core.indexes.api import (@@ -120,6 +121,10 @@ ) from pandas.core.internals import BlockManager from pandas.core.methods.describe import describe_ndframe+from pandas.core.methods.filter import (+    filter_mask,+    is_mask,+) from pandas.core.missing import (     clean_fill_method,     clean_reindex_fill_method,@@ -5592,84 +5597,233 @@ def _reindex_with_indexers(      def filter(         self,-        items=None,+        arg=None,+        /,         like: str | None = None,         regex: str | None = None,         axis: Axis | None = None,+        *,+        items=None,+        cond=None,+        na: Literal["raise"] | bool = False,     ) -> Self:         """-        Subset the DataFrame or Series according to the specified index labels.+        Subset the rows or columns according to a boolean mask or the labels. -        For DataFrame, filter rows or columns depending on ``axis`` argument.-        Note that this routine does not filter based on content.-        The filter is applied to the labels of the index.+        Rows (or columns with ``axis=1``) are kept where a boolean mask is+        True, or selected by their labels with ``items``, ``like``, or+        ``regex``.          Parameters         -----------        items : list-like-            Keep labels from axis which are in items.+        arg : callable, expression, or list-like, optional+            Positional-only. A callable or an expression created with+            :func:`pandas.col` is a boolean mask, see ``cond``. Any other+            list-like selects labels, see ``items``. A list-like of booleans+            also selects labels, but since it is likely intended as a mask a+            warning is issued; pass ``items`` or ``cond`` instead to be+            explicit.         like : str-            Keep labels from axis for which "like in label == True".+            Keep labels from axis for which "like in label == True". This+            will be deprecated in a future version; use+            ``obj.filter(lambda obj: obj.columns.astype(str).str.contains(like,+            regex=False), axis=1)`` instead.         regex : str (regular expression)             Keep labels from axis for which re.search(regex, label) == True.+            This will be deprecated in a future version; use+            ``obj.filter(lambda obj: obj.columns.astype(str).str.contains(regex),+            axis=1)`` instead.         axis : {0 or 'index', 1 or 'columns', None}, default None             The axis to filter on, expressed either as an index (int)-            or axis name (str). By default this is the info axis, 'columns' for-            ``DataFrame``. For ``Series`` this parameter is unused and defaults to-            ``None``.+            or axis name (str). Defaults to the index for a boolean mask,+            and to the info axis ('columns' for ``DataFrame``) when+            selecting labels. An expression only supports the index. For+            ``Series`` this parameter is unused and defaults to ``None``.+        items : list-like, optional+            Keep labels from axis which are in ``items``. This will be+            deprecated in a future version; use :meth:`DataFrame.select`+            when all of ``items`` are present in the columns, or+            ``obj.loc[:, pd.Index(items).intersection(obj.columns)]`` (or+            the equivalent for the index) otherwise.+        cond : array-like of bool, callable, or expression, optional+            A boolean mask selecting the entries to keep. A :class:`Series`+            is aligned with the labels of the filtered axis; any other+            array-like must have the same length as that axis. A callable is+            called with the object and must return a boolean mask. An+            expression such as ``pd.col("a") > 1`` is evaluated against the+            DataFrame and is only supported with ``axis=0``.+        na : {"raise", True, False}, default False+            How to treat missing values in a boolean mask. ``True`` or+            ``False`` treats missing values as that value, matching+            ``obj[mask]`` for a mask with nullable boolean dtype; ``"raise"``+            raises a ``ValueError``. Ignored when selecting labels.          Returns         -------         Same type as caller             The filtered subset of the DataFrame or Series. +        Raises+        ------+        TypeError+            If none or more than one of the positional argument, ``items``,+            ``cond``, ``like``, and ``regex`` is passed, or if ``cond`` is+            not a one-dimensional boolean mask.+        ValueError+            If a mask contains missing values and ``na="raise"``, if a+            boolean array is not one-dimensional, or if an expression is+            passed with ``axis=1``.+        IndexError+            If a mask that is not a Series has a different length than the+            filtered axis.+        IndexingError+            If a Series mask cannot be aligned with the filtered axis.++        Warns+        -----+        UserWarning+            If a list-like of booleans is passed positionally.+         See Also         --------         DataFrame.loc : Access a group of rows and columns             by label(s) or a boolean array.+        DataFrame.where : Replace values where the condition is False.          Notes         ------        The ``items``, ``like``, and ``regex`` parameters are-        enforced to be mutually exclusive.+        The positional argument is a boolean mask only when it is a callable+        or an expression; any other value, including a list-like of booleans,+        selects labels. Use ``cond`` to filter with a boolean array or+        :class:`Series`. -        ``axis`` defaults to the info axis that is used when indexing-        with ``[]``.+        Selecting labels with ``items``, ``like``, or ``regex`` will be+        deprecated in a future version.          Examples         --------         >>> df = pd.DataFrame(-        ...     np.array(([1, 2, 3], [4, 5, 6])),+        ...     {"one": [1, 4], "two": [2, 5], "three": [3, 6]},         ...     index=["mouse", "rabbit"],-        ...     columns=["one", "two", "three"],         ... )         >>> df                 one  two  three         mouse     1    2      3         rabbit    4    5      6 -        >>> # select columns by name-        >>> df.filter(items=["one", "three"])-                 one  three+        Filter rows with a boolean Series.++        >>> df.filter(cond=df["two"] > 2)+                one  two  three+        rabbit    4    5      6++        The same using an expression or a callable, which are convenient in+        method chains and may be passed positionally.++        >>> df.filter(pd.col("two") > 2)+                one  two  three+        rabbit    4    5      6+        >>> df.filter(lambda df: df["two"] > 2)+                one  two  three+        rabbit    4    5      6++        Filter columns with a boolean array.++        >>> df.filter(cond=df.columns.str.endswith("e"), axis=1)+                one  three         mouse     1      3         rabbit    4      6 -        >>> # select columns by regular expression+        Missing values in the mask are treated as False by default; pass+        ``na="raise"`` to raise instead, or ``na=True`` to keep them.++        >>> mask = pd.array([True, None], dtype="boolean")+        >>> df.filter(cond=mask)+               one  two  three+        mouse    1    2      3+        >>> df.filter(cond=mask, na=True)+                one  two  three+        mouse     1    2      3+        rabbit    4    5      6++        Select columns by their labels.++        >>> df.filter(items=["one", "three"])+                one  three+        mouse     1      3+        rabbit    4      6         >>> df.filter(regex="e$", axis=1)-                 one  three+                one  three         mouse     1      3         rabbit    4      6--        >>> # select rows containing 'bbi'         >>> df.filter(like="bbi", axis=0)-                 one  two  three+                one  two  three         rabbit    4    5      6         """-        nkw = common.count_not_none(items, like, regex)+        nkw = common.count_not_none(arg, items, cond, like, regex)         if nkw > 1:             raise TypeError(-                "Keyword arguments `items`, `like`, or `regex` are mutually exclusive"+                "The positional argument and the keyword arguments `items`, "+                "`cond`, `like`, and `regex` are mutually exclusive"             )+        if nkw == 0:+            raise TypeError(+                "Must pass a positional argument or one of `items`, `cond`, "+                "`like`, or `regex`"+            )++        if na is not True and na is not False and na != "raise":+            raise ValueError(f"na must be 'raise', True, or False, got {na!r}")++        if arg is not None:+            # Only a callable or expression is unambiguously a mask when passed+            # positionally. Anything else selected labels before masks were+            # supported and keeps doing so, since a boolean list-like is a+            # valid (if unusual) list of labels.+            if callable(arg):+                cond = arg+            else:+                if is_mask(arg):+                    warnings.warn(+                        "A list-like of booleans passed positionally to "+                        f"{type(self).__name__}.filter selects labels. Pass "+                        "cond=... to filter with a boolean mask, or items=... "+                        "to select labels without this warning.",+                        UserWarning,+                        stacklevel=find_stack_level(),+                    )+                items = arg++        if cond is not None:+            mask_axis = 0 if axis is None else self._get_axis_number(axis)+            if isinstance(cond, Expression) and self.ndim != 2:+                raise TypeError(+                    "Expressions such as pd.col(...) are only supported by "+                    "DataFrame.filter"+                )+            if isinstance(cond, Expression) and mask_axis != 0:+                raise ValueError(+                    "Expressions such as pd.col(...) are only supported by "+                    f"{type(self).__name__}.filter with axis=0, since they "+                    "evaluate to a mask aligned with the index"+                )+            if callable(cond):+                # Expression defines __call__, so it enters here too+                mask = common.apply_if_callable(cond, self)+                if not is_mask(mask):+                    kind = "expression" if isinstance(cond, Expression) else "callable"+                    raise TypeError(+                        f"The {kind} passed to {type(self).__name__}.filter "+                        "must evaluate to a one-dimensional boolean mask"+                    )+            elif is_mask(cond):+                mask = cond+            else:+                raise TypeError(+                    f"cond passed to {type(self).__name__}.filter must be a "+                    "one-dimensional boolean mask"+                )+            return filter_mask(self, mask, mask_axis, na)          if axis is None:             axis = self._info_axis_name@@ -5699,7 +5853,10 @@ def f(x) -> bool:             values = labels.map(f)             return self.loc(axis=axis)[values]         else:-            raise TypeError("Must pass either `items`, `like`, or `regex`")+            raise TypeError(+                "Must pass a positional argument or one of `items`, `cond`, "+                "`like`, or `regex`"+            )      @final     def head(self, n: int = 5) -> Self:

@@ -0,0 +1,104 @@+"""+Implementation of DataFrame.filter and Series.filter with a boolean mask.+"""++from __future__ import annotations++from typing import (+    TYPE_CHECKING,+    Literal,+)++import numpy as np++from pandas._libs import lib++from pandas.core.dtypes.common import is_bool_dtype+from pandas.core.dtypes.generic import (+    ABCDataFrame,+    ABCMultiIndex,+    ABCSeries,+)++from pandas.core.construction import (+    array as pd_array,+    extract_array,+)+from pandas.core.indexing import check_bool_indexer++if TYPE_CHECKING:+    from pandas._typing import (+        AnyArrayLike,+        AxisInt,+        NDFrameT,+    )++    from pandas import Series+++def is_mask(key: object) -> bool:+    """+    Whether ``key`` is a boolean mask, possibly holding missing values.++    A boolean dtype is a mask. Otherwise ``key`` must be a list or an+    object-dtype array-like that is one-dimensional with every non-missing+    element a bool. A tuple or DataFrame is never a mask.+    """+    if isinstance(key, ABCDataFrame):+        return False+    if isinstance(key, list):+        values = np.asarray(key, dtype=object)+    else:+        dtype = getattr(key, "dtype", None)+        if dtype is None or isinstance(key, ABCMultiIndex):+            return False+        if is_bool_dtype(dtype):+            return True+        if dtype != np.object_:+            return False+        values = np.asarray(key, dtype=object)+    if values.ndim != 1:+        # lib.is_bool_array iterates a 2-D array flat, so a list of bool tuples+        # (labels for a MultiIndex) would otherwise be mistaken for a mask.+        return False+    return lib.is_bool_array(values, skipna=True)+++def filter_mask(+    obj: NDFrameT,+    mask: list | AnyArrayLike,+    axis: AxisInt,+    na: Literal["raise"] | bool,+) -> NDFrameT:+    """+    Select the entries of ``obj`` along ``axis`` where ``mask`` is True.++    ``mask`` must already satisfy ``is_mask``, so its values are booleans or+    missing and can be coerced to a nullable boolean array without loss.+    """+    labels = obj._get_axis(axis)++    if getattr(mask, "ndim", 1) != 1:+        raise ValueError(+            f"The mask passed to {type(obj).__name__}.filter must be one-dimensional"+        )++    values = extract_array(mask, extract_numpy=True)+    if isinstance(values, np.ndarray) and values.dtype == np.bool_:+        # A NumPy bool array cannot hold missing values, so skip the copy+        # through BooleanArray that the NA handling below requires.+        np_mask = values+    else:+        values = pd_array(values, dtype="boolean", copy=False)+        if values.isna().any():+            if na == "raise":+                raise ValueError("The mask contains missing values")+            values = values.fillna(na)+        np_mask = values.to_numpy(dtype=bool)+    key: Series | np.ndarray+    if isinstance(mask, ABCSeries):+        key = mask._constructor(np_mask, index=mask.index, copy=False)+    else:+        key = np_mask+    aligned_mask = check_bool_indexer(labels, key)+    return obj.loc(axis=axis)[aligned_mask]

@@ -1,6 +1,8 @@ import numpy as np import pytest +import pandas.util._test_decorators as td+ import pandas as pd import pandas._testing as tm @@ -50,11 +52,12 @@ def test_filter(self, float_frame, float_string_frame):         tm.assert_frame_equal(filtered, expected)          # pass in None-        with pytest.raises(TypeError, match="Must pass"):+        no_arg_msg = "Must pass a positional argument or one of `items`, `cond`"+        with pytest.raises(TypeError, match=no_arg_msg):             float_frame.filter()-        with pytest.raises(TypeError, match="Must pass"):+        with pytest.raises(TypeError, match=no_arg_msg):             float_frame.filter(items=None)-        with pytest.raises(TypeError, match="Must pass"):+        with pytest.raises(TypeError, match=no_arg_msg):             float_frame.filter(axis=1)          # test mutually exclusive arguments@@ -152,3 +155,356 @@ def test_filter_different_dtype(self):         result = df.filter(items=["B", "A"])         expected = df[[]]         tm.assert_frame_equal(result, expected)+++@pytest.fixture+def df():+    return pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}, index=["x", "y", "z"])+++@pytest.mark.parametrize(+    "mask",+    [+        [False, True, True],+        np.array([False, True, True]),+        pd.array([False, True, True], dtype="boolean"),+        pd.Index([False, True, True]),+        pd.Series([False, True, True], index=["x", "y", "z"]),+        lambda df: df["a"] > 1,+        pd.col("a") > 1,+    ],+)+def test_filter_cond_rows(df, mask):+    # GH#61317+    result = df.filter(cond=mask)+    expected = df.iloc[1:]+    tm.assert_frame_equal(result, expected)+++@pytest.mark.parametrize("mask", [lambda df: df["a"] > 1, pd.col("a") > 1])+def test_filter_positional_callable_is_mask(df, mask):+    # GH#61317+    result = df.filter(mask)+    expected = df.iloc[1:]+    tm.assert_frame_equal(result, expected)+++@pytest.mark.parametrize(+    "mask",+    [+        [True, False, True],+        [True, None, False],+        np.array([True, False, True]),+        pd.array([True, False, True], dtype="boolean"),+        pd.Index([True, False, True]),+        pd.Series([True, False, True], index=["x", "y", "z"]),+    ],+)+def test_filter_positional_bools_select_labels(df, mask):+    # GH#61317+    # A list-like of booleans passed positionally selects labels, as it did+    # before masks were supported, but warns since it is likely meant as a mask+    msg = "A list-like of booleans passed positionally to DataFrame.filter"+    with tm.assert_produces_warning(UserWarning, match=msg):+        result = df.filter(mask)+    expected = df.iloc[:, []]+    tm.assert_frame_equal(result, expected)++    # the explicit keyword does not warn+    result = df.filter(items=mask)+    tm.assert_frame_equal(result, expected)+++@pytest.mark.parametrize(+    "arg, cols",+    [+        (["b", "a"], ["b", "a"]),+        (("b", "a"), ["b", "a"]),+        ((True, False, True), []),+    ],+)+def test_filter_positional_labels_no_warning(df, arg, cols):+    # GH#61317+    # tuples are never masks+    result = df.filter(arg)+    expected = df[cols]+    tm.assert_frame_equal(result, expected)+++def test_filter_na_label():+    # GH#61317+    df = pd.DataFrame({np.nan: [1], "a": [2]})+    result = df.filter(items=[np.nan])+    expected = df.iloc[:, [0]]+    tm.assert_frame_equal(result, expected, check_column_type=False)++    df = pd.DataFrame({"a": [1, 2]}, index=[np.nan, "x"])+    result = df.filter(items=[np.nan], axis=0)+    expected = df.iloc[[0]]+    tm.assert_frame_equal(result, expected, check_index_type=False)+++def test_filter_tuple_labels_multiindex():+    # GH#61317+    mi = pd.MultiIndex.from_tuples([(True, False), (False, True)])+    df = pd.DataFrame({"a": [1, 2]}, index=mi)+    result = df.filter([(True, False)], axis=0)+    expected = df.iloc[[0]]+    tm.assert_frame_equal(result, expected)+++@pytest.mark.parametrize(+    "dtype",+    [+        "bool",+        "boolean",+        pytest.param("bool[pyarrow]", marks=td.skip_if_no("pyarrow")),+    ],+)+def test_filter_bool_labels(dtype):+    # GH#61317+    df = pd.DataFrame({"a": [1, 2]}, index=pd.Index([True, False], dtype=dtype))+    result = df.filter(items=[True], axis=0)+    expected = df.iloc[[0]]+    tm.assert_frame_equal(result, expected)++    msg = "A list-like of booleans passed positionally to DataFrame.filter"+    with tm.assert_produces_warning(UserWarning, match=msg):+        result = df.filter([True], axis=0)+    tm.assert_frame_equal(result, expected)++    result = df.filter(cond=[False, True], axis=0)+    expected = df.iloc[[1]]+    tm.assert_frame_equal(result, expected)+++@pytest.mark.parametrize(+    "kwargs",+    [+        {"items": ["a"]},+        {"cond": [True, False, True]},+        {"like": "a"},+        {"regex": "a"},+    ],+)+def test_filter_positional_and_keyword_mutually_exclusive(df, kwargs):+    # GH#61317+    msg = "mutually exclusive"+    with pytest.raises(TypeError, match=msg):+        df.filter(["a"], **kwargs)+    with pytest.raises(TypeError, match=msg):+        df.filter(cond=[True, False, True], items=["a"])+++@pytest.mark.parametrize(+    "cond",+    [["a"], [1, 0, 1], (True, False, True), "abc", np.array([1.0, 0.0, 1.0])],+)+def test_filter_cond_not_mask_raises(df, cond):+    # GH#61317+    msg = "cond passed to DataFrame.filter must be a one-dimensional boolean mask"+    with pytest.raises(TypeError, match=msg):+        df.filter(cond=cond)+++def test_filter_cond_series_aligns(df):+    # GH#61317+    mask = pd.Series([True, True, False], index=["z", "y", "x"])+    result = df.filter(cond=mask)+    expected = df.iloc[1:]+    tm.assert_frame_equal(result, expected)+++def test_filter_cond_series_unalignable(df):+    # GH#61317+    mask = pd.Series([True, True, False], index=["z", "y", "w"])+    msg = "Unalignable boolean Series provided as indexer"+    with pytest.raises(pd.errors.IndexingError, match=msg):+        df.filter(cond=mask)+++def test_filter_cond_wrong_length(df):+    # GH#61317+    msg = "Boolean index has wrong length: 2 instead of 3"+    with pytest.raises(IndexError, match=msg):+        df.filter(cond=np.array([True, False]))+++@pytest.mark.parametrize("axis", [1, "columns"])+def test_filter_cond_columns(df, axis):+    # GH#61317+    result = df.filter(cond=[False, True], axis=axis)+    expected = df[["b"]]+    tm.assert_frame_equal(result, expected)+++@pytest.mark.parametrize("axis", [1, "columns"])+def test_filter_callable_columns(df, axis):+    # GH#61317+    result = df.filter(lambda df: df.columns == "b", axis=axis)+    expected = df[["b"]]+    tm.assert_frame_equal(result, expected)++    result = df.filter(cond=lambda df: df.columns == "b", axis=axis)+    tm.assert_frame_equal(result, expected)+++@pytest.mark.parametrize("axis", [1, "columns"])+def test_filter_expression_columns_raises(df, axis):+    # GH#61317+    cond = pd.col("a") > 1+    msg = (+        "Expressions such as pd.col\\(...\\) are only supported by DataFrame.filter "+        "with axis=0"+    )+    with pytest.raises(ValueError, match=msg):+        df.filter(cond, axis=axis)+    with pytest.raises(ValueError, match=msg):+        df.filter(cond=cond, axis=axis)+++def test_filter_cond_2d_raises(df):+    # GH#61317+    msg = "cond passed to DataFrame.filter must be a one-dimensional boolean mask"+    with pytest.raises(TypeError, match=msg):+        df.filter(cond=df > 1)+    msg = "must evaluate to a one-dimensional boolean mask"+    with pytest.raises(TypeError, match=msg):+        df.filter(lambda df: df > 1)++    msg = "The mask passed to DataFrame.filter must be one-dimensional"+    with pytest.raises(ValueError, match=msg):+        df.filter(cond=np.array([[True, False, True]]))+++def test_filter_positional_frame_selects_labels(df):+    # GH#61317+    # a DataFrame is not a mask positionally; it fails as a list of labels+    # without warning+    msg = "Index data must be 1-dimensional"+    with pytest.raises(ValueError, match=msg):+        df.filter(df > 1)+++def test_filter_callable_must_return_mask(df):+    # GH#61317+    msg = "The callable passed to DataFrame.filter must evaluate to a one-dimensional"+    with pytest.raises(TypeError, match=msg):+        df.filter(lambda df: ["a"])+    with pytest.raises(TypeError, match=msg):+        df.filter(cond=lambda df: ["a"])+++def test_filter_expression_must_return_mask(df):+    # GH#61317+    msg = "The expression passed to DataFrame.filter must evaluate to a one-dimensional"+    with pytest.raises(TypeError, match=msg):+        df.filter(pd.col("a"))+++@pytest.mark.parametrize(+    "mask",+    [+        [True, None, False],+        np.array([True, None, False], dtype=object),+        pd.array([True, None, False], dtype="boolean"),+        pd.Series([True, None, False], dtype="boolean", index=["x", "y", "z"]),+        pd.Index([True, None, False], dtype=object),+    ],+)+def test_filter_cond_na(df, mask):+    # GH#61317+    # missing values are treated as False by default, matching df[mask]+    # for a nullable boolean mask+    result = df.filter(cond=mask)+    expected = df.iloc[[0]]+    tm.assert_frame_equal(result, expected)++    result = df.filter(cond=mask, na=False)+    tm.assert_frame_equal(result, expected)++    msg = "The mask contains missing values"+    with pytest.raises(ValueError, match=msg):+        df.filter(cond=mask, na="raise")++    result = df.filter(cond=mask, na=True)+    expected = df.iloc[[0, 1]]+    tm.assert_frame_equal(result, expected)+++@td.skip_if_no("pyarrow")+def test_filter_cond_na_pyarrow(df):+    # GH#61317+    mask = pd.array([True, None, False], dtype="bool[pyarrow]")+    result = df.filter(cond=mask)+    expected = df.iloc[[0]]+    tm.assert_frame_equal(result, expected)++    msg = "The mask contains missing values"+    with pytest.raises(ValueError, match=msg):+        df.filter(cond=mask, na="raise")++    result = df.filter(cond=mask, na=True)+    expected = df.iloc[[0, 1]]+    tm.assert_frame_equal(result, expected)+++def test_filter_cond_all_na(df):+    # GH#61317+    result = df.filter(cond=[None, None, None])+    expected = df.iloc[[]]+    tm.assert_frame_equal(result, expected)++    result = df.filter(cond=[None, None, None], na=True)+    tm.assert_frame_equal(result, df)++    with pytest.raises(ValueError, match="The mask contains missing values"):+        df.filter(cond=[None, None, None], na="raise")+++def test_filter_invalid_na(df):+    # GH#61317+    msg = "na must be 'raise', True, or False, got 'ignore'"+    with pytest.raises(ValueError, match=msg):+        df.filter(cond=[True, False, True], na="ignore")+++@pytest.mark.parametrize(+    "kwargs",+    [+        {"items": ["a"]},+        {"items": ["x"], "axis": 0},+        {"like": "a"},+        {"like": "x", "axis": "index"},+        {"regex": "a"},+        {"regex": "x", "axis": 0},+    ],+)+def test_filter_labels(df, kwargs):+    # GH#61317+    # label-based usage keeps working and defaults to the columns+    result = df.filter(**kwargs)+    if kwargs.get("axis") in (0, "index"):+        expected = df.iloc[[0]]+    else:+        expected = df[["a"]]+    tm.assert_frame_equal(result, expected)+++def test_filter_cond_default_axis_is_index(df):+    # GH#61317+    # unlike the label-based usage, a mask defaults to the index+    result = df.filter(cond=[True, False, True])+    expected = df.iloc[[0, 2]]+    tm.assert_frame_equal(result, expected)+++def test_filter_like_regex_positional():+    # GH#61317+    # like, regex, and axis remain positional-or-keyword after the+    # positional-only first argument+    df = pd.DataFrame({"a": [1]}, index=["x"])+    result = df.filter(None, "x", None, 0)+    tm.assert_frame_equal(result, df)+    result = df.filter(None, None, "^x", 0)+    tm.assert_frame_equal(result, df)

@@ -0,0 +1,160 @@+import numpy as np+import pytest++import pandas as pd+import pandas._testing as tm+++@pytest.fixture+def ser():+    return pd.Series([1, 2, 3], index=["x", "y", "z"])+++@pytest.mark.parametrize(+    "mask",+    [+        [False, True, True],+        np.array([False, True, True]),+        pd.array([False, True, True], dtype="boolean"),+        pd.Series([False, True, True], index=["x", "y", "z"]),+        lambda ser: ser > 1,+    ],+)+def test_filter_cond(ser, mask):+    # GH#61317+    result = ser.filter(cond=mask)+    expected = ser.iloc[1:]+    tm.assert_series_equal(result, expected)+++def test_filter_positional_callable_is_mask(ser):+    # GH#61317+    result = ser.filter(lambda ser: ser > 1)+    expected = ser.iloc[1:]+    tm.assert_series_equal(result, expected)+++@pytest.mark.parametrize(+    "mask",+    [+        [True, False, True],+        np.array([True, False, True]),+        pd.Series([True, False, True], index=["x", "y", "z"]),+    ],+)+def test_filter_positional_bools_select_labels(ser, mask):+    # GH#61317+    msg = "A list-like of booleans passed positionally to Series.filter"+    with tm.assert_produces_warning(UserWarning, match=msg):+        result = ser.filter(mask)+    expected = ser.iloc[[]]+    tm.assert_series_equal(result, expected)++    result = ser.filter(items=mask)+    tm.assert_series_equal(result, expected)+++def test_filter_bool_labels():+    # GH#61317+    ser = pd.Series([1, 2, 3], index=[True, False, "c"])+    result = ser.filter(items=[True, False])+    expected = ser.iloc[:2]+    tm.assert_series_equal(result, expected)++    msg = "A list-like of booleans passed positionally to Series.filter"+    with tm.assert_produces_warning(UserWarning, match=msg):+        result = ser.filter([True, False])+    tm.assert_series_equal(result, expected)++    result = ser.filter(cond=[False, True, True])+    expected = ser.iloc[1:]+    tm.assert_series_equal(result, expected)+++def test_filter_cond_series_aligns(ser):+    # GH#61317+    mask = pd.Series([True, True, False], index=["z", "y", "x"])+    result = ser.filter(cond=mask)+    expected = ser.iloc[1:]+    tm.assert_series_equal(result, expected)+++def test_filter_expression_raises(ser):+    # GH#61317+    msg = "Expressions such as pd.col\\(...\\) are only supported by DataFrame.filter"+    with pytest.raises(TypeError, match=msg):+        ser.filter(pd.col("a") > 1)+    with pytest.raises(TypeError, match=msg):+        ser.filter(cond=pd.col("a") > 1)+++def test_filter_callable_must_return_mask(ser):+    # GH#61317+    msg = "The callable passed to Series.filter must evaluate to a one-dimensional"+    with pytest.raises(TypeError, match=msg):+        ser.filter(lambda ser: ["x"])+++def test_filter_cond_not_mask_raises(ser):+    # GH#61317+    msg = "cond passed to Series.filter must be a one-dimensional boolean mask"+    with pytest.raises(TypeError, match=msg):+        ser.filter(cond=["x"])+++def test_filter_cond_2d_raises(ser):+    # GH#61317+    mask = pd.DataFrame({"a": [False, True, True]}, index=["x", "y", "z"])+    msg = "cond passed to Series.filter must be a one-dimensional boolean mask"+    with pytest.raises(TypeError, match=msg):+        ser.filter(cond=mask)+++@pytest.mark.parametrize(+    "mask",+    [+        [True, None, False],+        pd.array([True, None, False], dtype="boolean"),+    ],+)+def test_filter_cond_na(ser, mask):+    # GH#61317+    result = ser.filter(cond=mask)+    expected = ser.iloc[[0]]+    tm.assert_series_equal(result, expected)++    result = ser.filter(cond=mask, na=False)+    tm.assert_series_equal(result, expected)++    msg = "The mask contains missing values"+    with pytest.raises(ValueError, match=msg):+        ser.filter(cond=mask, na="raise")++    result = ser.filter(cond=mask, na=True)+    expected = ser.iloc[[0, 1]]+    tm.assert_series_equal(result, expected)+++def test_filter_na_label():+    # GH#61317+    ser = pd.Series([1, 2], index=[np.nan, "x"])+    result = ser.filter(items=[np.nan])+    expected = ser.iloc[[0]]+    tm.assert_series_equal(result, expected, check_index_type=False)+++def test_filter_tuple_labels_multiindex():+    # GH#61317+    mi = pd.MultiIndex.from_tuples([(True, False), (False, True)])+    ser = pd.Series([1, 2], index=mi)+    result = ser.filter([(True, False)])+    expected = ser.iloc[[0]]+    tm.assert_series_equal(result, expected)+++@pytest.mark.parametrize("kwargs", [{"items": ["x"]}, {"like": "x"}, {"regex": "x"}])+def test_filter_labels(ser, kwargs):+    # GH#61317+    result = ser.filter(**kwargs)+    expected = ser.iloc[[0]]+    tm.assert_series_equal(result, expected)