Skip to content

Commit e045526

Browse files
TomAugspurgerproost
TomAugspurger
authored and
proost
committed
DEPR: Remove SparseSeries and SparseDataFrame (pandas-dev#28425)
1 parent 06c8584 commit e045526

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

74 files changed

+227
-8569
lines changed

‎doc/redirects.csv

-2
Original file line numberDiff line numberDiff line change
@@ -503,7 +503,6 @@ generated/pandas.DataFrame.to_parquet,../reference/api/pandas.DataFrame.to_parqu
503503
generated/pandas.DataFrame.to_period,../reference/api/pandas.DataFrame.to_period
504504
generated/pandas.DataFrame.to_pickle,../reference/api/pandas.DataFrame.to_pickle
505505
generated/pandas.DataFrame.to_records,../reference/api/pandas.DataFrame.to_records
506-
generated/pandas.DataFrame.to_sparse,../reference/api/pandas.DataFrame.to_sparse
507506
generated/pandas.DataFrame.to_sql,../reference/api/pandas.DataFrame.to_sql
508507
generated/pandas.DataFrame.to_stata,../reference/api/pandas.DataFrame.to_stata
509508
generated/pandas.DataFrame.to_string,../reference/api/pandas.DataFrame.to_string
@@ -1432,7 +1431,6 @@ generated/pandas.Series.to_msgpack,../reference/api/pandas.Series.to_msgpack
14321431
generated/pandas.Series.to_numpy,../reference/api/pandas.Series.to_numpy
14331432
generated/pandas.Series.to_period,../reference/api/pandas.Series.to_period
14341433
generated/pandas.Series.to_pickle,../reference/api/pandas.Series.to_pickle
1435-
generated/pandas.Series.to_sparse,../reference/api/pandas.Series.to_sparse
14361434
generated/pandas.Series.to_sql,../reference/api/pandas.Series.to_sql
14371435
generated/pandas.Series.to_string,../reference/api/pandas.Series.to_string
14381436
generated/pandas.Series.to_timestamp,../reference/api/pandas.Series.to_timestamp

‎doc/source/reference/frame.rst

-8
Original file line numberDiff line numberDiff line change
@@ -356,15 +356,7 @@ Serialization / IO / conversion
356356
DataFrame.to_msgpack
357357
DataFrame.to_gbq
358358
DataFrame.to_records
359-
DataFrame.to_sparse
360359
DataFrame.to_dense
361360
DataFrame.to_string
362361
DataFrame.to_clipboard
363362
DataFrame.style
364-
365-
Sparse
366-
~~~~~~
367-
.. autosummary::
368-
:toctree: api/
369-
370-
SparseDataFrame.to_coo

‎doc/source/reference/series.rst

-11
Original file line numberDiff line numberDiff line change
@@ -576,18 +576,7 @@ Serialization / IO / conversion
576576
Series.to_sql
577577
Series.to_msgpack
578578
Series.to_json
579-
Series.to_sparse
580579
Series.to_dense
581580
Series.to_string
582581
Series.to_clipboard
583582
Series.to_latex
584-
585-
586-
Sparse
587-
------
588-
589-
.. autosummary::
590-
:toctree: api/
591-
592-
SparseSeries.to_coo
593-
SparseSeries.from_coo

‎doc/source/user_guide/io.rst

+8
Original file line numberDiff line numberDiff line change
@@ -4641,6 +4641,14 @@ Several caveats.
46414641

46424642
See the `Full Documentation <https://github.com./wesm/feather>`__.
46434643

4644+
.. ipython:: python
4645+
:suppress:
4646+
4647+
import warnings
4648+
# This can be removed once building with pyarrow >=0.15.0
4649+
warnings.filterwarnings("ignore", "The Sparse", FutureWarning)
4650+
4651+
46444652
.. ipython:: python
46454653
46464654
df = pd.DataFrame({'a': list('abc'),

‎doc/source/user_guide/sparse.rst

+5-15
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,6 @@
66
Sparse data structures
77
**********************
88

9-
.. note::
10-
11-
``SparseSeries`` and ``SparseDataFrame`` have been deprecated. Their purpose
12-
is served equally well by a :class:`Series` or :class:`DataFrame` with
13-
sparse values. See :ref:`sparse.migration` for tips on migrating.
14-
159
Pandas provides data structures for efficiently storing sparse data.
1610
These are not necessarily sparse in the typical "mostly 0". Rather, you can view these
1711
objects as being "compressed" where any data matching a specific value (``NaN`` / missing value, though any value
@@ -168,6 +162,11 @@ the correct dense result.
168162
Migrating
169163
---------
170164

165+
.. note::
166+
167+
``SparseSeries`` and ``SparseDataFrame`` were removed in pandas 1.0.0. This migration
168+
guide is present to aid in migrating from previous versions.
169+
171170
In older versions of pandas, the ``SparseSeries`` and ``SparseDataFrame`` classes (documented below)
172171
were the preferred way to work with sparse data. With the advent of extension arrays, these subclasses
173172
are no longer needed. Their purpose is better served by using a regular Series or DataFrame with
@@ -366,12 +365,3 @@ row and columns coordinates of the matrix. Note that this will consume a signifi
366365
367366
ss_dense = pd.Series.sparse.from_coo(A, dense_index=True)
368367
ss_dense
369-
370-
371-
.. _sparse.subclasses:
372-
373-
Sparse subclasses
374-
-----------------
375-
376-
The :class:`SparseSeries` and :class:`SparseDataFrame` classes are deprecated. Visit their
377-
API pages for usage.

‎doc/source/whatsnew/v0.16.0.rst

+2-4
Original file line numberDiff line numberDiff line change
@@ -91,8 +91,7 @@ Interaction with scipy.sparse
9191

9292
Added :meth:`SparseSeries.to_coo` and :meth:`SparseSeries.from_coo` methods (:issue:`8048`) for converting to and from ``scipy.sparse.coo_matrix`` instances (see :ref:`here <sparse.scipysparse>`). For example, given a SparseSeries with MultiIndex we can convert to a `scipy.sparse.coo_matrix` by specifying the row and column labels as index levels:
9393

94-
.. ipython:: python
95-
:okwarning:
94+
.. code-block:: python
9695
9796
s = pd.Series([3.0, np.nan, 1.0, 3.0, np.nan, np.nan])
9897
s.index = pd.MultiIndex.from_tuples([(1, 2, 'a', 0),
@@ -121,8 +120,7 @@ Added :meth:`SparseSeries.to_coo` and :meth:`SparseSeries.from_coo` methods (:is
121120
The from_coo method is a convenience method for creating a ``SparseSeries``
122121
from a ``scipy.sparse.coo_matrix``:
123122

124-
.. ipython:: python
125-
:okwarning:
123+
.. code-block:: python
126124
127125
from scipy import sparse
128126
A = sparse.coo_matrix(([3.0, 1.0, 2.0], ([1, 0, 0], [0, 2, 3])),

‎doc/source/whatsnew/v0.18.1.rst

+2-4
Original file line numberDiff line numberDiff line change
@@ -393,8 +393,7 @@ used in the ``pandas`` implementation (:issue:`12644`, :issue:`12638`, :issue:`1
393393

394394
An example of this signature augmentation is illustrated below:
395395

396-
.. ipython:: python
397-
:okwarning:
396+
.. code-block:: python
398397
399398
sp = pd.SparseDataFrame([1, 2, 3])
400399
sp
@@ -409,8 +408,7 @@ Previous behaviour:
409408
410409
New behaviour:
411410

412-
.. ipython:: python
413-
:okwarning:
411+
.. code-block:: python
414412
415413
np.cumsum(sp, axis=0)
416414

‎doc/source/whatsnew/v0.19.0.rst

+2-4
Original file line numberDiff line numberDiff line change
@@ -1235,8 +1235,7 @@ Operators now preserve dtypes
12351235

12361236
- Sparse data structure now can preserve ``dtype`` after arithmetic ops (:issue:`13848`)
12371237

1238-
.. ipython:: python
1239-
:okwarning:
1238+
.. code-block:: python
12401239
12411240
s = pd.SparseSeries([0, 2, 0, 1], fill_value=0, dtype=np.int64)
12421241
s.dtype
@@ -1245,8 +1244,7 @@ Operators now preserve dtypes
12451244
12461245
- Sparse data structure now support ``astype`` to convert internal ``dtype`` (:issue:`13900`)
12471246

1248-
.. ipython:: python
1249-
:okwarning:
1247+
.. code-block:: python
12501248
12511249
s = pd.SparseSeries([1., 0., 2., 0.], fill_value=0)
12521250
s

‎doc/source/whatsnew/v0.20.0.rst

+2-3
Original file line numberDiff line numberDiff line change
@@ -338,8 +338,7 @@ See the :ref:`documentation <sparse.scipysparse>` for more information. (:issue:
338338

339339
All sparse formats are supported, but matrices that are not in :mod:`COOrdinate <scipy.sparse>` format will be converted, copying data as needed.
340340

341-
.. ipython:: python
342-
:okwarning:
341+
.. code-block:: python
343342
344343
from scipy.sparse import csr_matrix
345344
arr = np.random.random(size=(1000, 5))
@@ -351,7 +350,7 @@ All sparse formats are supported, but matrices that are not in :mod:`COOrdinate
351350
352351
To convert a ``SparseDataFrame`` back to sparse SciPy matrix in COO format, you can use:
353352

354-
.. ipython:: python
353+
.. code-block:: python
355354
356355
sdf.to_coo()
357356

‎doc/source/whatsnew/v0.25.0.rst

+1-2
Original file line numberDiff line numberDiff line change
@@ -902,8 +902,7 @@ by a ``Series`` or ``DataFrame`` with sparse values.
902902
903903
**Previous way**
904904
905-
.. ipython:: python
906-
:okwarning:
905+
.. code-block:: python
907906
908907
df = pd.SparseDataFrame({"A": [0, 0, 1, 2]})
909908
df.dtypes

‎doc/source/whatsnew/v1.0.0.rst

+9
Original file line numberDiff line numberDiff line change
@@ -92,8 +92,17 @@ Deprecations
9292

9393
.. _whatsnew_1000.prior_deprecations:
9494

95+
96+
Removed SparseSeries and SparseDataFrame
97+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
98+
99+
``SparseSeries`` and ``SparseDataFrame`` have been removed (:issue:`28425`).
100+
We recommend using a ``Series`` or ``DataFrame`` with sparse values instead.
101+
See :ref:`sparse.migration` for help with migrating existing code.
102+
95103
Removal of prior version deprecations/changes
96104
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
105+
97106
- Removed the previously deprecated :meth:`Series.get_value`, :meth:`Series.set_value`, :meth:`DataFrame.get_value`, :meth:`DataFrame.set_value` (:issue:`17739`)
98107
- Changed the the default value of `inplace` in :meth:`DataFrame.set_index` and :meth:`Series.set_axis`. It now defaults to False (:issue:`27600`)
99108
- :meth:`pandas.Series.str.cat` now defaults to aligning ``others``, using ``join='left'`` (:issue:`27611`)

‎pandas/__init__.py

+20-7
Original file line numberDiff line numberDiff line change
@@ -114,12 +114,7 @@
114114
DataFrame,
115115
)
116116

117-
from pandas.core.sparse.api import (
118-
SparseArray,
119-
SparseDataFrame,
120-
SparseSeries,
121-
SparseDtype,
122-
)
117+
from pandas.core.sparse.api import SparseArray, SparseDtype
123118

124119
from pandas.tseries.api import infer_freq
125120
from pandas.tseries import offsets
@@ -196,8 +191,9 @@
196191
if pandas.compat.PY37:
197192

198193
def __getattr__(name):
194+
import warnings
195+
199196
if name == "Panel":
200-
import warnings
201197

202198
warnings.warn(
203199
"The Panel class is removed from pandas. Accessing it "
@@ -211,6 +207,17 @@ class Panel:
211207
pass
212208

213209
return Panel
210+
elif name in {"SparseSeries", "SparseDataFrame"}:
211+
warnings.warn(
212+
"The {} class is removed from pandas. Accessing it from "
213+
"the top-level namespace will also be removed in the next "
214+
"version".format(name),
215+
FutureWarning,
216+
stacklevel=2,
217+
)
218+
219+
return type(name, (), {})
220+
214221
raise AttributeError("module 'pandas' has no attribute '{}'".format(name))
215222

216223

@@ -219,6 +226,12 @@ class Panel:
219226
class Panel:
220227
pass
221228

229+
class SparseDataFrame:
230+
pass
231+
232+
class SparseSeries:
233+
pass
234+
222235

223236
# module level doc-string
224237
__doc__ = """

‎pandas/_typing.py

+1-4
Original file line numberDiff line numberDiff line change
@@ -12,13 +12,10 @@
1212
from pandas.core.dtypes.dtypes import ExtensionDtype # noqa: F401
1313
from pandas.core.indexes.base import Index # noqa: F401
1414
from pandas.core.series import Series # noqa: F401
15-
from pandas.core.sparse.series import SparseSeries # noqa: F401
1615
from pandas.core.generic import NDFrame # noqa: F401
1716

1817

19-
AnyArrayLike = TypeVar(
20-
"AnyArrayLike", "ExtensionArray", "Index", "Series", "SparseSeries", np.ndarray
21-
)
18+
AnyArrayLike = TypeVar("AnyArrayLike", "ExtensionArray", "Index", "Series", np.ndarray)
2219
ArrayLike = TypeVar("ArrayLike", "ExtensionArray", np.ndarray)
2320
DatetimeLikeScalar = TypeVar("DatetimeLikeScalar", "Period", "Timestamp", "Timedelta")
2421
Dtype = Union[str, np.dtype, "ExtensionDtype"]

‎pandas/compat/pickle_compat.py

+51-3
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,14 @@
55
import copy
66
import pickle as pkl
77
import sys
8+
from typing import TYPE_CHECKING
9+
import warnings
810

911
from pandas import Index
1012

13+
if TYPE_CHECKING:
14+
from pandas import Series, DataFrame
15+
1116

1217
def load_reduce(self):
1318
stack = self.stack
@@ -54,6 +59,41 @@ def load_reduce(self):
5459
raise
5560

5661

62+
_sparse_msg = """\
63+
64+
Loading a saved '{cls}' as a {new} with sparse values.
65+
'{cls}' is now removed. You should re-save this dataset in its new format.
66+
"""
67+
68+
69+
class _LoadSparseSeries:
70+
# To load a SparseSeries as a Series[Sparse]
71+
def __new__(cls) -> "Series":
72+
from pandas import Series
73+
74+
warnings.warn(
75+
_sparse_msg.format(cls="SparseSeries", new="Series"),
76+
FutureWarning,
77+
stacklevel=6,
78+
)
79+
80+
return Series()
81+
82+
83+
class _LoadSparseFrame:
84+
# To load a SparseDataFrame as a DataFrame[Sparse]
85+
def __new__(cls) -> "DataFrame":
86+
from pandas import DataFrame
87+
88+
warnings.warn(
89+
_sparse_msg.format(cls="SparseDataFrame", new="DataFrame"),
90+
FutureWarning,
91+
stacklevel=6,
92+
)
93+
94+
return DataFrame()
95+
96+
5797
# If classes are moved, provide compat here.
5898
_class_locations_map = {
5999
("pandas.core.sparse.array", "SparseArray"): ("pandas.core.arrays", "SparseArray"),
@@ -101,12 +141,12 @@ def load_reduce(self):
101141
"SparseArray",
102142
),
103143
("pandas.sparse.series", "SparseSeries"): (
104-
"pandas.core.sparse.series",
105-
"SparseSeries",
144+
"pandas.compat.pickle_compat",
145+
"_LoadSparseSeries",
106146
),
107147
("pandas.sparse.frame", "SparseDataFrame"): (
108148
"pandas.core.sparse.frame",
109-
"SparseDataFrame",
149+
"_LoadSparseFrame",
110150
),
111151
("pandas.indexes.base", "_new_Index"): ("pandas.core.indexes.base", "_new_Index"),
112152
("pandas.indexes.base", "Index"): ("pandas.core.indexes.base", "Index"),
@@ -139,6 +179,14 @@ def load_reduce(self):
139179
"pandas.core.indexes.numeric",
140180
"Float64Index",
141181
),
182+
("pandas.core.sparse.series", "SparseSeries"): (
183+
"pandas.compat.pickle_compat",
184+
"_LoadSparseSeries",
185+
),
186+
("pandas.core.sparse.frame", "SparseDataFrame"): (
187+
"pandas.compat.pickle_compat",
188+
"_LoadSparseFrame",
189+
),
142190
}
143191

144192

0 commit comments

Comments
 (0)