Skip to content

Commit cd0dd3a

Browse files
committed
BUG: Raise OutOfBoundsDatetime when replacing with out-of-bounds datetime64[ns] (GH#61671)
1 parent cfe54bd commit cd0dd3a

File tree

3 files changed

+32
-0
lines changed

3 files changed

+32
-0
lines changed

doc/source/whatsnew/v3.0.0.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -685,6 +685,7 @@ Datetimelike
685685
- Bug in :func:`tseries.frequencies.to_offset` would fail to parse frequency strings starting with "LWOM" (:issue:`59218`)
686686
- Bug in :meth:`DataFrame.fillna` raising an ``AssertionError`` instead of ``OutOfBoundsDatetime`` when filling a ``datetime64[ns]`` column with an out-of-bounds timestamp. Now correctly raises ``OutOfBoundsDatetime``. (:issue:`61208`)
687687
- Bug in :meth:`DataFrame.min` and :meth:`DataFrame.max` casting ``datetime64`` and ``timedelta64`` columns to ``float64`` and losing precision (:issue:`60850`)
688+
- Bug in :meth:`DataFrame.replace` where attempting to replace a ``datetime64[ns]`` column with an out-of-bounds timestamp would raise an ``AssertionError`` or silently coerce. Now correctly raises ``OutOfBoundsDatetime``. (:issue:`61671`)
688689
- Bug in :meth:`Dataframe.agg` with df with missing values resulting in IndexError (:issue:`58810`)
689690
- Bug in :meth:`DatetimeIndex.is_year_start` and :meth:`DatetimeIndex.is_quarter_start` does not raise on Custom business days frequencies bigger then "1C" (:issue:`58664`)
690691
- Bug in :meth:`DatetimeIndex.is_year_start` and :meth:`DatetimeIndex.is_quarter_start` returning ``False`` on double-digit frequencies (:issue:`58523`)

pandas/core/dtypes/cast.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@
5353
ensure_str,
5454
is_bool,
5555
is_complex,
56+
is_datetime64_dtype,
5657
is_float,
5758
is_integer,
5859
is_object_dtype,
@@ -1358,6 +1359,24 @@ def find_result_type(left_dtype: DtypeObj, right: Any) -> DtypeObj:
13581359
dtype, _ = infer_dtype_from(right)
13591360
new_dtype = find_common_type([left_dtype, dtype])
13601361

1362+
# special case: datetime64[ns] inferred but the value may be out of bounds
1363+
if (
1364+
is_datetime64_dtype(new_dtype)
1365+
and lib.is_scalar(right)
1366+
and isinstance(right, (np.datetime64, dt.datetime, Timestamp))
1367+
):
1368+
try:
1369+
ts = Timestamp(right)
1370+
casted = np.datetime64(ts, "ns")
1371+
if Timestamp(casted) != ts:
1372+
raise OutOfBoundsDatetime(
1373+
f"{right!r} overflows datetime64[ns] during dtype inference"
1374+
)
1375+
except (OverflowError, ValueError) as e:
1376+
raise OutOfBoundsDatetime(
1377+
f"Cannot safely store {right!r} in inferred dtype 'datetime64[ns]'"
1378+
) from e
1379+
13611380
return new_dtype
13621381

13631382

pandas/tests/frame/methods/test_replace.py

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1430,6 +1430,18 @@ def test_replace_with_nil_na(self):
14301430
result = ser.replace("nil", "anything else")
14311431
tm.assert_frame_equal(expected, result)
14321432

1433+
def test_replace_outofbounds_datetime_raises(self):
1434+
# GH 61671
1435+
df = DataFrame([np.nan], dtype="datetime64[ns]")
1436+
too_big = datetime(3000, 1, 1)
1437+
from pandas.errors import OutOfBoundsDatetime
1438+
1439+
with pytest.raises(
1440+
OutOfBoundsDatetime,
1441+
match="Cannot safely store .* in inferred dtype 'datetime64\\[ns\\]'",
1442+
):
1443+
df.replace(np.nan, too_big)
1444+
14331445

14341446
class TestDataFrameReplaceRegex:
14351447
@pytest.mark.parametrize(

0 commit comments

Comments
 (0)