Skip to content

Conversation

@briangow
Copy link
Contributor

@briangow briangow commented Oct 13, 2025

The patch makes the label‑store dtype explicit (int64) and replaces row‑wise .loc assignment with a keyed, vectorized update.

@briangow briangow force-pushed the bg_fix_test_overflow branch from e3a23f4 to 43cf077 Compare October 13, 2025 16:28
@briangow briangow changed the title [DRAFT] Fix annotation test overflow error Fix OverflowError: Python int too large to convert to C long raised during annotation label map creation with custom labels. Oct 13, 2025
@bemoody
Copy link
Collaborator

bemoody commented Oct 17, 2025 via email

@briangow
Copy link
Contributor Author

@bemoody this is to fix the test error currently seen in #544. My understanding is that this is a known issue when Pandas tries to unify data types. Here is the explanation (along with the full test error):

WFDB’s own constraints keep label_store small (1–49), so it isn’t your annotation values that are “too big.” The overflow comes from pandas’ internal attempt to unify dtypes when a row assignment ends up behaving like an append under the hood. On some platforms/pandas versions, that path tries to convert Python ints through a C long and can raise OverflowError even if your data are reasonable.

=================================== FAILURES ===================================
____________________________ TestAnnotation.test_3 _____________________________

self = <tests.test_annotation.TestAnnotation testMethod=test_3>

    def test_3(self):
        """
        Annotation file with custom annotation types
    
        Target file created with:
            rdann -r sample-data/1003 -a atr > ann-3
        """
>       annotation = wfdb.rdann("sample-data/1003", "atr")

tests/test_annotation.py:183: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
wfdb/io/annotation.py:2028: in rdann
    annotation.set_label_elements(return_label_elements)
wfdb/io/annotation.py:1408: in set_label_elements
    self.convert_label_attribute(contained_elements[0], e)
wfdb/io/annotation.py:1471: in convert_label_attribute
    label_map = self.create_label_map(inplace=False)
wfdb/io/annotation.py:911: in create_label_map
    label_map.loc[i] = self.custom_labels.loc[i]
.venv/lib/python3.9/site-packages/pandas/core/indexing.py:912: in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)
.venv/lib/python3.9/site-packages/pandas/core/indexing.py:1933: in _setitem_with_indexer
    self._setitem_with_indexer_missing(indexer, value)
.venv/lib/python3.9/site-packages/pandas/core/indexing.py:2329: in _setitem_with_indexer_missing
    self.obj._mgr = self.obj._append(value)._mgr
.venv/lib/python3.9/site-packages/pandas/core/frame.py:10581: in _append
    other = row_df.infer_objects(copy=False).rename_axis(
.venv/lib/python3.9/site-packages/pandas/core/generic.py:6910: in infer_objects
    new_mgr = self._mgr.convert(copy=copy)
.venv/lib/python3.9/site-packages/pandas/core/internals/managers.py:466: in convert
    return self.apply("convert", copy=copy, using_cow=using_copy_on_write())
.venv/lib/python3.9/site-packages/pandas/core/internals/managers.py:363: in apply
    applied = getattr(b, f)(**kwargs)
.venv/lib/python3.9/site-packages/pandas/core/internals/blocks.py:656: in convert
    blocks = self.split_and_operate(
.venv/lib/python3.9/site-packages/pandas/core/internals/blocks.py:473: in split_and_operate
    rbs = func(nb, *args, **kwargs)
.venv/lib/python3.9/site-packages/pandas/core/internals/blocks.py:675: in convert
    res_values = lib.maybe_convert_objects(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   OverflowError: Python int too large to convert to C long

pandas/_libs/lib.pyx:2619: OverflowError
=========================== short test summary info ============================
FAILED tests/test_annotation.py::TestAnnotation::test_3 - OverflowError: Python int too large to convert to C long
=================== 1 failed, 69 passed in 281.48s (0:04:41) ===================
Error: Process completed with exit code 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants