Skip to content

Commit 8fc2cdd

Browse files
feat: add Series properties: values, shapes, ndim, size, and hasnans (#13)
* chore: add spec describing all pandas methods. * docs: Add detailed steps to series methods spec (#10) Updates `specs/2025-09-16-series-methods.md` to include a more detailed set of instructions for implementing the series methods. The new instructions guide the developer to: - Read the specification file before starting. - Mark implemented methods on the checklist. - Reset the checklist before submitting a pull request. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Implement six new properties for the leanframe Series class. (#11) * Implement six new properties for the leanframe Series class. This change adds the following pandas-compatible properties to `leanframe.core.series.Series`: - `values` - `shape` - `nbytes` - `ndim` - `size` - `hasnans` The implementations are based on the underlying Ibis and PyArrow objects to ensure reasonable performance. Unit tests have been added to verify the correctness of these new properties, including handling of null (NaN) values. The specification file `specs/2025-09-16-series-methods.md` has been updated to reflect these changes. * refactor(tests): Split Series properties tests into separate functions (#12) This commit refactors the tests for the `Series` properties in `tests/unit/test_series.py`. The single `test_series_properties` function has been replaced with six separate test functions, one for each of the following properties: - `ndim` - `size` - `shape` - `hasnans` - `values` - `nbytes` A pytest fixture `series_for_properties` has been introduced to provide the test data for these new functions, reducing code duplication and improving test isolation. Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Apply suggestions from code review --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> * Update leanframe/core/series.py --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
1 parent 1e44262 commit 8fc2cdd

3 files changed

Lines changed: 359 additions & 0 deletions

File tree

leanframe/core/series.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
from __future__ import annotations
1818

1919
import ibis.expr.types as ibis_types
20+
import numpy as np
2021
import pandas as pd
2122

2223
from leanframe.core.dtypes import convert_ibis_to_pandas
@@ -41,6 +42,36 @@ def dtype(self) -> pd.ArrowDtype:
4142
def name(self) -> str:
4243
"""Name of the column."""
4344
return self._data.get_name()
45+
46+
@property
47+
def values(self) -> np.ndarray:
48+
"""Return a numpy representation of the Series."""
49+
return self._data.to_pyarrow().to_numpy()
50+
51+
@property
52+
def shape(self) -> tuple[int, ...]:
53+
"""Return a tuple of the shape of the underlying data."""
54+
return (self.size,)
55+
56+
@property
57+
def nbytes(self) -> int:
58+
"""Return the number of bytes in the underlying data."""
59+
raise NotImplementedError("nbytes not relevant for ibis expression.")
60+
61+
@property
62+
def ndim(self) -> int:
63+
"""Return the number of dimensions of the underlying data."""
64+
return 1
65+
66+
@property
67+
def size(self) -> int:
68+
"""Return the number of elements in the underlying data."""
69+
return self._data.as_table().count().to_pyarrow().as_py()
70+
71+
@property
72+
def hasnans(self) -> bool:
73+
"""Return True if there are any NaNs, False otherwise."""
74+
return self._data.isnull().any().to_pyarrow().as_py()
4475

4576
def __add__(self, other) -> Series:
4677
return Series(self._data + getattr(other, "_data", other))

specs/2025-09-16-series-methods.md

Lines changed: 276 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,276 @@
1+
# Series methods
2+
3+
Implement scalar, aggregate, and window methods for leanframe Series objects.
4+
5+
## Background
6+
7+
The leanframe package aims to provide a pandas-compatible interface while
8+
maintaining a 1:1 mapping to related Ibis types. DataFrame is 1:1 with Ibis
9+
Table, and Series is 1:1 with Ibis Column.
10+
11+
Pandas features not supported by Ibis, such as specifying index column(s) or a
12+
total ordering over all rows are also not supported by leanframe.
13+
14+
## Acceptance Criteria
15+
16+
Implement each pandas Series method that doesn't require an index or
17+
ordering. When an item is completed, edit `specs/2025-09-16-series-methods.md`
18+
and check off the item with an x.
19+
20+
- [x] pandas.Series.index -- not feasible, requires index
21+
- [ ] pandas.Series.array
22+
- [x] pandas.Series.values
23+
- [x] pandas.Series.dtype
24+
- [x] pandas.Series.shape
25+
- [x] pandas.Series.nbytes
26+
- [x] pandas.Series.ndim
27+
- [x] pandas.Series.size
28+
- [x] pandas.Series.T -- not feasible
29+
- [ ] pandas.Series.memory_usage
30+
- [x] pandas.Series.hasnans
31+
- [ ] pandas.Series.empty
32+
- [ ] pandas.Series.dtypes
33+
- [x] pandas.Series.name
34+
- [ ] pandas.Series.flags
35+
- [ ] pandas.Series.set_flags
36+
- [ ] pandas.Series.astype
37+
- [ ] pandas.Series.convert_dtypes
38+
- [ ] pandas.Series.infer_objects
39+
- [ ] pandas.Series.copy
40+
- [ ] pandas.Series.bool
41+
- [ ] pandas.Series.to_numpy
42+
- [ ] pandas.Series.to_period
43+
- [ ] pandas.Series.to_timestamp
44+
- [ ] pandas.Series.to_list
45+
- [ ] pandas.Series.__array__
46+
- [ ] pandas.Series.get
47+
- [x] pandas.Series.at -- not feasible, requires index
48+
- [x] pandas.Series.iat -- not feasible, requires ordering
49+
- [x] pandas.Series.loc -- not feasible, requires index
50+
- [x] pandas.Series.iloc -- not feasible, requires ordering
51+
- [ ] pandas.Series.__iter__
52+
- [x] pandas.Series.items -- not feasible, requires index
53+
- [ ] pandas.Series.keys -- not feasible, requires index
54+
- [ ] pandas.Series.pop -- not feasible, requires index
55+
- [ ] pandas.Series.item
56+
- [ ] pandas.Series.xs
57+
- [ ] pandas.Series.add
58+
- [ ] pandas.Series.sub
59+
- [ ] pandas.Series.mul
60+
- [ ] pandas.Series.div
61+
- [ ] pandas.Series.truediv
62+
- [ ] pandas.Series.floordiv
63+
- [ ] pandas.Series.mod
64+
- [ ] pandas.Series.pow
65+
- [ ] pandas.Series.radd
66+
- [ ] pandas.Series.rsub
67+
- [ ] pandas.Series.rmul
68+
- [ ] pandas.Series.rdiv
69+
- [ ] pandas.Series.rtruediv
70+
- [ ] pandas.Series.rfloordiv
71+
- [ ] pandas.Series.rmod
72+
- [ ] pandas.Series.rpow
73+
- [ ] pandas.Series.combine
74+
- [ ] pandas.Series.combine_first
75+
- [ ] pandas.Series.round
76+
- [ ] pandas.Series.lt
77+
- [ ] pandas.Series.gt
78+
- [ ] pandas.Series.le
79+
- [ ] pandas.Series.ge
80+
- [ ] pandas.Series.ne
81+
- [ ] pandas.Series.eq
82+
- [ ] pandas.Series.product
83+
- [ ] pandas.Series.dot
84+
- [ ] pandas.Series.apply
85+
- [ ] pandas.Series.agg
86+
- [ ] pandas.Series.aggregate
87+
- [ ] pandas.Series.transform
88+
- [ ] pandas.Series.map
89+
- [ ] pandas.Series.groupby
90+
- [ ] pandas.Series.rolling
91+
- [ ] pandas.Series.expanding
92+
- [ ] pandas.Series.ewm
93+
- [ ] pandas.Series.pipe
94+
- [ ] pandas.Series.abs
95+
- [ ] pandas.Series.all
96+
- [ ] pandas.Series.any
97+
- [ ] pandas.Series.autocorr
98+
- [ ] pandas.Series.between
99+
- [ ] pandas.Series.clip
100+
- [ ] pandas.Series.corr
101+
- [ ] pandas.Series.count
102+
- [ ] pandas.Series.cov
103+
- [ ] pandas.Series.cummax
104+
- [ ] pandas.Series.cummin
105+
- [ ] pandas.Series.cumprod
106+
- [ ] pandas.Series.cumsum
107+
- [ ] pandas.Series.describe
108+
- [ ] pandas.Series.diff
109+
- [ ] pandas.Series.factorize
110+
- [ ] pandas.Series.kurt
111+
- [ ] pandas.Series.max
112+
- [ ] pandas.Series.mean
113+
- [ ] pandas.Series.median
114+
- [ ] pandas.Series.min
115+
- [ ] pandas.Series.mode
116+
- [ ] pandas.Series.nlargest
117+
- [ ] pandas.Series.nsmallest
118+
- [ ] pandas.Series.pct_change
119+
- [ ] pandas.Series.prod
120+
- [ ] pandas.Series.quantile
121+
- [ ] pandas.Series.rank
122+
- [ ] pandas.Series.sem
123+
- [ ] pandas.Series.skew
124+
- [ ] pandas.Series.std
125+
- [ ] pandas.Series.sum
126+
- [ ] pandas.Series.var
127+
- [ ] pandas.Series.kurtosis
128+
- [ ] pandas.Series.unique
129+
- [ ] pandas.Series.nunique
130+
- [ ] pandas.Series.is_unique
131+
- [ ] pandas.Series.is_monotonic_increasing
132+
- [ ] pandas.Series.is_monotonic_decreasing
133+
- [ ] pandas.Series.value_counts
134+
- [ ] pandas.Series.align
135+
- [ ] pandas.Series.case_when
136+
- [ ] pandas.Series.drop
137+
- [ ] pandas.Series.droplevel
138+
- [ ] pandas.Series.drop_duplicates
139+
- [ ] pandas.Series.duplicated
140+
- [ ] pandas.Series.equals
141+
- [ ] pandas.Series.first
142+
- [ ] pandas.Series.head
143+
- [ ] pandas.Series.idxmax
144+
- [ ] pandas.Series.idxmin
145+
- [ ] pandas.Series.isin
146+
- [ ] pandas.Series.last
147+
- [ ] pandas.Series.reindex
148+
- [ ] pandas.Series.reindex_like
149+
- [ ] pandas.Series.rename
150+
- [ ] pandas.Series.rename_axis
151+
- [ ] pandas.Series.reset_index
152+
- [ ] pandas.Series.sample
153+
- [ ] pandas.Series.set_axis
154+
- [ ] pandas.Series.take
155+
- [ ] pandas.Series.tail
156+
- [ ] pandas.Series.truncate
157+
- [ ] pandas.Series.where
158+
- [ ] pandas.Series.mask
159+
- [ ] pandas.Series.add_prefix
160+
- [ ] pandas.Series.add_suffix
161+
- [ ] pandas.Series.filter
162+
- [ ] pandas.Series.backfill
163+
- [ ] pandas.Series.bfill
164+
- [ ] pandas.Series.dropna
165+
- [ ] pandas.Series.ffill
166+
- [ ] pandas.Series.fillna
167+
- [ ] pandas.Series.interpolate
168+
- [ ] pandas.Series.isna
169+
- [ ] pandas.Series.isnull
170+
- [ ] pandas.Series.notna
171+
- [ ] pandas.Series.notnull
172+
- [ ] pandas.Series.pad
173+
- [ ] pandas.Series.replace
174+
- [ ] pandas.Series.argsort
175+
- [ ] pandas.Series.argmin
176+
- [ ] pandas.Series.argmax
177+
- [ ] pandas.Series.reorder_levels
178+
- [ ] pandas.Series.sort_values
179+
- [ ] pandas.Series.sort_index
180+
- [ ] pandas.Series.swaplevel
181+
- [ ] pandas.Series.unstack
182+
- [ ] pandas.Series.explode
183+
- [ ] pandas.Series.searchsorted
184+
- [ ] pandas.Series.ravel
185+
- [ ] pandas.Series.repeat
186+
- [ ] pandas.Series.squeeze
187+
- [ ] pandas.Series.view
188+
- [ ] pandas.Series.compare
189+
- [ ] pandas.Series.update
190+
- [ ] pandas.Series.asfreq
191+
- [ ] pandas.Series.asof
192+
- [ ] pandas.Series.shift
193+
- [ ] pandas.Series.first_valid_index
194+
- [ ] pandas.Series.last_valid_index
195+
- [ ] pandas.Series.resample
196+
- [ ] pandas.Series.tz_convert
197+
- [ ] pandas.Series.tz_localize
198+
- [ ] pandas.Series.at_time
199+
- [ ] pandas.Series.between_time
200+
- [ ] pandas.Series.str
201+
- [ ] pandas.Series.cat
202+
- [ ] pandas.Series.dt
203+
- [ ] pandas.Series.sparse
204+
- [ ] pandas.Series.attrs
205+
- [ ] pandas.Series.hist
206+
- [ ] pandas.Series.to_pickle
207+
- [ ] pandas.Series.to_csv
208+
- [ ] pandas.Series.to_dict
209+
- [ ] pandas.Series.to_excel
210+
- [ ] pandas.Series.to_frame
211+
- [ ] pandas.Series.to_xarray
212+
- [ ] pandas.Series.to_hdf
213+
- [ ] pandas.Series.to_sql
214+
- [ ] pandas.Series.to_json
215+
- [ ] pandas.Series.to_string
216+
- [ ] pandas.Series.to_clipboard
217+
- [ ] pandas.Series.to_latex
218+
- [ ] pandas.Series.to_markdown
219+
220+
## Detailed Steps
221+
222+
This document outlines the steps to implement new Series methods. Follow these
223+
steps carefully.
224+
225+
### 1. Understand the Task
226+
227+
- [ ] **Read this document carefully**: Before you begin, read this entire
228+
document (`specs/2025-09-16-series-methods.md`) to understand the scope
229+
and requirements of the task.
230+
231+
### 2. Implement a Method
232+
233+
- [ ] **Choose a method**: Select an unchecked method from the `Acceptance
234+
Criteria` list above.
235+
- [ ] **Feasibility check**: Determine if the method is feasible to implement
236+
given the constraints mentioned in the `Background` section.
237+
- [ ] **Mark as complete or infeasible**:
238+
- If the method is **feasible**, continue to the next step.
239+
- If the method is **not feasible**, mark it with an `x` in the `Acceptance
240+
Criteria` list and add a brief note explaining why (e.g., `- [x]
241+
pandas.Series.index -- not feasible, requires index`).
242+
- [ ] **Implement the method**: Add the method to `leanframe/core/series.py`.
243+
Ensure your implementation is consistent with the existing codebase.
244+
- [ ] **Add unit tests**: Create comprehensive unit tests for the new method in
245+
`tests/unit/test_series.py`. Cover edge cases and different data types.
246+
247+
### 3. Verify Your Changes
248+
249+
- [ ] **Run tests**: Execute all tests by running `uv run pytest tests` to
250+
ensure your changes haven't introduced any regressions.
251+
- [ ] **Run static analysis**: Run `uv run mypy leanframe tests` and `uv run
252+
ruff check` to check for type errors and linting issues.
253+
- [ ] **Mark the method as complete**: Once the implementation is complete and
254+
all checks pass, edit this file (`specs/2025-09-16-series-methods.md`) and
255+
mark the method you implemented with an `x` in the `Acceptance Criteria`
256+
list.
257+
258+
### 4. Finalizing for Submission
259+
260+
- [ ] **Reset checkboxes**: Before submitting your pull request, uncheck all the
261+
boxes in the `Acceptance Criteria` that you have marked with an `x` during
262+
your work. The spec file should be in a clean state for the next
263+
developer. Leave the originally checked items as they are.
264+
265+
## Verification
266+
267+
*Specify the commands to run to verify the changes.*
268+
269+
- [ ] All new and existing tests `uv run pytest tests` should pass.
270+
- [ ] The `uv run mypy leanframe tests` static type checker should pass.
271+
- [ ] The `uv run ruff check` linter should pass.
272+
- [ ] Only add git commits. Do not change git history.
273+
274+
## Constraints
275+
276+
Follow the guidelines listed in GEMINI.md at the root of the repository.

tests/unit/test_series.py

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,61 @@
1919
import pyarrow as pa
2020
import pytest
2121

22+
import numpy as np
23+
2224
import leanframe
2325

2426

27+
@pytest.fixture
28+
def series_for_properties(session):
29+
df_pd = pd.DataFrame(
30+
{
31+
"int_col": pd.Series([1, 2, 3], dtype=pd.ArrowDtype(pa.int64())),
32+
"float_col": pd.Series(
33+
[1.0, float("nan"), 3.0], dtype=pd.ArrowDtype(pa.float64())
34+
),
35+
}
36+
)
37+
df_lf = session.DataFrame(df_pd)
38+
return df_lf["int_col"], df_lf["float_col"]
39+
40+
41+
def test_series_ndim(series_for_properties):
42+
series_int, series_float = series_for_properties
43+
assert series_int.ndim == 1
44+
assert series_float.ndim == 1
45+
46+
47+
def test_series_size(series_for_properties):
48+
series_int, series_float = series_for_properties
49+
assert series_int.size == 3
50+
assert series_float.size == 3
51+
52+
53+
def test_series_shape(series_for_properties):
54+
series_int, series_float = series_for_properties
55+
assert series_int.shape == (3,)
56+
assert series_float.shape == (3,)
57+
58+
59+
def test_series_hasnans(series_for_properties):
60+
series_int, series_float = series_for_properties
61+
assert not series_int.hasnans
62+
assert series_float.hasnans
63+
64+
65+
def test_series_values(series_for_properties):
66+
series_int, series_float = series_for_properties
67+
np.testing.assert_array_equal(series_int.values, np.array([1, 2, 3]))
68+
69+
70+
def test_series_nbytes(series_for_properties):
71+
series_int, series_float = series_for_properties
72+
73+
with pytest.raises(NotImplementedError, match="nbytes"):
74+
assert series_int.nbytes
75+
76+
2577
@pytest.mark.parametrize(
2678
("column", "expected_dtype"),
2779
[

0 commit comments

Comments
 (0)