Commit 93fd1d7
committed
fix: prepared-statement cache id-recycling corruption for DataFrame/Arrow params
_pybind_value_signature used raw id(value) for pointer-type parameters
(DataFrames, Arrow tables, Polars DataFrames). Python recycles object ids
after garbage collection (~99.7% of the time a new allocation reuses a freed
id). On a shared connection (conn_db_readonly), this caused the cache to
return a stale prepared statement compiled for a completely different
DataFrame — e.g. string columns from test_pyarrow_string reused for
dictionary-encoded int32 columns in test_pyarrow_dict — producing data
corruption (dictionary indices read as string offsets).
Fix: replace raw id(value) with a weakref-validated generation counter.
_pybind_pointer_tracker[id(obj)] -> (gen, weakref.ref(obj))
On each call:
If id is found AND the weakref is still alive → reuse the cached
generation (same object → cache hit, correct).
If the weakref is dead (original object was GCd and id recycled) →
purge the stale entry and assign a fresh generation.
If not found → assign a fresh generation and store the weakref.
This preserves correct caching for repeated calls with the same object
while being immune to id recycling.
Fixes test_pyarrow_dict intermittent AssertionError.1 parent b03788a commit 93fd1d7
1 file changed
Lines changed: 26 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
117 | 118 | | |
118 | 119 | | |
119 | 120 | | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
120 | 131 | | |
121 | 132 | | |
122 | 133 | | |
| |||
175 | 186 | | |
176 | 187 | | |
177 | 188 | | |
178 | | - | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
179 | 204 | | |
180 | 205 | | |
181 | 206 | | |
| |||
0 commit comments