Commit c0d5a59
authored
### Rationale for this change
This PR restores the behavior previous to version 23 for floating-point parsing on overflow and subnormal.
`fast_float` didn't assign an error code on overflow in version `3.10.1` and assigned `±Inf` on overflow and `0.0` on subnormal. With the update to version `8.1`, it started to assign `std::errc::result_out_of_range` in such cases.
### What changes are included in this PR?
Ignores `std::errc::result_out_of_range` and produce `±Inf` / `0.0` as appropriate instead of failing the conversion.
### Are these changes tested?
Yes. Created tests for overflow with positive and negative signed mantissa, and also created tests for subnormal, all of them for binary{16,32,64}.
### Are there any user-facing changes?
It's a user facing change. The CSV reader on version `libarrow==23` was assigning them as strings, while before it was parsing it as `0` or `+- inf`.
With this patch, the CSV reader in PyArrow outputs:
```python
>>> import pyarrow
>>> import pyarrow.csv
>>> import io
>>> table = pyarrow.csv.read_csv(io.BytesIO(f"data\n10E-617\n10E617\n-10E617".encode()))
>>> print(table)
pyarrow.Table
data: double
----
data: [[0,inf,-inf]]
```
Closes #49003
* GitHub Issue: #49003
Authored-by: Alvaro-Kothe <kothe65@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
1 parent 7532327 commit c0d5a59
2 files changed
Lines changed: 24 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | | - | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
39 | 42 | | |
40 | 43 | | |
41 | 44 | | |
42 | 45 | | |
43 | 46 | | |
44 | 47 | | |
45 | 48 | | |
46 | | - | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
47 | 53 | | |
48 | 54 | | |
49 | 55 | | |
| |||
53 | 59 | | |
54 | 60 | | |
55 | 61 | | |
56 | | - | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
57 | 66 | | |
58 | 67 | | |
59 | 68 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
141 | 141 | | |
142 | 142 | | |
143 | 143 | | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
144 | 148 | | |
145 | 149 | | |
146 | 150 | | |
| |||
166 | 170 | | |
167 | 171 | | |
168 | 172 | | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
169 | 177 | | |
170 | 178 | | |
171 | 179 | | |
| |||
185 | 193 | | |
186 | 194 | | |
187 | 195 | | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
188 | 200 | | |
189 | 201 | | |
190 | 202 | | |
| |||
0 commit comments