Update PyArrow conversion and arrow/parquet tests for pyarrow 19.0 #60716
+96
−31
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jorisvandenbossche
added
Compat
|
Thanks for tackling this. Maybe we should also have a note in our string guide about preferring PyArrow > 19 to ensure better type preservation? |
mroeschke
approved these changes
Jan 22, 2025
|
Merging to get the CI to green. Happy to have the doc note about Pyarrow interop in a follow up PR |
|
Owee, I'm MrMeeseeks, Look at me. There seem to be a conflict, please backport manually. Here are approximate instructions:
And apply the correct labels and milestones. Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon! Remember to remove the If these instructions are inaccurate, feel free to suggest an improvement. |
jorisvandenbossche
added a commit
to jorisvandenbossche/pandas
that referenced
this pull request
Jan 22, 2025
…andas-dev#60716) * Update PyArrow conversion and arrow/parquet tests for pyarrow 19.0 * update pypi index * extra filterwarnings * more test updates * temp enable infer_string option * Adapt test_get_handle_pyarrow_compat for pyarrow 19 * Use pa_version_under19p0 in test_get_handle_pyarrow_compat * Adjust test_string_inference for using_infer_string * Fix test_string_inference for feather --------- Co-authored-by: Matthew Roeschke <[email protected]> (cherry picked from commit 5efac82)
jorisvandenbossche
added a commit
that referenced
this pull request
Jan 22, 2025
…r pyarrow 19.0 (#60716) (#60755) Co-authored-by: Matthew Roeschke <[email protected]> (cherry picked from commit 5efac82) * fixup * don't hardcode object dtype * also enable CoW when enabling future.infer_string
|
@mroeschke thanks for finishing this! Manual backport -> #60755 |
asharmalik19
pushed a commit
to asharmalik19/pandas
that referenced
this pull request
Jan 22, 2025
…andas-dev#60716) * Update PyArrow conversion and arrow/parquet tests for pyarrow 19.0 * update pypi index * extra filterwarnings * more test updates * temp enable infer_string option * Adapt test_get_handle_pyarrow_compat for pyarrow 19 * Use pa_version_under19p0 in test_get_handle_pyarrow_compat * Adjust test_string_inference for using_infer_string * Fix test_string_inference for feather --------- Co-authored-by: Matthew Roeschke <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With the upcoming pyarrow 19.0, it should start returning the future default string dtype itself, which will require some (test) changes on our side.