GH-48978: [Python] test failures on pandas 3.0 for fastparquet and for zoneinfo w/o pytz

[tadeja]

Rationale for this change

Closes #48978

What changes are included in this PR?

Update to parquet/test_basic.py test_fastparquet_cross_compatibility for fastparquet string and categorical dtype differences causing failure Attribute "dtype" are different
Update to test_pandas.py‎ test_timestamp_as_object_non_nanosecond for failure ValueError: fromutc: dt.tzinfo is not self.

Are these changes tested?

Yes. Initially tested locally with pandas upgraded to 3.0 as CI was still running with pandas 2.3.3 cached.

Are there any user-facing changes?

No.

• GitHub Issue: [Python] test failures on pandas 3.0 for fastparquet and for zoneinfo w/o pytz #48978

[github-actions]

⚠️ GitHub issue #48978 has been automatically assigned in GitHub to PR creator.

[tadeja]

Failures replicated here on CI job AMD64 Conda Python 3.13 Pandas latest
with temporary CI modification as follows:
pip install --upgrade pandas fastparquet
pip uninstall -q -y pytz

[tadeja]

Proposed fixes complete previously failing tests with success, you can see it is with pandas 3.0, with fastparquet and without pytz here:
AMD64 Conda Python 3.13 Pandas latest succeeded

• fastparquet 2025.12.0 pypi_0 pypi
• pandas 3.0.0 pypi_0 pypi
• ...
• = 7784 passed, 283 skipped, 22 xfailed, 2 xpassed, 54 warnings in 241.86s (0:04:01) =

[h-vetinari]

I backported this to the conda-forge feedstock in conda-forge/pyarrow-feedstock#169, and can confirm that it works! Thanks!

[AlenkaF]

Thank you for the fix @tadeja! I am adding my comments bellow.

[AlenkaF]

Would it make sense to use integers for categories until fastparquet fully supports pandas 3.0? That way we can also check the dtype roundtrip here.

[tadeja]

Ok, @AlenkaF here's the new approach with
"f": pd.Categorical([5, 6, 7]),

This fails the test at

tm.assert_frame_equal(table_fp.to_pandas(), df_for_fp, check_dtype=False)
  pyarrow/tests/parquet/test_basic.py:769:
...
...
E           AssertionError: Categorical Expected type <class 'pandas.Categorical'>, found <class 'numpy.ndarray'> instead

so we have to add another check_categorical=False on line 769
as already done before at line 756 for the same reason.

[tadeja]

@AlenkaF, @rok this now includes comments and # TODO: once fastparquet supports pandas 3.0 dtypes revert string/categorical test - also according to #48978 (comment)

[rok]

Just a comment suggestion.

[rok]

Timezone part looks good. Suggesting a more readable fastparquet workaround. Proposed changes are not tested.

[tadeja]

I backported this to the conda-forge feedstock in conda-forge/pyarrow-feedstock#169, and can confirm that it works! Thanks!

@h-vetinari, just a ping to let you know that slightly different fixes for pyarrow/tests/parquet/test_basic.py‎ and test_pandas.py have just been merged.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit ba61297.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 9 possible false positives for unstable benchmarks that are known to sometimes produce them.

[h-vetinari]

Thanks for the heads-up @tadeja; I was following along here, and will pick up the changes on the feedstock when I get the chance (but since it was really just about making tests pass with pandas v3, the details don't matter all that much).