Skip to content

Commit 8601aeb

Browse files
SCHJonathandongjoon-hyun
authored andcommitted
[SPARK-54348][INFRA] Recover Python unit tests CI by installing zstandard==0.25.0
### What changes were proposed in this pull request? In #53024 (comment), PR CI Python unit tests failed due to ``` pyspark.errors.exceptions.base.PySparkImportError: [PACKAGE_NOT_INSTALLED] zstandard >= 0.25.0 must be installed; however, it was not found. ``` This PR add the required dependency to the pre-merge CI. ### Why are the changes needed? Recover Python unit tests CI ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? PR #53024 Python CI back to healthy with this change ### Was this patch authored or co-authored using generative AI tooling? No Closes #53058 from SCHJonathan/jonathan-chang_data/fix-python-ci-dep. Authored-by: Yuheng Chang <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit a916690) Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 01b6e73 commit 8601aeb

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

.github/workflows/build_and_test.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -361,7 +361,7 @@ jobs:
361361
- name: Install Python packages (Python 3.11)
362362
if: (contains(matrix.modules, 'sql') && !contains(matrix.modules, 'sql-')) || contains(matrix.modules, 'connect') || contains(matrix.modules, 'yarn')
363363
run: |
364-
python3.11 -m pip install 'numpy>=1.22' pyarrow pandas pyyaml scipy unittest-xml-reporting 'lxml==4.9.4' 'grpcio==1.76.0' 'grpcio-status==1.76.0' 'protobuf==6.33.0'
364+
python3.11 -m pip install 'numpy>=1.22' pyarrow pandas pyyaml scipy unittest-xml-reporting 'lxml==4.9.4' 'grpcio==1.76.0' 'grpcio-status==1.76.0' 'protobuf==6.33.0' 'zstandard==0.25.0'
365365
python3.11 -m pip list
366366
# Run the tests.
367367
- name: Run tests

.github/workflows/maven_test.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,7 @@ jobs:
175175
- name: Install Python packages (Python 3.11)
176176
if: contains(matrix.modules, 'resource-managers#yarn') || (contains(matrix.modules, 'sql#core')) || contains(matrix.modules, 'connect')
177177
run: |
178-
python3.11 -m pip install 'numpy>=1.22' pyarrow pandas pyyaml scipy unittest-xml-reporting 'grpcio==1.76.0' 'grpcio-status==1.76.0' 'protobuf==6.33.0'
178+
python3.11 -m pip install 'numpy>=1.22' pyarrow pandas pyyaml scipy unittest-xml-reporting 'grpcio==1.76.0' 'grpcio-status==1.76.0' 'protobuf==6.33.0' 'zstandard==0.25.0'
179179
python3.11 -m pip list
180180
# Run the tests using script command.
181181
# BSD's script command doesn't support -c option, and the usage is different from Linux's one.

0 commit comments

Comments
 (0)