Respect each example requirements and use uv #1330
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Dmitry Rogozhkin <[email protected]>
Current version of the example requires `numpy<2` otherwise the following error can be seen: ``` AttributeError: module 'numpy' has no attribute 'bool8'. Did you mean: 'bool'? ``` Signed-off-by: Dmitry Rogozhkin <[email protected]>
Current version of examples require `torch<2.6` otherwise the following
error can be seen:
```
File "/pytorch/examples/time_sequence_prediction/train.py", line 47, in <module>
data = torch.load('traindata.pt')
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/pytorch/examples/time_sequence_prediction/.venv/lib/python3.12/site-packages/torch/serialization.py", line 1524, in load
raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
```
Signed-off-by: Dmitry Rogozhkin <[email protected]>
This commit introduces few changes to CI by modifying `run_*_examples.sh` and respective github workflows: * Switched to uv * Added tearup and teardown stages for tests (`start()` and `stop()` methods wrapping up test bodies - these are called automatically) * Tearup (`start()`) installs example dependencies and, optionally (if `VIRTUAL_ENV=.venv` is passed), creates uv virtual environment * Teardown (`stop()`) removes uv virtual environment if it was created (to save space) * If no `VIRTUAL_ENV` set, then scripts expect to be executed in the existing virtual environment. These can be `python -m venv`, `uv env` or `conda env`. In this case example dependencies will be installed in this environment potentially reinstalling existing packages (including `torch`!). * Dropped automated detection of CUDA platform. Now scripts require `USE_CUDA=True` to be passed explicitly * Added `PIP_INSTALL_ARGS` environment variable to be passed to `uv pip install` calls for each example dependencies. This allows to adjust torch indices and other options. Execute all tests in current virtual environment (might rewrite packages): ``` ./run_distributed_examples.sh ``` Execute all tests creating separate environment for each example: ``` VIRTUAL_ENV=.venv ./run_distributed_examples.sh ``` Run with CUDA: ``` USE_CUDA=True ./run_distributed_examples.sh ``` Adjust index: ``` PIP_INSTALL_ARGS="--pre -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html" \ ./run_distributed_examples.sh ``` Signed-off-by: Dmitry Rogozhkin <[email protected]>
✅ Deploy Preview for pytorch-examples-preview canceled.
|
4 tasks
|
Nice! It's promising that CI is green. Will review asap |
msaroufim
approved these changes
Apr 26, 2025
This was referenced Apr 29, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For: #1329
This commit introduces few changes to CI by modifying
run_*_examples.shand respective github workflows:start()andstop()methods wrapping up test bodies - these are called automatically)start()) installs example dependencies and, optionally (ifVIRTUAL_ENV=.venvis passed), creates uv virtual environmentstop()) removes uv virtual environment if it was created (to save space)VIRTUAL_ENVset, then scripts expect to be executed in the existing virtual environment. These can bepython -m venv,uv envorconda env. In this case example dependencies will be installed in this environment potentially reinstalling existing packages (includingtorch!).USE_CUDA=Trueto be passed explicitlyPIP_INSTALL_ARGSenvironment variable to be passed touv pip installcalls for each example dependencies. This allows to adjust torch indices and other options.Execute all tests in current virtual environment (might rewrite packages):
Execute all tests creating separate environment for each example:
Run with CUDA:
Adjust index:
Few changes were required in examples
requirements.txtfiles:reinforcement_learningrequirement for number to be<2due to:time_sequence_predictionandword_language_modelrequirement for torch to be<2.6due to:CC: @msaroufim, @malfet, @atalman