Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make datapipe.py compatible with torchdata==0.11 or later #10080

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

drivanov
Copy link
Contributor

With torch 2.7/torchdata 0.11 releases, the datapype.py example fails

python3 /workspace/examples/datapipe.py 
Traceback (most recent call last):
  File "/workspace/examples/datapipe.py", line 17, in <module>
    from torchdata.datapipes.iter import FileLister, FileOpener, IterDataPipe
ModuleNotFoundError: No module named 'torchdata.datapipes'

fails because some classes have been moved from torchdata to torch nd a few methods, including in_memory_cache, have been removed.

The current PR resolves this issue in that example.

NOTES:
Running this example with the option --task mesh was failing even with previous versions of torch/torchdata packages

  • because 3991 out of 7981 are binary, not text files in the required "OFF" format;
  • if the meshio and torch_claster packages are not installed.

Proposed changes address both these issues.

@drivanov drivanov requested a review from wsad1 as a code owner February 28, 2025 03:35
@akihironitta akihironitta changed the title Fixing datapipe.py example. Make datapipe.py compatible with torchdata==0.11 or later Mar 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants