You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Because s3fs (fsspec) supports some glob patterns (*, **, ?, and some of []), there are file names that should not be passed to self.fs.cp or self.fs.rm without escape.
To reproduce
Example 1.
Add to files test_file.txt and test?file.txt.
A request to remove test?file.txt removes both files.
Example 2.
Create directory test?dir.
Removing (or renaming) the directory test?dir fails (by RecursionError: maximum recursion depth exceeded while calling a Python object).
Because s3fs (fsspec) supports some glob patterns (
*
,**
,?
, and some of[]
), there are file names that should not be passed toself.fs.cp
orself.fs.rm
without escape.To reproduce
Example 1.
test_file.txt
andtest?file.txt
.test?file.txt
removes both files.Example 2.
test?dir
.test?dir
fails (by RecursionError: maximum recursion depth exceeded while calling a Python object).To fix the issue
For now, adding
glob.escape
doesn't fix the issue becausefsspec
does not interpret the pattern[?]
correctly.https://github.com/fsspec/filesystem_spec/blob/2023.5.0/fsspec/asyn.py#L648
Instead, I configured the fs not to use glob by
and turning off the if-branch https://github.com/fsspec/filesystem_spec/blob/2023.5.0/fsspec/asyn.py#L763-L772.
The text was updated successfully, but these errors were encountered: