-
Notifications
You must be signed in to change notification settings - Fork 63
Supporting pathlib's Path objects in FileDataStream #377
base: master
Are you sure you want to change the base?
Conversation
Will this work with Python 2.7? It looks like pathlib was added in version 3.4. |
with tempfile.NamedTemporaryFile(mode='w', delete=False) as f: | ||
df.to_csv(f, sep=',', index=False) | ||
|
||
fi = FileDataStream.read_csv(Path(f.name), sep=',') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The unit test is failing here. Looks like a similar change might also be required in internal\utils\data_schema.py
(in read_schema(*data, **options)
).
I haven't tested it but maybe updating the part of the code around line 844 might fix the issue:
elif hasattr(X, 'read') or isinstance(X, str) or (
six.PY2 and isinstance(X, (str, text_type))):
It looks like you're right, and that Path objects won't work with python 2.7. I tried several different things to try and get it to work but with no luck. Do you have any suggestions on how to proceed? What if we checked the system python version and used Path objects only when that value is greater than 3.4? |
Checking for a particular Python version is done in some other parts of the code (including the snippet in my previous comment). Search for |
|
Fixes #269 .
pathlib
's Path objects can be converted to strings just by casting, and vice versa. I added a check inFileDataStream
's init function to convert a Path object to a string. I also wrote a test for this, but callingFileDataStream.read_csv()
on a Path object produces the following error (using tool=None/'pandas'):@ganik do I need to define my own schema for a file path as a Path object, even though the contents of the file should be the same? I'm a little stuck here and any help would be appreciated!