How to list files from a Shared Drive and download files from a Shared Drive?

[vadimkantorov]

I successfully created fs instance using the script in README:

import gdrivefs

# use this the first time you run
# token = 'browser'
# use this on subsequent attempts
token = 'cache'
fs = gdrivefs.GoogleDriveFileSystem(token=token)

# how to do something similar for a shared drive via its name? or via a given folder ID?
# print(fs.ls(...))
# specifying it as follows does not work and fails with 404 gdrivefs.GoogleDriveFileSystem(root_file_id="MYSHAREDDRIVEFOLDERIDHERE", token=token).ls("") 

How can I list the contents of a shared drive (e.g. via folder ID, like found in URLs like https://drive.google.com/drive/folders/MYSHAREDDRIVEFOLDERIDHERE)?

(Same question for using with gdrive:// protocol without explicit import of gdrivefs)

Thanks!

---

Related:

• Support for Shared Drives #40
• License and shared-drive #26

[martindurant]

We simply haven't supported setting the drive to work with yet. But it can be done - I have some scratch code working, like

fs.drives
# lists shared drives, includes IDs
fs.service.list(driveId="0AFWwup7Xi422Uk9PVA", includeItemsFromAllDrives=True, corpora="drive", supportsAllDrives=True, q="'0AFWwup7Xi422Uk9PVA' in parents and trashed = false").execute()
# ^ requires extra flags, making it slower; root of drive has "parents" ID the same as the drive itself
fs.service.list(driveId="0AFWwup7Xi422Uk9PVA", includeItemsFromAllDrives=True, corpora="drive", supportsAllDrives=True, q="'14cmVJlRuEoyo3DmZhYDWPT5BjnE0ab54' in parents and trashed = false").execute()
# ^ listing one of the directories, ID revealed by line above

So it can be done with some work, but the API isn't exactly clear. Should it be using the root ID as you had hoped, or a separate drive argument or form a URL standard like "gdrive://driveID/folder/file", perhaps with the convenience "default" for the current usage.

[vadimkantorov]

I was basically looking in how to:

1. download a file from a shared drive via its file ID (and ideally how to construct a gdrive:// URLs for files, drives and folders - so that I can supply these URLs wherever fsspec is supported)
2. walk a shared drive recursively (maybe skipping shortcuts) and be able to download all files (to emulate rclone sync)
3. list recursively for every file in a shared drive its owner and "Share" permissions - this is useful for coding up an audit util for a team's shared drive

(also gdrive has a problem when a "malicious" file cannot be downloaded, at least rclone hasn't completely solved it for sure)

[vadimkantorov]

Regarding the file ID's, can gdrivefs express "global" urls such as gdrive://driveID/ or gdrive://folderID/ or gdrive://fileID? These are also useful (in complement to traditional gdrive://some folder/path/to/file.ext), as these IDs can be parsed out of the URLs to https://drive.google.com

[martindurant]

Regarding the file ID's, can gdrivefs express "global" urls such as gdrive://driveID/ or gdrive://folderID/ or gdrive://fileID

We probably could, if there is a concrete reference we can find that allows us to determine whether an ID is a drive or a file. The folders are just files with the file type, so we can't tell those apart.

I must say, though, that something like "gdrive://fileID" doesn't make a lot of sense to me - use the HTTP URL. Also for folder IDs, now you need to know whether it's on your default drive or elsewhere; or maybe we change things to always search "everywhere".

[vadimkantorov]

I would propose to extend the URI schema as:
gdrive://file@FILEID, gdrive://folder@FOLDERID, gdrive://folder@FOLDERID@drive@DRIVEID, gdrive://drive@DRIVEID (or in reverse gdrive://FILEID@file) or something in this spirit - to allow the user to provide additional info, and make it that the direct addressing via ID is separate from existing gdrive://some/path/to/file.ext

maybe we change things to always search "everywhere".

Yeah, maybe this would be nice! for avoiding things like gdrive://folder@FOLDERID@drive@DRIVEID

"gdrive://fileID" doesn't make a lot of sense to me - use the HTTP URL

This is not a viable alternative for me, as my goal is to be able to use gdrive file URIs with other software/packages that already support fsspec and can consume fsspec URIs (and thus gdrivefs when it is installed) - so passing in http url is not possible and the fsspec-using software would not know how to work with it

Allowing addressing with these global IDs (as an option) is nice because they are persistent and do not change when the user moves the file around, and sometimes this is nice to benefit from this property

---

What also might be nice is being able to specify "format" for the file to be exported, e.g. gdrive://file@FILEID[docx] or gdrive://file@SHEETID[csv] (Google Docs export APIs support this and other tools in this spirit support passing this argument too: e.g. https://github.com/wkentaro/gdown)

[martindurant]

make it that the direct addressing via ID is separate from existing

It's OK to break the API, since this repo isn't exactly released or used - if we end up making something better out of it.

my goal is to be able to use gdrive file URIs with other software/packages that already support fsspec and can consume fsspec URIs

fsspec does support http(s), though :)

What also might be nice is being able to specify "format" for the file to be exported

There is an export method already, taking a MIME type. It's not the same as download, though, because you cannot have random access.

[punya]

@martindurant, you wrote

I must say, though, that something like "gdrive://fileID" doesn't make a lot of sense to me - use the HTTP URL.

Are you suggesting that we should use https://drive.google.com/file/d/1sHM02MBcXRZuoPpGJ_Szxv8w4c7gzY3n directly with fsspec? Is that supported (with auth etc)?