New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test out final copy to OSN #9

Open

jbusecke opened this issue Apr 9, 2024 · 0 comments

Contributor

jbusecke commented Apr 9, 2024

I would like to already future proof this for when we have the m2lines OSN pod.

In the final copy stage of the recipe I would like to just read the written store from GCS and then write/copy it to OSN.

I am not quite sure how to actually do this.

Can fsspec handle this sort of transfer? (That would probably be the least hassle rn)
I might have to investigate skyplane - There I am wondering how/if they support the OSN s3 interface and how to set up the authentication specifically.

jbusecke mentioned this issue

Main Planning Issue #10

Open

4 tasks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment