snapshot_download should offer a way to get progress

[jeffmaury]

When processing a model download, it should be fine to get progress about the download.

Proposal: offer a callback that returns the percentage, maybe after each file is downloaded ?

WDYT @axel7083

[coyotte508]

For reference, there are two function to upload files, uploadFiles which returns a Promise, and uploadFilesWithProgress which returns an async generator.

The same can be done here

[chriszs]

To do this for downloadFile in an Electron app, I'm passing in a custom fetch implementation that looks like this:

async function fetchWithProgress(
  input: string | URL | globalThis.Request,
  init?: RequestInit
): Promise<Response> {
  const response = await fetch(input, init);
  const contentLength = response.headers.get("content-length");
  const total = contentLength ? parseInt(contentLength, 10) : 0;

  let loaded = 0;

  return new Response(
    new ReadableStream({
      start(controller) {
        if (!response.body) {
          throw new Error("Response body is null");
        }

        const reader = response.body.getReader();

        read();
        function read() {
          reader
            .read()
            .then(({ done, value }) => {
              if (done) {
                controller.close();
                return;
              }
              loaded += value.byteLength;
              if (mainWindow?.webContents && loaded) {
                mainWindow.webContents.send("download-progress", {
                  input,
                  loaded,
                  total,
                });
              }

              controller.enqueue(value);
              read();
            })
            .catch((error) => {
              console.error(error);
              controller.error(error);
            });
        }
      },
    })
  );
}

Though this is a little hacky and may or may not work with the new Xet URLs.

This is pretty similar to the solution drafted in #1334.