[registry-facade] Properly retry fetching mainfests/config as well

[geropl]

Description

Turns out that similarly to #20879 we have a retry mechanism in place, but it does capture this scenario:

And they happen because if io.ReadAll(rc) fails, we do not re-try in any way because:

rc is a httpReadSeeker, so does not only read a body, but might also do new range requests

containerds dockerFetcher does uses http.DefaultClient for subsequent requests, instead of the one we pass into the dockerResolver here 😬 :facepalm-picard:

So the solution to all of those is to implement a retry around the io.ReadAll and .Fetch(...).

Related Issue(s)

Fixes CLC-1365

Related: CLC-195, CLC-1279

How to test

• tests are green: ✔️
• start a workspace: ✔️
• start [an image build](https://gpl-1365-rf23ba3a1ae.preview.gitpod-dev.com/new#https://github.com/geropl/gitpod-test-repo/tree/gpl/docker-build-30s]: ✔️

Documentation

Preview status

gitpod:summary

Build Options

Build

• /werft with-werft
  Run the build with werft instead of GHA
• leeway-no-cache
• /werft no-test
  Run Leeway with --dont-test

Publish

• /werft publish-to-npm
• /werft publish-to-jb-marketplace

Installer

• analytics=segment
• with-dedicated-emulation
• workspace-feature-flags
  Add desired feature flags to the end of the line above, space separated

Preview Environment / Integration Tests

• /werft with-local-preview
  If enabled this will build install/preview
• /werft with-preview
• /werft with-large-vm
• /werft with-gce-vm
  If enabled this will create the environment on GCE infra
• /werft preemptible
  Saves cost. Untick this only if you're really sure you need a non-preemtible machine.
• with-integration-tests=all
  Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
• with-monitoring

/hold

[corneliusludmann]

Worked as advertised. Code looks good.

[geropl]

/unhold