Authenticated Docker Hub Mirror

[brunoabreu0]

Image I'm using:
bottlerocket-aws-k8s-1.32-x86_64-v1.30*

What I expected to happen:
Pull images from my authenticated docker-hub mirror.

What actually happened:
403 error

How to reproduce the problem:
I have setup my docker hub mirror and its credentials in the toml userdata, and when I create pods that references the mirror directly, the images are pulled properly (the authentication works fine). However, when I do not reference the images directly (so they can use the mirror), the node tries to pull from mirror correctly, however, without the credentials:

This is my toml userdata:

[[settings.container-registry.mirrors]]
registry = "docker.io"
endpoint = ["https://my-mirrors.com/docker-hub"]

[[settings.container-registry.credentials]]
registry = "my-mirrors.com"
username = "username"
password = "*****"

The error I get when trying to pull the image image: natsio/nats-server-config-reloader:0.14.0 is:

Failed to pull image "natsio/nats-server-config-reloader:0.14.0": failed to pull and unpack image "docker.io/natsio/nats-server-config-reloader:0.14.0": failed to resolve reference "docker.io/natsio/nats-server-config-reloader:0.14.0": unexpected status from HEAD request to https://my-mirrors.com/docker-hub/natsio/nats-server-config-reloader/manifests/0.14.0?ns=docker.io: 403

Since the error mentions my setup mirror, I assume the mirror configuration at [[settings.container-registry.mirrors]] is working fine, despite not getting the authentication configuration from [[settings.container-registry.credentials]]

If I setup the pod to pull the image image: my-mirros.com/docker-hub/natsio/nats-server-config-reloader:0.14.0, the image is pulled successfully, which makes me assume that the configuration set at [[settings.container-registry.credentials]] is also correct, despite the mirror configuration not working together.