docker pull fails

[FlorinAndrei]

What is the version?

4.5.2-4.8.1

What happened?

This worked fine for a long time, until two days ago. Now it doesn't work.

# docker pull nvcr.io/nvidia/k8s/dcgm-exporter:4.5.2-4.8.1-ubuntu22.04
4.5.2-4.8.1-ubuntu22.04: Pulling from nvidia/k8s/dcgm-exporter
error from registry: Incorrect Repository Format

All other images on this machine are fine.

This is Ubuntu 24.04 x86_64

Docker CE 29.5.0

https://docs.docker.com/engine/install/

What did you expect to happen?

Pull the image if it doesn't exist, or exit without doing anything.

I do not expect an error there.

What is the GPU model?

No response

What is the environment?

Ubuntu machine + Docker CE

How did you deploy the dcgm-exporter and what is the configuration?

Made a docker compose file.

  dcgm-exporter:
    image: nvcr.io/nvidia/k8s/dcgm-exporter:4.5.2-4.8.1-ubuntu22.04
    container_name: dcgm-exporter
    gpus: all
    cap_add:
      - SYS_ADMIN
    ports:
      - "9400:9400"
    extra_hosts:
      - "host.docker.internal:host-gateway

How to reproduce the issue?

Probably it's enough to just try to pull it on an Ubuntu machine with Docker CE installed.

Anything else we need to know?

Only this particular image seems to fail.

[Java4all]

it is for all containers on nvcr.io only if you are pulling them from:

• NVIDIA GPU instances (hosting on gcp)
  I cannot download anything from there on NVIDIA instances like L4, L40 and so on

[FlorinAndrei]

It's a system in my living room, running Ubuntu.

Only that container fails.

[karlg-dyad]

I have also encountered this on a GCP Ubuntu machine with Nvidia card. Note that all tags I've tested for this specific image seem to be affected.

I was able to work around this by pulling on my local Ubuntu workstation and saving the image as a tarball via docker save. Then, I copied it across to the target machine and loaded it via docker load. This worked without issue.

[Java4all]

@karlg-dyad All of mate. I told that Nvidia and got an answer - try downloading from another repo :)
So, just using AWS instances instead. They are a little bit more expensive in Brev, but more stable