Skip to content

ci: reduce Docker Hub rate limit pressure#709

Open
agentotto[bot] wants to merge 3 commits into
mainfrom
otto/fix-docker-rate-limits
Open

ci: reduce Docker Hub rate limit pressure#709
agentotto[bot] wants to merge 3 commits into
mainfrom
otto/fix-docker-rate-limits

Conversation

@agentotto
Copy link
Copy Markdown
Contributor

@agentotto agentotto Bot commented Apr 30, 2026

Summary

This switches the repo off Docker Hub for its in-repo public image references by using AWS ECR Public mirrors instead.

ECR Public image mapping

  • postgres:latestpublic.ecr.aws/docker/library/postgres:latest
  • redis:latestpublic.ecr.aws/docker/library/redis:latest

Why this approach

  • It avoids Docker Hub unauthenticated/authenticated pull-rate limits entirely for the images referenced directly in this repo.
  • It also removes the temporary Docker Hub login workaround from CI.
  • The Docker-based cargo-deny action is replaced with native cargo-deny, so that step also no longer depends on Docker Hub.

What was pushed successfully

  • docker-compose.yml

What could not be pushed by this GitHub App token

  • .github/workflows/rust-ci.yml (403: missing workflows permission)

The full intended patch is included verbatim below so the remaining workflow change can be applied manually if needed.

Full patch

diff --git a/.github/workflows/rust-ci.yml b/.github/workflows/rust-ci.yml
index a625ef7..24dd426 100644
--- a/.github/workflows/rust-ci.yml
+++ b/.github/workflows/rust-ci.yml
@@ -204,18 +204,7 @@ jobs:
     permissions:
       contents: read
 
-    # Log in to Docker Hub if a token is provided to avoid public rate limiting
-    env:
-      IS_DOCKER_TOKEN_SET: ${{ (secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_TOKEN != '' && secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_USERNAME != '') && 'true' || 'false' }}
-
     steps:
-      - name: Log in to Docker Hub
-        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 https://github.com/docker/login-action/releases/tag/v3.6.0
-        if: ${{ env.IS_DOCKER_TOKEN_SET == 'true' }}
-        with:
-          username: ${{ secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_USERNAME }}
-          password: ${{ secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_TOKEN }}
-
       - name: Checkout code
         uses: actions/checkout@v5
 
@@ -275,18 +264,7 @@ jobs:
     permissions:
       contents: read
 
-    # Log in to Docker Hub if a token is provided to avoid public rate limiting
-    env:
-      IS_DOCKER_TOKEN_SET: ${{ (secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_TOKEN != '' && secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_USERNAME != '') && 'true' || 'false' }}
-
     steps:
-      - name: Log in to Docker Hub
-        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 https://github.com/docker/login-action/releases/tag/v3.6.0
-        if: ${{ env.IS_DOCKER_TOKEN_SET == 'true' }}
-        with:
-          username: ${{ secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_USERNAME }}
-          password: ${{ secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_TOKEN }}
-
       - name: Checkout code
         uses: actions/checkout@v5
 
@@ -327,24 +305,17 @@ jobs:
 
     continue-on-error: ${{ matrix.checks == 'advisories' }}
 
-    # Log in to Docker Hub if a token is provided to avoid public rate limiting
-    env:
-      IS_DOCKER_TOKEN_SET: ${{ (secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_TOKEN != '' && secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_USERNAME != '') && 'true' || 'false' }}
-
     steps:
-      - name: Log in to Docker Hub
-        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 https://github.com/docker/login-action/releases/tag/v3.6.0
-        if: ${{ env.IS_DOCKER_TOKEN_SET == 'true' }}
-        with:
-          username: ${{ secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_USERNAME }}
-          password: ${{ secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_TOKEN }}
-
       - name: Checkout code
         uses: actions/checkout@v4
 
-      - uses: EmbarkStudios/cargo-deny-action@v2.0.14
+      - name: Install cargo-deny
+        uses: taiki-e/install-action@e24b8b7a939c6a537188f34a4163cb153dd85cf6 # v2.69.1 https://github.com/taiki-e/install-action/releases/tag/v2.69.1
         with:
-          command: check ${{ matrix.checks }}
+          tool: cargo-deny
+
+      - name: Run cargo deny
+        run: cargo deny check ${{ matrix.checks }}
 
   docker:
     name: Test Docker build
@@ -358,18 +329,7 @@ jobs:
     permissions:
       contents: read
 
-    # Log in to Docker Hub if a token is provided to avoid public rate limiting
-    env:
-      IS_DOCKER_TOKEN_SET: ${{ (secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_TOKEN != '' && secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_USERNAME != '') && 'true' || 'false' }}
-
     steps:
-      - name: Log in to Docker Hub
-        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 https://github.com/docker/login-action/releases/tag/v3.6.0
-        if: ${{ env.IS_DOCKER_TOKEN_SET == 'true' }}
-        with:
-          username: ${{ secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_USERNAME }}
-          password: ${{ secrets.DOCKER_HUB_PULL_PUBLIC_IMAGES_TOKEN }}
-
       - name: Checkout code
         uses: actions/checkout@v5
 
diff --git a/docker-compose.yml b/docker-compose.yml
index dc98a75..5292a16 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -3,7 +3,7 @@
 name: world-id-protocol-services
 services:  # world-id-indexer
   postgres:
-    image: postgres:latest
+    image: public.ecr.aws/docker/library/postgres:latest
     ports:
       - "5432:5432"
     environment:
@@ -12,7 +12,7 @@ services:  # world-id-indexer
       - POSTGRES_DB=indexer_tests
   # world-id-gateway
   redis:
-    image: redis:latest
+    image: public.ecr.aws/docker/library/redis:latest
     ports:
       - "6379:6379"
     healthcheck:

Note

Low Risk
Low risk: changes only affect local/dev and integration-test container image sources and Redis testcontainer startup, not production logic. Main risk is CI/test flakiness if the mirrored images or readiness message differ from the upstream tags.

Overview
Switches in-repo Docker image references from Docker Hub to the AWS ECR Public mirrors for postgres:latest and redis:latest (in docker-compose.yml) to reduce pull-rate limit failures.

Updates gateway integration tests to start Redis via testcontainers using a GenericImage pointed at public.ecr.aws/docker/library/redis:latest, including explicit port exposure and a stdout readiness wait, and adjusts types accordingly.

Reviewed by Cursor Bugbot for commit 7d8279c. Bugbot is set up for automated code reviews on this repo. Configure here.

@agentotto agentotto Bot requested a review from a team as a code owner April 30, 2026 17:18
Copy link
Copy Markdown
Contributor

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 147bc9c. Configure here.

Comment thread docker-compose.yml
@agentotto
Copy link
Copy Markdown
Contributor Author

agentotto Bot commented May 4, 2026

On it. I'll spawn a worker to fix the testcontainers Redis image reference in services/gateway/tests/common.rs to match the ECR mirror.
I've acknowledged the comment and kicked off a worker to fix it. The issue is that services/gateway/tests/common.rs uses Redis::default().with_tag("latest") in testcontainers, which still resolves to Docker Hub — defeating the ECR migration in docker-compose.yml. The fix is to switch to GenericImage::new("public.ecr.aws/docker/library/redis", "latest") so all Redis pulls go through ECR. I'll follow up once the commit is pushed.

Replace `Redis::default().with_tag("latest")` with a GenericImage
pointing to `public.ecr.aws/docker/library/redis:latest`, consistent
with the ECR mirror already set in docker-compose.yml (PR #709).

This avoids Docker Hub pull-rate-limit failures in CI by sourcing the
Redis image from ECR Public instead of docker.io.
@agentotto
Copy link
Copy Markdown
Contributor Author

agentotto Bot commented May 4, 2026

The fix is done and pushed. Here's what was changed in commit 7d8279c on otto/fix-docker-rate-limits:

services/gateway/tests/common.rs

  • Swapped Redis::default().with_tag("latest") (Docker Hub) for a GenericImage pointing to the ECR Public mirror:
    GenericImage::new("public.ecr.aws/docker/library/redis", "latest")
        .with_exposed_port(REDIS_PORT.tcp())
        .with_wait_for(WaitFor::message_on_stdout("Ready to accept connections"))
        .start()...
  • Updated imports accordingly (Redis removed, GenericImage + WaitFor + IntoContainerPort added)
  • ContainerAsync<Redis> field type updated to ContainerAsync<GenericImage>

No other Docker Hub Redis/Postgres references were found in test code — other testcontainer usages already used GenericImage with non-Docker-Hub images. cargo check passed cleanly for the world-id-gateway crate.
Done — replied on the PR thread with the fix details.

@agentotto
Copy link
Copy Markdown
Contributor Author

agentotto Bot commented May 4, 2026

The subagent just cleaned up its workspace. Everything is done — no further action needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant