[core][sandbox] Support a Docker Hub pull-through mirror - #65745
Merged
Conversation
Sandbox image pulls are anonymous Docker Hub pulls, so a fleet of nodes pulling benchmark images concurrently runs straight into Docker Hub's anonymous rate limits and pays WAN latency per node. RAY_SANDBOX_REGISTRY_MIRROR names a registry that mirrors Docker Hub as host[:port][/repo-prefix] — an ECR pull-through cache, an Artifact Registry remote repository, or an in-cluster registry:2 proxy — and Docker Hub pulls are rewritten to it (other registries pass through untouched). Mirrors Docker's own registry-mirrors semantics, minus the fallback: when set, the mirror is authoritative. Signed-off-by: xyuzh <xinyzng@gmail.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces support for routing Docker Hub pulls through a configured pull-through mirror using the RAY_SANDBOX_REGISTRY_MIRROR environment variable. It adds the apply_registry_mirror helper function to rewrite registry and repository paths for Docker Hub, updates the image pulling logic to apply this mirror, and includes corresponding unit tests. Feedback suggests handling potential scheme prefixes (like http:// or https://) in the mirror URL to prevent malformed host and prefix partitioning, along with adding a test case to verify this behavior.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
Reviewed by Cursor Bugbot for commit b78668e. Configure here.
Contributor
|
This looks great to me, some thoughts:
|
ray-gardener
Bot
added
docs
2 tasks
andrewsykim
reviewed
Aug 27, 2026
xyuzh
force-pushed
the
sandbox-registry-mirror
branch
from
August 28, 2026 00:30
b2b8332 to
f4460a2
Compare
Review feedback: in-cluster pull-through proxies (a plain registry:2) speak HTTP, and a scheme-prefixed mirror value previously produced broken https://http://... URLs. An explicit http:// or https:// scheme on RAY_SANDBOX_REGISTRY_MIRROR is now parsed and honored; bare hosts keep the https default. Signed-off-by: xyuzh <xinyzng@gmail.com>
xyuzh
force-pushed
the
sandbox-registry-mirror
branch
from
August 28, 2026 00:31
f4460a2 to
d5e82a3
Compare
andrewsykim
approved these changes
Aug 28, 2026
Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
Add a Container images section covering how sandbox images are pulled and cached, and a subsection for RAY_SANDBOX_REGISTRY_MIRROR: why a concurrent cluster needs a mirror, the host[:port][/repo-prefix] value with worked examples for ECR pull-through caches, Artifact Registry remote repositories, and an in-cluster registry:2 proxy, plus the three behaviors that surprise: HTTPS by default, no fallback to Docker Hub, and anonymous pulls only. Point the image-pull troubleshooting bullet at it. Signed-off-by: xyuzh <xinyzng@gmail.com>
Member
Author
|
docs updated |
Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
pcmoritz
approved these changes
Aug 31, 2026
Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
pcmoritz
enabled auto-merge (squash)
September 1, 2026 01:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Sandbox image pulls are anonymous Docker Hub pulls. A cluster of nodes pulling distinct multi-GB benchmark images concurrently runs straight into Docker Hub's anonymous rate limits and pays WAN latency on every node — during a Terminal-Bench 2.1 evaluation under Harbor, big-image tasks failed in the concurrent sweep but passed in isolation.
RAY_SANDBOX_REGISTRY_MIRRORnames a registry that mirrors Docker Hub, ashost[:port][/repo-prefix]:<acct>.dkr.ecr.<region>.amazonaws.com/dockerhub),registry:2proxy.Docker Hub pulls are rewritten to the mirror (the prefix prepended to the repository, as ECR requires); other registries pass through untouched. This mirrors Docker's own
registry-mirrorssemantics, minus the fallback: when set, the mirror is authoritative, and it uses the same anonymous token flow as any registry.Related issue number
Follow-up to #65570; sibling of #65737 and #65744.
Checks
test_registry_mirror_rewrites_docker_hub_only); runs without gVisor or network.