[26.1] Add namespace stripping to CachedExplicitSingularityContainerResolver - #22878
Conversation
|
Will this not break existing caches? |
|
The core of the logic is if self.namespace:
prefix = f"docker://quay.io/{self.namespace}/"
if image_id.startswith(prefix):
image_id = image_id[len(prefix):]namespace is an option you can configure (so you could put non-biocontainers on a different path). |
|
Ah yeah, no existing caches would have namespace configured. |
8813ad7 to
b0a6364
Compare
|
So this also doesn't use configurable cachers (i.e. CacheDirectory from mulled) meaning there's going to be a dir listing every time - this is prohibitively expensive, hence why the FWIW I already did that in #20447. |
b0a6364 to
4391493
Compare
48dd1c6 to
fbecda0
Compare
Add an optional `namespace` config parameter that strips
`docker://quay.io/{namespace}/` from image identifiers when constructing
the local cache path, producing flat filenames compatible with
CVMFS-hosted biocontainer caches (e.g. /cvmfs/singularity.galaxyproject.org/all/).
Mirrors the same stripping already done in pull_mulled_singularity_command.
fbecda0 to
329e36b
Compare
|
As an aside, very hard to tell what changes with each iteration. For someone watching and reviewing it'd be easier if Claude pushed new commits and then just squashed at the end when we're happy. |
Uses galaxyproject/galaxy#22878 to resolve explicit docker://quay.io/biocontainers/ container specs against /cvmfs/singularity.galaxyproject.org/all before falling back to pulling. Applied to all destinations that mount singularity.galaxyproject.org (cyclone, jetstream2, tacc_hpc, bridges2, expanse, anvil, stampede3_nvdimm).
|
This PR was merged without a "kind/" label, please correct. |
Add an optional
namespaceconfig parameter that stripsdocker://quay.io/{namespace}/from image identifiers when constructing the local cache path, producing flat filenames compatible with CVMFS-hosted biocontainer caches (e.g. /cvmfs/singularity.galaxyproject.org/all/).Mirrors the same stripping already done in pull_mulled_singularity_command.
How to test the changes?
(Select all options that apply)
License