Skip to content

multiple caches for our object stores - #2310

Open
bgruening wants to merge 3 commits into
masterfrom
multiple-caches
Open

multiple caches for our object stores#2310
bgruening wants to merge 3 commits into
masterfrom
multiple-caches

Conversation

@bgruening

@bgruening bgruening commented Sep 1, 2026

Copy link
Copy Markdown
Member

@sj213 fyi, we can now do also round-robin over cache dirs ...

@sj213

sj213 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What about the associated JWDs? Currently, all jobs using the S3-cache for input will have their JWD in jwd06, meaning that this JWD share sees much heavier usage than the rest, see here.

Will the round-robin also apply to JWDs for jobs using S3?

@bgruening

Copy link
Copy Markdown
Member Author

Yes, see here: #2311

But, it can be that for one single job the JWD is on different share then the OBC.

@sj213

sj213 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Do I understand this correctly: With PR-2311 in place (assuming all Galaxy software upgrade prerequisites are met), the JWD will no longer be tied to a particular backend through its extra_dirs.type:job_work attribute but rather selected round-robin indenpendently?

@bgruening

Copy link
Copy Markdown
Member Author

Correct! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants