Skip to content

Monitor cleanup operations - #2188

Draft
gsaudade99 wants to merge 10 commits into
usegalaxy-eu:masterfrom
gsaudade99:feat/monitor-cleanup
Draft

Monitor cleanup operations#2188
gsaudade99 wants to merge 10 commits into
usegalaxy-eu:masterfrom
gsaudade99:feat/monitor-cleanup

Conversation

@gsaudade99

@gsaudade99 gsaudade99 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

There is a critical point here. Ideally I would run the script directly in telegraf, but I think is better to keep users telegraf and galaxy separated.
For this, I changed the script to write a galaxy_cleanup_status_file with weather the cleanup script was a success or not based on the exit status of the python cleanup script.

Closes https://github.com/usegalaxy-eu/issues/issues/995.

@gsaudade99
gsaudade99 marked this pull request as ready for review July 29, 2026 15:10
@gsaudade99 gsaudade99 changed the title WIP: monitor cleanup operations Monitor cleanup operations Jul 29, 2026
Comment thread roles/usegalaxy_eu.galaxy_cleanup/defaults/main.yml Outdated

@bgruening bgruening left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in principle, I'm just wondering the err files should not contain an error message already that you can reuse.

Co-authored-by: Björn Grüning <bjoern@gruenings.eu>
@gsaudade99

Copy link
Copy Markdown
Contributor Author

Looks good in principle, I'm just wondering the err files should not contain an error message already that you can reuse.

There are many warnings in this files, but I guess I could maybe grep for ERROR and catch stuff like this:

2026-07-05 22:55:39,443 ERROR _exists_remotely(): Trouble checking existence of S3 key '9/0/d/dataset_90d19a7e-6f2c-4d6e-ae19-5df930509df7_files/'
2026-07-06 01:01:18,562 ERROR _get_bucket(): Could not get bucket 'fr-galaxy-scratch-01', attempt 1/5

I think my concern here, is more if the cleanup script gets stuck. I will rethink a better solution

@gsaudade99
gsaudade99 marked this pull request as draft July 31, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants