When the CI build cache gets a hit, it updates the timetamp of ("touches") the objects so they don't get garbage collected while they're still relevant. To do this though, we need to copy an S3 object over itself, which can take a while for large files (e.g. 20 GiB images).
The images are needed by osbuild's CI to run image info on images built by older versions and verify that the info for an image hasn't changed when the same manifest is rebuilt into another image. If we stop touching these objects, osbuild's CI will fail.
One idea is to stop saving images in the cache altogether and instead store the image info alongside the build information and the manifest. This would save on storage and transfer costs and might even speed up osbuild's CI a bit since it wont have to download the image itself and generate its image info.
This conflicts with another idea / plan we had with @supakeen however for reusing images to boot test them when they've already been built but not successfully boot tested.
CC @thozza for thoughts on the osbuild tests.
When the CI build cache gets a hit, it updates the timetamp of ("touches") the objects so they don't get garbage collected while they're still relevant. To do this though, we need to copy an S3 object over itself, which can take a while for large files (e.g. 20 GiB images).
The images are needed by osbuild's CI to run image info on images built by older versions and verify that the info for an image hasn't changed when the same manifest is rebuilt into another image. If we stop touching these objects, osbuild's CI will fail.
One idea is to stop saving images in the cache altogether and instead store the image info alongside the build information and the manifest. This would save on storage and transfer costs and might even speed up osbuild's CI a bit since it wont have to download the image itself and generate its image info.
This conflicts with another idea / plan we had with @supakeen however for reusing images to boot test them when they've already been built but not successfully boot tested.
CC @thozza for thoughts on the osbuild tests.