Charm config option to disable checksum

[selcem-artan]

Bug Description

Hello,
for S3 backend Dell ObjectScale v.4.0.1, backup was failing with error on checksum. Output of

sudo k8s kubectl get BackupStorageLocation default -n <namespace> -o yaml

shows that the backup couldn't succeed

status:
  expiration: "2025-11-29T15:37:55Z"
  failureReason: 'rpc error: code = Unknown desc = error putting object ipe-kaas2-backup/backups/mybackup/velero-backup.json:
    operation error S3: PutObject, https response error StatusCode: 400, RequestID:
    810d40de:197dae41bfe:105f05:48d, HostID: a7f966aeddeecec44ecdacf53196ef346bae8ac1a838bd905af74b2fd4d22949,
    api error XAmzContentSHA256Mismatch: The Content-SHA256 you specified did not
    match what we received'
  formatVersion: 1.1.0

To Reproduce

juju deploy velero-operator --channel=1.15/stable
juju deploy s3-integrator
juju integrate s3-integrator velero-operator

Environment

Juju v3.6 , velero-operator and s3-integrator versions are as below

App                Version  Status  Scale  Charm              Channel      Rev  Address      Exposed  Message
grafana-agent-k8s  0.40.4   active      1  grafana-agent-k8s  1/stable     164  10.0.42.41   no       tracing: off
s3-integrator               active      1  s3-integrator      1/stable     145  10.0.36.96   no       
velero-operator             active      1  velero-operator    1.15/stable  421  10.0.33.216  no       Unit is Ready

Unit                  Workload  Agent  Address      Ports  Message
grafana-agent-k8s/0*  active    idle   10.0.25.18          tracing: off
s3-integrator/0*      active    idle   10.0.24.200         
velero-operator/0*    active    idle   10.0.17.4           Unit is Ready

Relevant Log Output

time="2025-10-31T08:43:48Z" level=info msg="Skipping resource customresourcedefinitions.apiextensions.k8s.io, because it's cluster-scoped and only specific namespaces or namespace scope types are included in the backup." backup=k8s-test/mybackup2 logSource="pkg/util/collections/includes_excludes.go:155"
time="2025-10-31T08:43:48Z" level=info msg="Found associated CRD ciliumendpoints.cilium.io to add to backup" backup=k8s-test/mybackup2 logSource="pkg/backup/backup.go:863"
time="2025-10-31T08:43:48Z" level=info msg="Backing up item" backup=k8s-test/mybackup2 logSource="pkg/backup/item_backupper.go:184" name=ciliumendpoints.cilium.io namespace= resource=customresourcedefinitions.apiextensions.k8s.io
time="2025-10-31T08:43:48Z" level=info msg="Executing custom action" backup=k8s-test/mybackup2 logSource="pkg/backup/item_backupper.go:357" name=ciliumendpoints.cilium.io namespace= resource=customresourcedefinitions.apiextensions.k8s.io
time="2025-10-31T08:43:48Z" level=info msg="Executing RemapCRDVersionAction" backup=k8s-test/mybackup2 cmd=/velero logSource="pkg/backup/actions/remap_crd_version_action.go:61" pluginName=velero
time="2025-10-31T08:43:48Z" level=info msg="Exiting RemapCRDVersionAction, the cluster does not support v1beta1 CRD" backup=k8s-test/mybackup2 cmd=/velero logSource="pkg/backup/actions/remap_crd_version_action.go:88" pluginName=velero
time="2025-10-31T08:43:48Z" level=info msg="Summary for skipped PVs: [{\"name\":\"pvc-acb927e7-6ebe-4a11-82b9-4d2a0223bdcd\",\"reasons\":[{\"approach\":\"podvolume\",\"reason\":\"opted out due to annotation in pod pv-writer-test\"},{\"approach\":\"volumeSnapshot\",\"reason\":\"no applicable volumesnapshotter found\"}]}]" backup=k8s-test/mybackup2 logSource="pkg/backup/backup.go:543"
time="2025-10-31T08:43:48Z" level=info msg="Backed up a total of 13 items" backup=k8s-test/mybackup2 logSource="pkg/backup/backup.go:547" progress=
time="2025-10-31T08:43:48Z" level=info msg="Setting up backup store to persist the backup" backup=k8s-test/mybackup2 logSource="pkg/controller/backup_controller.go:731"
time="2025-10-31T08:43:48Z" level=error msg="Error uploading log file" backup=mybackup2 bucket=kaas2-backup error="rpc error: code = Unknown desc = error putting object backups/mybackup2/mybackup2-logs.gz: operation error S3: PutObject, https response error StatusCode: 400, RequestID: 810d40df:197dae8479f:109275:43a9, HostID: de9481a091f336781ceef543b51d0e29dd84d68acd51dd103467d9ea51af8473, api error XAmzContentSHA256Mismatch: The Content-SHA256 you specified did not match what we received" error.file="/go/src/velero-plugin-for-aws/velero-plugin-for-aws/object_store.go:280" error.function="main.(*ObjectStore).PutObject" logSource="pkg/persistence/object_store.go:256" prefix=
time="2025-10-31T08:43:48Z" level=info msg="Initial backup processing complete, moving to Finalizing" backup=k8s-test/mybackup2 logSource="pkg/controller/backup_controller.go:745"
time="2025-10-31T08:43:48Z" level=info msg="plugin process exited" backup=k8s-test/mybackup2 cmd=/velero id=28129 logSource="pkg/plugin/clientmgmt/process/logrus_adapter.go:80" plugin=/velero
time="2025-10-31T08:43:48Z" level=info msg="plugin process exited" backup=k8s-test/mybackup2 cmd=/plugins/velero-plugin-for-aws id=28143 logSource="pkg/plugin/clientmgmt/process/logrus_adapter.go:80" plugin=/plugins/velero-plugin-for-aws
time="2025-10-31T08:43:48Z" level=error msg="backup failed" backuprequest=k8s-test/mybackup2 controller=backup error="rpc error: code = Unknown desc = error putting object backups/mybackup2/velero-backup.json: operation error S3: PutObject, https response error StatusCode: 400, RequestID: 810d40e0:197daed0e80:109369:bdd, HostID: c975f9a8e3ac4bab68ba741207576a74f6ff84f959c6a73c8f6baa13e4af018f, api error XAmzContentSHA256Mismatch: The Content-SHA256 you specified did not match what we received" logSource="pkg/controller/backup_controller.go:291"

Additional Context

This seems to be an issue we have seen for other S3 backends and Velero velero-io/velero#7696. Trying their suggestion from the issue and setting checksumAlgorithm: "" resolved the issue

spec:
  accessMode: ReadWrite
  config:
    checksumAlgorithm: ""
    region: default
    s3ForcePathStyle: "true"
    s3Url: https://s3.scc.kit.edu:9021

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-8123.

This message was autogenerated

[kimwnasptd]

We had a sync today about this one.

The suggested way forward would be to add a new config option in the charm like aws-checksum-algorithm for configuring this field of the BackupStorageLocation CR that gets created by the Velero Operator charm.