Can't delete snapshot if snapshot is missing in truenas

[rmb938]

If a snapshot was deleted directly in truenas, it can not longer be deleted in openstack

2023-01-28 18:35:59.059 390037 INFO cinder.volume.drivers.ixsystems.iscsi [None req-44fae453-482e-4004-b645-00f6ae36e79c fff44df7b41d4fd39edf513ca497a8c4 ba20d749b68045bba695de9dd0123471 - - - -] iXsystems Delete Snapshot
2023-01-28 18:35:59.094 390037 INFO cinder.volume.drivers.ixsystems.freenasapi [None req-44fae453-482e-4004-b645-00f6ae36e79c fff44df7b41d4fd39edf513ca497a8c4 ba20d749b68045bba695de9dd0123471 - - - -] Error returned from server: "{'message': 'Snapshot tank/vms/volume-2dbadf00@snap-18f1a640 not found', 'errno': 14}"
2023-01-28 18:35:59.113 390037 INFO cinder.message.api [None req-44fae453-482e-4004-b645-00f6ae36e79c fff44df7b41d4fd39edf513ca497a8c4 ba20d749b68045bba695de9dd0123471 - - - -] Creating message record for request_id = req-44fae453-482e-4004-b645-00f6ae36e79c
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server [None req-44fae453-482e-4004-b645-00f6ae36e79c fff44df7b41d4fd39edf513ca497a8c4 ba20d749b68045bba695de9dd0123471 - - - -] Exception during message handling: cinder.volume.drivers.ixsystems.freenasapi.FreeNASApiError: FREENAS api failed. Reason - Unexpected error:FREENAS api failed. Reason - Unexpected error:Error while deleting snapshot: 422:Unprocessable Entity
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/volume/drivers/ixsystems/common.py", line 383, in _delete_snapshot
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     raise FreeNASApiError('Unexpected error', msg)
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server cinder.volume.drivers.ixsystems.freenasapi.FreeNASApiError: FREENAS api failed. Reason - Unexpected error:Error while deleting snapshot: 422:Unprocessable Entity
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server 
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server 
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 221, in wrapper
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     skip_clean = func(self, context, snapshot, *args, **kwargs)
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "<decorator-gen-725>", line 2, in delete_snapshot
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/coordination.py", line 200, in _synchronized
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     return f(*a, **k)
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 1364, in delete_snapshot
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     self.force_reraise()
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     raise self.value
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/volume/manager.py", line 1350, in delete_snapshot
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     self.driver.delete_snapshot(snapshot)
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/volume/drivers/ixsystems/iscsi.py", line 250, in delete_snapshot
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     self.common._delete_snapshot(freenas_snapshot['name'],
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/cinder/volume/drivers/ixsystems/common.py", line 385, in _delete_snapshot
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server     raise FreeNASApiError('Unexpected error', e)
2023-01-28 18:35:59.126 390037 ERROR oslo_messaging.rpc.server cinder.volume.drivers.ixsystems.freenasapi.FreeNASApiError: FREENAS api failed. Reason - Unexpected error:FREENAS api failed. Reason - Unexpected error:Error while deleting snapshot: 422:Unprocessable Entity

Obviously this should never happen, but it was accidentally done and the only way to delete the snapshot from openstack was to re-create the snapshot in truenas with the expected name.

[YiHuangDB]

Hi Ryan, thank you for reporting this issue. We will evaluate this issue and enhance driver stability for snapshot related issue soon.

[YiHuangDB]

Hi Ryan, this issue is now fixed with PR #34, feel free let us know you have any further questions or comments. Thanks for your reporting this issue as well.