Tracking changes to delete method (purge and permanently delete vs "Status Deleted")

[wrandtkeflvc]

Update 3/11: Please see case notes. May need to be a project.

Original Case Description:

In testing a new Islandora deletion method (delete status instead of permanent deletion) a problem with orphaned child issues was discovered. See https://docs.google.com/document/d/1UsdaEcqeqKoweQa1BAOfZSTbMZ5vMZUpmBel7yYppOs/edit?usp=sharing for the full testing results.

Please see case notes for full original email with formatting.

From: Lydia Motyka
Sent: Thursday, September 6, 2018 14:40
To: FLVC Help Desk help@flvc.org
Subject: Islandora deleted status: orphaned serials children when parent is marked as deleted

This is for the Student and General Applications queue, Islandora category. Gail Lewis is familiar with this problem

In testing a new Islandora deletion method (delete status instead of permanent deletion) a problem with orphaned child issues was discovered. See https://docs.google.com/document/d/1UsdaEcqeqKoweQa1BAOfZSTbMZ5vMZUpmBel7yYppOs/edit?usp=sharing for the full testing results.

This problem (highlighted) can be recreated by following the testing actions:

Serials test

• Results of deletion of parent: will this create orphans? I don’t think it’s possible to re-link issues or hierarchy to a new title, so by rights the children should be deleted. Keep list of child object PIDs.
• PIDs of test serial:
• Spanish River Papers (serial title) dev:1706
o Volume 1(stub) dev:1707
 Number 1 (stub) dev:1712
• PDF islandora:225
 Number 2 (stub) dev:1713
• PDF islandora:226
 Number 3 (stub) dev:1714
o Volume 2 (stub) dev:1708
 Number 1 (stub) dev:1715
• PDF islandora:227
 Number 2 (stub) dev:1716
• PDF islandora:228

Test #1: Delete top-level stub and examine results
• Volume 2 (stub) dev:1708 - Properties -> Permanently remove…->Delete
• Results:
o Volume 2 stub and children no longer appear in tree
o Number 1 (stub) dev:1715 has status “active” (I believe that we knew this would happen.) Gail’s history info: we noticed that regular deletes aren’t recursively applied to children. Known issue that we would have to address ourselves, perhaps with a button “Do you want to delete children as well?” Not a show-stopper.
o PDF islandora:227 (child) has status “active”, and accessing it directly by PID displays breadcrumbs “Islandora Repository >> …”, so you can’t get to any of the parent stubs or title via breadcrumbs, but clicking on “view all issues” will return you to the tree. (This is because the child is orphaned. Search is not aware of orphans: if it’s active it’s indexed. Updating orphaned child will cause site info to be lost - if parent is deleted. Known, ongoing problem.)
o Searching “Spanish River Papers” retrieves the PDF in search results. (because it’s indexed, see above)
• Returning dev:1708 (Volume 2 stub) status to “active” via Properties->State->Active
• Results:
o Volume 2 stub and children appear in tree
o Breadcrumbs return to normal

[wrandtkeflvc]

I went through Lydia's notes on deletions on development and test (where changes from delete method to status deleted are in place) and compared against production.

There aren't any new problems with status deleted.

The kinds of problems with status deleted are when a top level object is deleted, and it has multiple parts, it may be the case that components of that object are not deleted. For example, deleting parts of a Serial likely will leave the Serial PDFs intact with a state "Active" and they show up in search results logged in or out. That happens with setting status deleted or with deleting and purging the parent or intermediate serial objects - same problematic behavior before and after change.

I didn't see any reason not to roll out changes, based on testing.

However, there should be some kind of workflow or interface for people to get to deleted objects.

[wrandtkeflvc]

From discussion at yesterday's FLVC Islandora Developer's meeting ( notes at https://docs.google.com/document/d/1aq90Vei3lFCejVTkgUQLL456yrll-eUs7YKiHprf9U4/edit ), here are open issues that should get looked at before moving deletion changes out to production:

Deletion:

• Garbage collection isn’t set up.
• (Also, there has never been a garbage collection for orphaned objects.)
• Clicking to “Islandora” -> “Manage Deleted Objects” gets a report of all status D objects.
• What should Purge button do?
• Can non-FLVC/FALSC employees get to that menu?

At this time (Fall 2019), the reasons for changing from delete and purge to status deleted are to allow restoration of backups if someone inadvertently deletes many objects.

The current way of restoring deleted objects is to grab from coop. Coop has not been synced to production since January 2019, when Blazegraph was installed on coop. Recovery from tape backup could be OK (but labor intensive) if it's single part objects, like PDFs or pics. If it's a serial or newspaper with many components and levels of hierarchy, that's much more labor intensive and difficult.

(Previously, a motive was supporting tombstones, which was a prerequisite for sharing records to Encore's OAI-PMH harvester. The Encore contract was canceled in Spring 2018.)