feat: storing a list of purged tasks on persistent storage

[nicholasturner1]

It can be costly when a large job fails in the middle and we cannot recover its progress. When we cancel a DAG run and purge task queues, it might be useful to store the contents of those queues on persistent storage somewhere so that we can start from where the previous job stopped.

[nicholasturner1]

Discussed this with Ran on slack. The current goal for this is to move tasks to use the redis database to track task completion. Seuron passes workers a run id to log task completion, and workers check this database before running a new task to see if they can skip it. Ran will add a flag to determine whether or not progress should be wiped from a database on future runs, and he suggests using periodic backups of this database to a hard drive if we require severe fault tolerance.