ENH: write 'done' file after successfull runs

[dengemann]

It would be nice to have a stupid 'done' file in a run folder.
This would come in handy to programmatically delete runs that were unsuccessful.

[kingjr]

I would write this as the last line of the log?

In the same spirit, I think it'd be useful to log time at (and duration between) the report creation and saving to allow a quick view of how long it took.

[dengemann]

I would write this as the last line of the log?

It has to happen in the last line of the script. The clean way to go is a finalize_logging/provenance_tracking runction that you call in the end., it could save the report and write a done file and do other final logs relate to total time elapsed, etc.

[kingjr]

It has to happen in the last line of the script. The clean way to go is a finalize_logging/provenance_tracking runction that you call in the end.,

+1

it could save the report and write a done file

I'm not super convinced of the elegance of having an additional file, as opposed to adding 'done' at the end of the log.

[dengemann]

I'm not super convinced of the elegance of having an additional file, as opposed to adding 'done' at the end of the log.

It's indeed more elegant. With a simple glob command in the shell you can look for folder that have the done file:

ls -l results/*/done

In the other case you need to look into each log.

[kingjr]

No, but I see where you're coming from. It's just that I don't see this strategy in the packages I use; eg. GNU parallel. But this isn't a strong argument, so go for it.

[dengemann]

I'm inspired by our EEG pipeline where this simple practice turend out to be quite handy. When GNU paralllel finishes we write a done file ;)

[kingjr]

I know (;