Restructure postprocessing functions

[Chris-Pedersen]

These are currently hardcoded with DiRAC submission scripts. I have scripts to run postprocessings on Hypatia that are not currently integrated with the repo, but at some point we should decide how to restructure these, i.e. just hardcode the functions with Hypatia scripts now, or come up with some more generalisable framework.

[andreufont]

Hi @Chris-Pedersen - One of the things that has been in my to-do list for a while now is to move some of the simulations to the local cluster in Barcelona. I might finally have time to do this during the next couple of weeks, and I could use the oportunity to update these scripts (I want to make sure I know how to run them myself).

[andreufont]

@Chris-Pedersen - I'm already at the point where I'd like to run post-processing for a suite of sims, and I would like to see what scripts you have in Hypatia to do this. Each machine might need somewhat different setups, but that's ok.

[Chris-Pedersen]

Hi @andreufont I've pushed the changes I made to get the postprocessings to work on Hypatia to a PR #104 which will point you to the parts of the code that need to be modified to run on PIC. This will all need to be tidied up a bit in the future as lots of things are hardcoded. For example the spectra.json files have the original DiRAC directories for each snapshot hardcoded into them as well, which is redundant information now. Parts of the postprocessing code attempted to read this which I have edited out for now.

[andreufont]

Thanks. I'll take a look at that, it will probably be enough to get started.

[Chris-Pedersen]

A summary of the changes to be made to the postprocessing code:

1. Calculate p1d along all 3 axes and take the average.
2. Have one place to store a template submission script, that will then be used to generate and submit scripts for each stage of the postprocessings.
3. Set up scripts that will run the postprocessings for individual sims as well as the LH suite. This will make it easier to add the postprocessings for new test sims as we go.

I have made a start on 2 and 3, and happy to take over the PR #107 to address 1.

[andreufont]

1. Hi @Chris-Pedersen - If possible, I'd try to treat the 3 axes similarly as to how we treat the plus/minus boxes. We have 6 "views" of the same universe. Note that each of these 6 views will have slightly different mean flux as well, and we want to take that into account when combining P1Ds (see what we do for plus / minus). An important difference is that variables that are computed from the box (and not from the skewers) are the same in the three axes, like T0, gamma and k_F.

2. and 3) make sense.