Implement bin config dump to file

[aliemen]

closes #242

This PR implements a new writer, Structure/BinningConfigWriter.h, that prints the binning configuration to a .json file attached to a BinningCmd object for every n-th timestep. The .json file has the following format:

[
  {
    "step": 0,
    "time": 0,
    "preMerge": true,
    "xMin": 0.15340109944906014,
    "binCounts": [
      1, 0, 0, 0, 1, 1, 0, 5, 0, 3, 3, 2, 1, 4, 1, 6, 
      5, 5, 6, 7, 14, 20, 17, 19, 18, 30, 39, 41, 47, 41, 62, 77, 
      104, 116, 123, 131, 148, 183, 189, 208, 268, 267, 339, 341, 400, 400, 454, 485, 
      507, 605, 637, 612, 725, 716, 775, 749, 806, 821, 854, 865, 939, 922, 909, 858, 
      843, 896, 892, 864, 872, 819, 847, 790, 741, 698, 706, 630, 543, 584, 555, 513, 
      452, 411, 401, 381, 302, 270, 263, 226, 175, 169, 147, 119, 115, 94, 93, 70, 
      69, 55, 40, 36, 33, 28, 15, 20, 9, 12, 8, 3, 8, 2, 6, 2, 
      5, 1, 1, 1, 0, 0, 0, 1
    ],
    "binWidths": [
      0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 
      0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 
      0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 
      0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 
      0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 
      0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 
      0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 
      0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296, 0.00065572253938711296
    ]
  },
  {
    "step": 0,
    "time": 0,
    "preMerge": false,
    "xMin": 0.15340109944906014,
    "binCounts": [
      32768
    ],
    "binWidths": [
      0.078686704726453741
    ]
  },
  ...
]

This ensures easy readability, e.g. in Python. For every write action, it saves the pre-merged binning configuration as well as the merged config ("preMerge": false).

Features

• Binning config dump to file. During space-charge steps, binning configuration (from AdaptBins) is written to a JSON file configured via BinningCmd (e.g. DUMPBINSFILE, DUMPBINSFREQ).
• JSON format. Each snapshot is one array element with: step, time, preMerge, xMin, binCounts, binWidths. Output is pretty-printed and the file is always a valid, closed JSON array after every write (meaning it is valid, even when the program crashes!).
• Two dump points per step. Dumps are triggered in PartBunch::computeSelfFields: once after full rebin (pre-merge histogram) and once after adaptive merge (post-merge), when dumpBinConfig(true) and dumpBinConfig(false) are called.
• A shared DataSink is passed into PartBunch at construction (from TrackRun/ParallelTracker). DataSink::dumpBinConfig(...) delegates to a dedicated writer; no use of OpalData inside computeSelfFields for this. In the process I also removed OpalDistribution_m from the bunch as it is unused (handled completely by ParallelTracker).

Changes to the input file, i.e. BinningCmd:

BINS1: BINNING, 
    MAXBINS      = 120, 
    DESIREDWIDTH = 0.2, 
    BINNINGALPHA = 1.1, 
    BINNINGBETA  = 1.6, 
    PARAMETER    = VELOCITYZ, // Could be "VELOCITYZ", "POSITIONZ", "PZ" or "GAMMAZ"
    DUMPBINSFILE = "data/bin-dump.json",
    DUMPBINSFREQ = 1; 

You can now append an (optional) file path and dump frequency. The dump will only be performed if a file path is given (otherwise the variable defaults to DUMPBINSFILE="NONE", so no dump).

Changes:

• DataSink has a new function dumpBinConfig(step, time, preMerge, binCounts, binWidths, xMin, fileName) that prints a data row to file on rank 0.
• PartBunch constructor takes std::shared_ptr<DataSink> dataSink;, computeSelfFields calls dumpBinConfig(true) and dumpBinConfig(false) instead of bins->print().
• TrackRun: initDataSink() builds/gets global DataSink and sets ds_m;. PartBunch and ParallelTracker are constructed with ds_m, now a shared pointer.
• BinningCmd::update(): when dumping is enabled, ensures dump filename has .json suffix and validates DUMPBINSFREQ;.
• BinHisto gets a new AdaptBins::getBinConfigHost function that writes the global bin configuration into a host std::vector for the

Testing

I have a new test file Drift-4-open-bindump.txt that has the bin configuration logging enabled, if you want to test it. If you just want to look at the new output, have a look at bin-dump.json (should also look nice in the browser!).

Looks good on GPU (Daint, GH200). Output also looks good, including the binning configuration.

=====================================================================
TEST SUMMARY REPORT
=====================================================================

Format: TEST_NAME[RANK] ... STATUS

✓ Drift-1-bins[1] ... OK
✓ Drift-1-bins[2] ... OK
✓ Drift-1-bins[4] ... OK
✓ Drift-1-fromfile[1] ... OK
✓ Drift-1-fromfile[2] ... OK
✓ Drift-1-fromfile[4] ... OK
✓ Drift-3-open[1] ... OK
✓ Drift-3-open[2] ... OK
✓ Drift-3-open[4] ... OK
✓ Drift-3-periodic[1] ... OK
✓ Drift-3-periodic[2] ... OK
✓ Drift-3-periodic[4] ... OK
✓ Drift-4-open-bins[1] ... OK
✓ Drift-4-open-bins[2] ... OK
✓ Drift-4-open-bins[4] ... OK
✓ Drift-4-open-bindump[1] ... OK
✓ Drift-4-open-bindump[2] ... OK
✓ Drift-4-open-bindump[4] ... OK
✓ FODO[1] ... OK
✓ FODO[2] ... OK
✓ FODO[4] ... OK
✓ Solenoid-1[1] ... OK
✓ Solenoid-1[2] ... OK
✓ Solenoid-1[4] ... OK

=====================================================================
Total: 24 tests | Passed: 24 | Failed: 0
=====================================================================

Also looks good on CPU:

=====================================================================
TEST SUMMARY REPORT
=====================================================================

Format: TEST_NAME[RANK] ... STATUS

✓ Drift-1-bins[1] ... OK | 0.138s
✓ Drift-1-bins[2] ... OK | 0.151s
✓ Drift-1-bins[4] ... OK | 0.350s
✓ Drift-1-fromfile[1] ... OK | 0.278s
✓ Drift-1-fromfile[2] ... OK | 0.289s
✓ Drift-1-fromfile[4] ... OK | 0.306s
✓ Drift-3-open[1] ... OK | 0.866s
✓ Drift-3-open[2] ... OK | 0.773s
✓ Drift-3-open[4] ... OK | 0.750s
✓ Drift-3-periodic[1] ... OK | 0.489s
✓ Drift-3-periodic[2] ... OK | 0.523s
✓ Drift-3-periodic[4] ... OK | 0.740s
✓ Drift-4-open-bins[1] ... OK | 5.790s
✓ Drift-4-open-bins[2] ... OK | 4.699s
✓ Drift-4-open-bins[4] ... OK | 4.319s
✓ Drift-4-open-bindump[1] ... OK | 5.711s
✓ Drift-4-open-bindump[2] ... OK | 4.553s
✓ Drift-4-open-bindump[4] ... OK | 4.412s
✓ FODO[1] ... OK | 7.234s
✓ FODO[2] ... OK | 11.098s
✓ FODO[4] ... OK | 21.885s
✓ Solenoid-1[1] ... OK | 17.949s
✓ Solenoid-1[2] ... OK | 22.829s
✓ Solenoid-1[4] ... OK | 48.246s

=====================================================================
Total: 24 tests | Passed: 24 | Failed: 0
Total runtime: 2m 44.900s
=====================================================================

Also ran all of the regression tests on cpu-serial, all of the ones we would expect it from passed (110/120). Oh and of course we also have some new unit tests.

Additional Information

Here is a small animation that can be created easily with this new output file:

If shows really nice how the flattop distribution propagates in the CONSTANTEFIELDCAVITY (note that x in this plot is normalized velocity $\frac{v}{c}$, where $[v] = \frac{\mathrm{m}}{\mathrm{s}}$).

[aaadelmann]

The IF (rank 0) must be in the body of this function!

[aliemen]

The if (rank 0) check is inside DataSink::dumpBinConfig similar to how some of the other dump functions handle it.

But you're right, it's at minimum inconsistent with the doxygen of the function you pointed out. For the moment I removed the rank-0-note from the doxygen, since the function works just fine on any rank (it also doesn't do MPI, the global host histograms exist through the binning routine anyways). But if you think that it would be better, I could of course add the redundant if (ippl::Comm->rank() != 0) { return -1; } inside AdaptBins::getBinConfigHost, just in case?