eGo Performance

[maltesc]

The recently performed eGo runs show that eGo has a performance problem. Especially the simulation of some MV grids is extremely time-consuming and delays the overall calculation.

Therefore ideas are welcome in order to improve eGo's performance...

[maltesc]

Currently I am working on improving the logging messages in order to (at least) give the user a better overview on where the calculation is stuck...

[ulfmueller]

Do you see a possibility to exclude certain long-lasting problematical mv grids on the run (after a threshold of calculation hours) and create an overall result from the remaining mv grids? Or would there be a new mv clustering necessary to get the correct weighting of the new number of mv grids considered?

[maltesc]

To you see a possibility to exclude certain long-lasting problematical mv grids on the run (after a threshold of calculation hours) and create an overall result from the remaining mv grids.

I think this is possible. For understanding: In case one ore more MV grids fail to calculate during an eGo run, the overall results are extrapolated. The extrapolation takes into account the weighting of the grids. This means, if a grid with a very low weighting fails to calculate, this will have only very little effect on the final results.

Based on this idea I would say it is OK to define maximum calculation times per MV grid. However, instead of defining fixed values, I suggest to determine a maximum calculation time per grid based on the grids weightings. In my opinion, a grid with a small weighting should get a smaller maximum calculation times than grids with many representatives.
I assume that the most time-consuming grids are rare grids with very few representatives (@ulfmueller can you check this?). In this case they have little effect on the final results and could be skipped without any problems...

[maltesc]

What could be a reasonable maximal calculation time per grid? Or rather, a reasonable calculation time per cluster percentage, in case you agree on the suggested method

I suggest to determine a maximum calculation time per grid based on the grids weightings.

A value could be like 100 % cluster percentage in 24 hours - the time is then allocated to the clusters respectively...

[ulfmueller]

difficult question. It also depends on the problem size i.e. the size of the given etrago problem. I would leave this decision to the user and write the max_calc_time_per_grid into the settings.

[ulfmueller]

I assume that the most time-consuming grids are rare grids with very few representatives (@ulfmueller can you check this?)

I can try. But only for a sample...

[ulfmueller]

Your assumption seems to be valid. At least for the grid 2583 which took over 30 hours in one of my calculations it was only representing 16 grids. For the others I have trouble saying it because the logs do not really tell me when which grid started and ended its calculation.

[ulfmueller]

Thinking of it, I really like this feature. When do you think can you have this ready?