Experiments

2D Benchmark

It has appeared that the 2D benchmark does not reflect the true potential of some mechanisms that rely on higher dimensions. However, it remains a good first environment to try and see how an algorithm may behave.

Test performances

The main results are available in the "Results" folder.

We found that Evolution Strategies (and therefore POET) does not perform well with sparse reward. We also found that the selection of environment is beneficial to the agent learning, compared to a fixed set of environments and to randomly generated new environments at each generation. Lastly, we found that the Local iterations in NNSGA does not significantly improve the performances of the generalist population.

The last CollectBall past executions are to be found here.

Perspectives

The most important part of the work to be done is finding the best objectives to drive the development of agents as well as environments. Some important comparisons of our algorithm should also be made on a different environment, more rich and complex such as a 3d control task.

The algorithm should also be compared to a state-of-the-art algorithm of Reinforcement Learning such as DDPG, to put in perspective the benefits of an evolutionary approach.