Docker for experiments (IGRIDA/Grid'5000)

[FAMILIAR-project]

Right now we're using Jupyter notebooks on our personal machine: it's nice for prototyping, but for computing in the large we need a more efficient and portable solution.

I propose we build a Docker image that can launch experiments we want.

As a first proof of concept, I propose we are capable of producing the results of feature_important.ipynb.

• basic Docker image (including Docker file to reproduce it) with Python 3, scikit-learn, pandas, numpy, etc.
• a way to transform notebook as pure Python 3 programs
• strategies to retrieve results (output of the console?)

Then the roadmap:

• deploy on IGRIDA (including the retrieval of results)
• a script to easily update the image to control the experiments we want to perform (inside the image through a git pull?)
• deploy in Grid5000 (if needs be)

[FAMILIAR-project]

We are at the step where we can execute any Python script of our git repo:
docker run -it --rm macher/sklearntux python3 size-analysis-fast.py
right now I have pre-download the .pkl dataset, but it could well be a .csv

[FAMILIAR-project]

Due to our refactoring (pkl dataset), the Docker image should be revised, but it's not a big deal.