i tried to use docker-compose to create 1 master and 1+ worker containers, however, it didn't work. any ideas? the container just dies (spark_master) and i can't get to the logs because it's too late.
i have something like the following.
spark_master:
image: epahomov/docker-spark
container_name: spark_master
command: /start-master.sh
ports:
- "7077:7077"
- "8080:8080"
spark_slave:
image: epahomov/docker-spark
command: /start-worker.sh
links:
- spark_master
would be nice to do docker-compose scale spark_slave=2
i tried to use docker-compose to create 1 master and 1+ worker containers, however, it didn't work. any ideas? the container just dies (spark_master) and i can't get to the logs because it's too late.
i have something like the following.
would be nice to do
docker-compose scale spark_slave=2