cnvrg.io supports uploading custom docker images, and using them to run experiments.
This enables any data scientist on the team to manage their own custom images for different experiments, while still using the benefits of cnvrg's experiments.
cnvrg.io supports two types of docker images upload -
- Dockerfile: Upload the content of the dockerfile to cnvrg
- Docker Tar image: Upload the docker tar file after exporting it using the docker export command
In order for the custom docker image to be supported, the image should contain the following:
1. An extension of an ubuntu distribution
2. The latest cnvrg cli is installed- you can use the following commands:
RUN apt-get update
RUN apt-get install wget -y
RUN apt-get install ruby-full -y
RUN gem install bundler --no-ri --no-rdoc && gem install cnvrg --no-ri --no-rdoc
3. Git is installed using the following command:
RUN apt-get install git -y
4. sudo is installed:
RUN apt-get install sudo -y
On Cloud supports, cnvrg will then create a dedicated aws AMI for the custom images. This could take a few minutes, and an email notification will be sent to the user when the image is ready to be used.
On on-premise supports, cnvrg will build the image on the machines, and will verify it was built successfully before running experiments.
Uploading custom images:
Navigate to the organization settings page, and click on the images tab
Click on new image, and fill out the form
make sure to fill out the following:
- The correct name and tag as it will appear in the image
- The image workdir
- The image user's- i.e. root/ubuntu
- choosing if the image needs a gpu support
- Dockerfile / docker tar
Once uploading, you will see the image progress in the newly created image
Using custom images to run experiments:
When the image is in a "Running" state, you'll be able to choose it from the images list when running experiments from the web:
Or from the cli:
cnvrg run --image=IMAGE_NAME python3 mnist.py