Virtual docker environment

From docker.com: "A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings."

So containers are very useful. Unfortunately there are lots of security concerns, if docker runs on multi-user systems. That's why on Elwetritsch singularity is used instead. The good news is, that docker images can be easily converted into singularity images.

To allow Elwetritsch users to build their own docker images right on Elwetritsch file systems, we offer a personalized virtual docker environment without root (for security reasons). This page explains in short examples how to create your own virtual machine (vm), build your own docker image on it, convert this image into a singularity image and then run that image on Elwetritsch with and without SLURM.

For general questions about docker and singularity please visit the website of both software products (Docker and Singularity).

1. Create your own docker vm with rz-docker

All of the following is done on Elwetritsch on one of the login nodes. First, get to know rz-docker

	[~]#rz-docker
	Welcome to the RPTU Dockerlab
	=============================
	
	Please use one of the following commands:
	* check   : check the state of your Docker VM
	* init    : initialize your Docker VM
	* start   : start your Docker environment
	* login   : log in to your Docker VM
	* reboot  : reboot your Docker VM
	* destroy : destroy and give back your Docker VM

If you haven’t used rz-docker before:

	[~]#rz-docker check
	ERROR: You do not have a Docker VM assigned.

That means you need to do a onetime initialization of your docker vm:

	[~]#rz-docker init
	Initialization has been requested.
	 * use 'rz-docker check' to check the state of your Docker environment.

The initialization can take a few minutes, depending on resources:
```
	[~]#rz-docker check
	Initialization of Docker VM requested.
	
```

When initialization has finished rz-docker check will tell you how to proceed:

	[~]#rz-docker check
	Your Docker VM docker018 is ready.
	 * use 'rz-docker start' to start your Docker environment.
	 * use 'rz-docker login' to log in to your Docker environment.

So to start and log in just follow the instructions of rz-docker (see next part of this documentation)

2. Start your docker vm and log in to it

	[~]#rz-docker start
	The authenticity of host 'docker018 (10.255.110.18)' can't be established.
	ECDSA key fingerprint is SHA256:0RkXKjU7AUcKRPvQ/qWBJ0YiQEzFwIfMN43EA2mBsWI.
	ECDSA key fingerprint is MD5:99:b0:2d:71:36:b6:9b:2f:ad:45:05:df:f3:da:36:cf.
	Are you sure you want to continue connecting (yes/no)? yes
	Docker environment has been started, please login with 'rz-docker login '.

Caution: Please accept the fingerprint of the docker vm with “yes” if necessary.

You now enter the newly created vm and leave Elwetritsch.
```
[~]#rz-docker login
```
You can leave the vm at any time with exit and return to it with rz-docker login.

3. Create your docker images inside your vm

All of the following is done on your personal docker vm (it won't work anywhere else!). If you want to use your created singularity images on Elwetritsch later on, it's best to change to your desired filesystem/folder before you begin. For really very small images your /home may be ok, but most of the time you want your images in /work. Of course you can also move your images later to wherever you want them to be.
```
[~]#cd
```
```
[~]#cd /work/<your-login-name>
```
You can work with standard Dockerfiles if you like. Here is a small example:
```
	# Dockerfile
	FROM fedora:latest
	
	RUN yum -qy update
	RUN yum -qy install banner
	
	CMD banner "Hello HPC"
	#
	
```
If you don't want to create that file you can simply download the Dockerfile. The Dockerfile must have this exact name and needs to be in the same folder where you do the following steps (otherwise you need to provide -f /path/to/a/Dockerfile after docker build).

Build your container:
```
docker build -t hello-hpc .
```
Run and test your container:
```
docker run -it --rm hello-hpc
```
Save your container image:
```
docker save hello-hpc -o hello-hpc.tar
```

4. Convert your docker image to a singularity image and test it

Convert docker image to singularity image (.simg):

singularity build hello-hpc.simg docker-archive://hello-hpc.tar

Run the converted image with singularity:
```
singularity run hello-hpc.simg
```

5. Run your singularity image on the HPC cluster Elwetritsch with rz-singularity

Return to one of the login nodes and start rz-singularity with your new image:

[~]#rz-singularity /full-path-to-image/hello-hpc.simg

Caution: Please make sure you provide the full path to the singularity image (.simg) starting with /!

Starting it as a SLURM job:
```
[~]#sbatch –n 1 –t 01:00 --wrap “rz-singularity /full-path-to-image/hello-hpc.simg”
```
```
Caution: Please make sure you provide the full path to the singularity image (.simg) starting with /!
```
Of course, you can also use SBATCH files like those that you use with your other SLURM jobs. After setting your #SBATCH parameters, the command to be run in your script is rz-singularity /full-path-to-image/hello-hpc.simg.

Main Navigation

Contents