Enabling a Runner on HPC

Step 1: Creating a Runner in the Gitlab Server

To enable a runner on the HPC cluster, you first need to create a new runner in the Gitlab server. Follow these steps:

Open the Gitlab server (e.g. Datahub)
Navigate to Settings -> CI/CD -> Runners
Click on Create project runner
Enter a tag, e.g. hpc

Step 2: Configuring the Runner on the HPC Cluster

After creating the runner in the Gitlab server, you need to configure it on the HPC cluster. Follow these steps:

module load gitlab-runner/latest
gitlab-runner register \
  --url https://datahub.rz.rptu.de \
  --token glrt--.... \
  --executor shell

Important note: The token glrt--.... has to be exactly the same, as the one being created in Step 1.

Step 3: Starting the Runner on the HPC Cluster

After configuring the runner, you can start it. Run the following command:

rz-start-gitlab-runner

Step 4: Testing the Runner

To test the runner, you can create a minimal pipeline. Just add a new file .gitlab-ci.yml in your project root in the Gitlab Server. Here is an example:

stages:
  - test

variables:
  SLURM_PARTITION: "haswell-64s"
  SLURM_TIMELIMIT: "01:00:00"

before_script:
  - module load gitlab-runner/latest

test:
  stage: test
  tags:
    - hpc
  script:
    - echo "Allocating SLURM resources..."
    - srun --partition=$SLURMPARTITION --time=$SLURMTIMELIMIT --mail-type=ALL uname -a
    - echo "Releasing SLURM resources..."

Important note: The tag must be exactly the same as definded in Step 1.

Automatic Startup of the Runner on Login Node Reboot

To automatically start the runner on login node reboot, you can create a cron job. Run the following command:

crontab -e

Add the following line:

@reboot /cluster/rz/bin/rz-start-gitlab-runner &

Important note: This is just a quick start guide for users familiar with the usage of Gitlab and SLURM. If you're new to HPC or Gitlab this guide is not for you.

If you need more background information please check the websites of TUD for example.

Main Navigation

Contents