High Speed Network

In order to provide enough bandwidth and short latencies to parallel (MPI) jobs the cluster is provisioned with a High Speed Network. Actually there are 2 High Speed Networks with similar characteristics but different vendors and these networks are not compatible. Therefore we try to prevent jobs that run in parallel on these different types.

Strictly speaking it is a Infiniband and an Omni-Path Fabric. You may recognize a nodes connectivity with help of the command

ibstat -l

which will return something like mlx5_0 in case of Infiniband and hfi1_0 in case of Omni-Path.

Normally you don't have to care about these details except:

OpenMP and Intel 2019

there the following settings have to be used:

source mpivars.sh release_mt
export FI_PROVIDER=verbs