The LUIS computing cluster is a heterogeneous general purpose system designed for a variety of workloads. All nodes in a sub-cluster (“partition”) are interconnected using Mellanox Infiniband (at least QDR) non-blocking fat tree network.

we use SLURM as the job scheduler

By policy, compute nodes cannot access the internet. If you need an exception to this rule, contact cluster support with a reason, information about the IP address (must belong to the university network!), port number(s) and protocol(s) needed as well as the duration and a contact person.

However, the compute nodes have access to cloud storage systems provides by LUIS. For detailed information please refer to the Rclone usage instructions.

NFS based HOME (for home directories) and Lustre based BIGWORK (for temporary files during computations) storage systems are available on all compute nodes. PROJECT is only available on Login and Transfer nodes.

you will notice that the columns “(useable) Memory/Node (MB)” and “Memory Total (GB)” differ slightly, which takes into account the difference of total physical memory per node vs. the memory configured in the batch scheduler SLURM avilable to jobs. The latter number is smaller since the operating system needs memory, too. If you want to autoritatively find out the maximum allocateable memory per node in SLURM, use the command scontrol show nodes <nodename>, e.g. “scontrol show nodes amo-n001” for a node in the Amo partition, and look for the “mem=” parameter.

to avoid unbalanced node allocations, there is another limit, which is the maximum memory that may be requested per cpu core. The scheduler automatically adjusts your ressource request if you request more memory per core than what is configured, resulting in more cores being requested. That mechanism avoids having nodes with no memory but some cores left that would effectively be unuseable. To see how much memory is allocateable per core, use the command scontrol show partition <partitionname>, e.g. scontrol show partition amo and look for the “MaxMemPerNode=” parameter. For example, if you were to request a job using 4 cores and 40 GB of total memory on an Amo node, the scheduler would change this to request 8 cores, since the configuration limits memory requests to 5120 MB/core and 8*5120 MB = 40.960 MB to keep the allocated cores/memory balanced.

The line “FCH” in this table aggregates the nodes we run for various institutes of the LUH under the conditions of the service "Forschungsclusterhousing". They contribute significant additional power to the cluster, mostly during the night and over the weekend, but are usually reserved exclusively for institute accounts on week days. Your jobs have a chance of running in the night when they request less than 12 hours of walltime, or during weekends, for jobs that request less than 60 hours.