While the cluster is mainly designed to run batch jobs without user interaction, we realize that in some cases it is necessary to also interactively work on a node - for example when setting up a 3D-model that is too large to fit into your workstation's memory in a simulation software, or if you need to postprocess/filter large amounts of data that has been generated by a job in your BIGWORK directory. To facilitate that, we have installed a special node that has specifically been set up to support interactive/graphical use. We will watch its usage and possibly add further nodes in the future, depending on user demand. CURRENTLY, THIS IS JUST A TEST.
Please note: in this initial phase and due to the very limited resources, only one visualization session (job) lasting no longer than 3 hours is allowed per user at the same time to give everyone a chance to test the service. When the session expires, it will be terminated automatically and without warning (!). A job can request a maximum of 8 CPU cores and 32 GB of main memory, and a maximum of three simultaneous sessions can run on the visualization node at any time to limit the influence of other users on the experience of others.
Please note: If you do not require an intensive 3D hardware acceleration, please use a regular remote desktop session instead to interactively work with your GUI applications (accessed via the menu Interactive Apps > Cluster Remote Desktop on the web portal).
The traditional approach to remote data visualization relies on tunneling the X11 protocol through an SSH connection to run graphical applications. That means adding up latencies between client and server, resulting in slow reactions and a “sluggy” handling of large 3D-geometries in particular. The approach of a visualization server is to perform all rendering on the server's graphics hardware, transferring only the resulting 2D-images to the remote workstation for display. This permits to operate 3D applications efficiently over standard network connections. In addition, the local workstation neither requires a special dedicated graphics card nor large memory or an installation of 3D processing tools.
Since the cluster visualization servers are managed by SLURM via the partition
vis, you need to submit a job to this partition to get access to the server's resources. Currently, the
vis partition consist of only one OpenGL-capable server with a single NVIDIA Quadro P4000 GPU, 20 CPU cores and 92 GB of main memory. The node runs a 3D X server and has all the required software tools
installed, incuding TurboVNC (Virtual Network Computing) and VirtualGL (an open source software that intercepts 3D remote-rendering commands). The server has a fast connection to both the BIGWORK and PROJECT file-systems (where your datasets are supposed to be placed).
To use the remote visualization service of the LUIS cluster, you need to authenticate to the cluster's
web portal and on the dashboard activate the Cluster Visualization Desktop item in the Interactive Apps menu, cf. figure 1. On the service page that opens, leave the checkbox
Enable OpenGL desktop session deactivated for the time being (see below the section Optimization).
Set the remaining session(job) parameters according to your requirements and click the Launch button at the bottom of the service page.
You will be redirected to the session information page, where you can see the status of your job, see the top window in figure 2. Wait for the job to get into running state, then click Launch Cluster Visualization Desktop, see the bottom window in figure 2, to connect to the remote VNC session that is allocated on the visualization server.
Once in the desktop session, invoke a terminal (
Terminal Emulator or similar, e.g.
Terminator from the sub-menu Applications > System) from the menu Applications at the top-left of the screen as shown in figure 3.
As a first test, we'll run the OpenGL program
glxspheres64 supplied with the VirtualGL package (cf. figure 4):
[user@vis-n001 ~]$ glxspheres64
You should see a new window containing some colourful animated spheres. Notice the frame rate. Exit the program (
ESC) and start it anew, this time using:
[user@vis-n001 ~]$ vglrun glxspheres64
You should see a noticeable difference in framerate and measured performance. Exit the tool. You do not need to do this every time, this was just a demonstration to give you a feel for the difference to expect.
You can now activate the application you want to use (by loading the appropriate modules, activating your conda environment, etc.) and run it using the
[user@vis-n001 ~]$ vglrun <your-application> <your-application-options>
To check whether the graphics cards of the server are actually being used by your application, open a second terminal
window and run the command
nvtop. nvtop (Neat Videocard TOP) is a
(h)top -like tool that provides real-time
information about the usage of NVIDIA and AMD GPUs as well as the processes executing on the GPUs (see the right tab
of the terminal window in figure 4).
nvidia-smi utility may also be useful.
Here are some further optimizations you could try:
Enable OpenGL desktop sessionin the session setup, the X server itself will be started using
vglrun. This should have the effect that software started from within a terminal opened on that desktop should automatically start with OpenGL acceleration enabled (without explicitly using
vglrun). In some cases, this may not work as expected — then just leave the option deactivated and start your software using
vglrun -fps 60 glxspheres64,
vglrun -fps 20 glxspheres64and
vglrun -fps 10 glxspheres64.
-qcould be of use over a limited connection.
Here are instructions on how to run some applications to enable hardware accelerated OpenGL rendering.
In a terminal window, load the ABAQUS module and type:
vglrun abaqus cae
Load the ANSYS module and run:
vglrun fluent -driver opengl
If ANSYS Fluent is launched from ANSYS Workbanch, the following variables must be set, otherwise ANSYS Fluent will not use hardware rendering.
export FLUENT_WB_OPTIONAL_ARGS="-driver opengl" export CORTEX_PRE=/opt/VirtualGL/bin/vglrun.
Load the COMSOL module and execute:
vglrun comsol -3drend ogl
Load the GaussView module and execute:
Load the MATLAB module and execute:
vglrun matlab -nosoftwareopengl
module load GCC/11.2.0 OpenMPI/4.1.1 ParaView/5.9.1-mpi vglrun paraview
module load GCC/10.2.0 OpenMPI/4.0.5 VMD/1.9.4a51 vglrun vmd