Kategorie: ‘RWTH-HPC’
Changes to the Module System on CLAIX-2023 and CLAIX-2025
Several changes to core modules and the general handling of modules are introduced on Thursday, October 29th:
CLAIX-2023
As announced during the CLAIX-2025 pilot phase, the new module families utilized on CLAIX-2025 are adopted on CLAIX-2023, too. This means that any change between compilers and toolchain meta modules automatically switches out incompatible modules of the same kind as it already happens when loading an MPI module. For more information on this topic, please take a look at our introductory page for CLAIX-2025.
CLAIX-2025
We have had several reports of multi-node jobs using OpenMPI sporadically crashing on startup. The same jobs on the same nodes would sometimes run to completion but the crash occured more often the larger a job was and was caused by a failure in OpenMPI to correctly set the network layer settings for all ranks. We have implemented a workaround that prevents this crash.
Time limit of CLAIX c25 partition extended
The max runtime limit within the c25* partitions has been increased from 24h to 48h.
Slurm privacy rules enforced
We have enforced the Slurm privacy settings and only personal jobs and settings can be seen now. Exceptions cannot be made.
Important changes to RWTH-s project application procedures with AI/ML focus
Changes to default partition for Claix jobs that do not define it.
Dear HPC Users,
With the introduction of Claix 2025, we have changed the way default partitions are defined for jobs that provide none.
Prior to this change, the default partition was statically defined for all Slurm project accounts. A job submitted without a partition would fall back to this hard-coded default partition.
This has now changed. We now take a best-effort approach to define the default partition for each job dynamically, based on the login / submit node from which the job was submitted, the account used and the resources requested.
Jobs submitted from Claix23 machines will set a default c23ms partition for CPU only jobs and c23g partition for GPU jobs.
The same will now happen with jobs submitted from Claix25 machines (c25ms and c25g).
IH projects and login nodes will also be taken into account.
We still ask for users to try and explicitly define partitions for their jobs to avoid any undesired behaviors.
The best practice is therefore still to define partitions in the jobs explicitly.
This change should not affect the majority of jobs or how they are billed.
(English) Important Changes for HPC Accounts – Usage of the NHR community AAI
GPU TDP reduced on CLAIX 2023 nodes
As of the 30.06.2026 at 11:30 the maximum TDP of the H100 GPUs on CLAIX 2023 nodes has been reduced from the previous value of 700W to 600W.
We implemented this lower TDP to reduce the thermal load on the central cooling system. This central cooling system is operating hotter due to weather and age, causing some specific GPU nodes to trigger thermal protection shutdowns that in turn kill any running jobs on said nodes.
This change is noteworthy for users that perform performance measurements and would like to compare their previous measurements to new ones under the reduced TDP.
For normal usage of the GPUs of CLAIX 2023, this change should not affect validity of results and should provide close to stock times to solution.
Users should not expect longer runtimes in most cases. For extremely intense computations, please recheck if your workflows require more runtime and adjust your batch scripts accordingly. We nevertheless, do not expect this change to affect most users of GPUs and is of no consequence to non-GPU users.
ML compute quota now shown in GPU-h
We would like to inform users of the following changes to r_wlm_usage.
Users can now see that the r_wlm_usage and r_wlm_usage -q commands display different quotas for MPI (CPU Only) and ML (GPUs) projects. The r_wlm_usage table will display for the same project, two distinct sections as follows:
=== HPC partitions (CPU Core-h) ===
….
=== ML partitions (GPU-h) ===
….
These changes are mostly in how the values are displayed as GPU-h. The used quotas have not changed and we expect no negative effects to previous, current or future quota utilization. Already used and allocated quotas within ML (GPU) systems were converted with the pre-existing ratio of 24CPU-h per 1 GPU-h. This change is to help distinguish between CPU and GPU usage within the NHR directives.
Additionally the default GPU-h quota for the ‚default‘ project of every account running on GPU nodes has been set to 10 GPU-h.
This means that users can use their ‚default‘ CPU-h quota without using the GPU-h quota and vice versa. The same rules of over-quota apply to both types.
Firefox Replaced on Dialog Nodes
The Firefox web browser was uninstalled from all HPC cluster dialog nodes due to lacking security updates for the given system configuration. Moreover, Firefox was deployed the modules tree to maintain its availability until the update problems using the OS packages can be fully resolved. However, this strategy requires to load the „Firefox“ module using the command line first and executing it then on the command line as well.
To workaround these limitations, we are providing a wrapper script and a desktop definition, such that you should be able to continue using Firefox seamlessly without any further actions required.
You can track any disruptions or security advisories that may occur due to the aforementioned change in the RWTH High Performance Computing (HPC) category on our status reporting portal.
Runtime limits for GPU Jobs
On the 06.03.2026 we changed how many GPU nodes from the c23g partition are allowed to run user GPU Jobs with runtimes longer than 24 hours.
The change has the following main goals:
- Increase the throughput and reduce the waiting times for GPU jobs with runtimes lower than 24 hours.
- Encourage users to submit shorter jobs and make use of resilience methods like checkpointing if necessary.
- Reduce the maintenance downtime of GPU nodes.
The change effectively limits the amount of long running GPU jobs to only half the GPU nodes in the c23g partition. Long running GPU jobs are user GPU jobs with runtimes longer than 24 hours. GPU jobs with less than 24 hours of runtime are considered short and will be scheduled to all available GPU nodes of the c23g partition.
It is understood that the waiting times of long running GPU jobs will increase and we therefore encourage users to change their workflows to accommodate shorter running GPU jobs.
This change is necessary to improve QoS for users of the c23g partition and to allow for faster maintenance works on the GPU nodes.
