IT Center Changes

Update 08.02.24:
We have installed a bugfix release for the affected software component and enabled user namespaces again.

Dear users,

due to an open security issue we are required to disable the feature of so-called user namespaces on the cluster. This feature is mainly used by containerization software and affects the way apptainer containers will behave. The changes are effective immediately. Most users should not experience any interruptions. If you experience any problems, please contact us as usual via servicedesk@itc.rwth-aachen.de with a precise description of the features you are using. We will reactivate user namespaces as soon as we can install the necessary fixes for the aforementioned vulnerability.

A recently discovered flaw in the implementation of the Secure Shell (SSH) protocol lead to an attack vector called “Terrapin Attack” enables an attacker to break the integrity of the “secure shell” connection in order weaken the overall security. TL;DR To implement an effective counter measure against the attack, we have disabled the affected methods in the HPC cluster’s SSH configuration. Consequently, these methods cannot be used until further notice:

• Ciphers: ChaCha20-Poly1305
• MACs: Any etm method (e.g. hmac-sha2-512-etm@openssh.com)

Please adapt your configuration accordingly if your configuration is  relying on the methods mentioned above.

The attack is only feasible when a using either the ChaCha20-Poly1305 Cipher or a combination of a Cipher Block Chaining (CBC) cipher (or, in theory, a Counter Mode (CTR) cipher) combined with an encrypt then MAC (etm) message authentication code (MAC) method and the attacker has the ability to act as a man-in-the-middle. (Example: A security suite on your client machine may perform a deep packet inspection (per definition a (hopefully “good”) man-in-the-middle) to protect you from other threats.)

The Galois Counter Mode (GCM) AES ciphers are not affected.

We encourage you to employ strong encryption ciphers such as aes256-gcm@openssh.com and a sufficiently strong MAC method (e.g. hmac-sha2-256 or hmac-sha2-512) immune to the attack vector.

Note:

Due to a bug in the Windows OpenSSH client employing the umac-128@openssh.com MAC as default, we disabled the problematic method in the SSH server configuration as well to minimize issues when connecting to the HPC cluster. Until further notice, only hmac-sha2-512 and hmac-sha2-256 can be employed as MAC. Please adapt your configuration accordingly, if required, e.g.:

Ciphers aes256-gcm@openssh.com,aes256-ctr
MACs hmac-sha2-512,hmac-sha2-256

You can track any disruptions or security advisories that may occur due to the aforementioned change in the Email category on our status reporting portal.

We will introduce a mandatory MFA only access to the HPC cluster on the 15 January 2024.
From that day on, logins to *any* login nodes will only be possible with MFA.

If not done yet, please follow this step-by-step guide to configure your MFA token in the RegApp:

https://help.itc.rwth-aachen.de/service/rhr4fjjutttf/article/475152f6390f448fa0904d02280d292d/

We will also offer three dates for a brief introduction:

* Friday, 12. January 2024 ► 13:00 – 13:45: https://blog.rwth-aachen.de/itc-events/en/event/using-multi-factor-authorization-for-claix/
* Monday, 15. January 2024 ► 15:30 – 16:15: https://blog.rwth-aachen.de/itc-events/en/event/using-multi-factor-authorization-for-claix-2/
* Monday, 22. January 2024 ► 10:00 – 10:45: https://blog.rwth-aachen.de/itc-events/en/event/using-multi-factor-authorization-for-claix-3/

Furthermore, you can use the monthly HPC consultation hours for any further questions:

HPC Consultation Hour

For more background information about HPC & 2FA, please read our blog entry:

Protecting the HPC account with MFA

Please contact servicedesk@itc.rwth-aachen.de for any further questions.

During the last cluster maintenance, the OS of the HPC cluster was upgraded to Rocky Linux 8.9 due to the EOL of Rocky 8.8 to ensure continous update support for the systems.

The upgrade provides a modernized system base and security enhancements. The user view, usage and the expectable performance of the cluster remain unchanged.

You can track any disruptions or security advisories that may occur due to the aforementioned change in the Email category on our status reporting portal.

Dear users of the RWTH compute cluster,

on 2023-11-27 the complete cluster will not be available from 8am to 12am due to system maintenance.

Kind regards,
Your HPC team

You can track any disruptions or security advisories that may occur due to the aforementioned change in the RWTH-HPC category on our status reporting portal.

Due to the high load on the login / dialog nodes affecting their usability, we decided to reduce the maximum usable cores on each login node to four cores for each user. Please note:  These login nodes should be used for programming, preparation and minimal post processing of batch jobs. They are not intended for production runs or performance tests. For longer tests  (max. 25 minutes), parallel debugging, compiling, etc., you can use our “devel” partition by adding “#SBATCH –partition=devel” to batch jobs or interactively with “salloc -p devel”.

For all productive jobs, please use our batch system **without**  “#SBATCH –partition=devel”. If you want to more learn more about the batch system, we invite you to our Slurm introduction.

You can track any disruptions or security advisories that may occur due to the aforementioned change in the RWTH-HPC category on our status reporting portal.

The FastX server component installed on the HPC frontend nodes was upgraded to version 3.3.39.
The update contains security enhancements and several bugfixes from which all users benefit when using FastX.

Please ensure to use the latest desktop client if you are using FastX when accessing the cluster.

For more information on how to access the RWTH Aachen Compute cluster via FastX, please refer to the ITC Help Page

You can track any disruptions or security advisories that may occur due to the aforementioned change in the Email category on our [status reporting](https://maintenance.rz.rwth-aachen.de/ticket/status/messages/14-rechner-cluster) portal.

The Linux Kernel on the CLAIX18 compute nodes is being upgraded to kernel version 4.18.0-477.21.1. To maximise the availability of the compute cluster, the mandatory reboot of the nodes is scheduled as a reboot job, thus allowing all already submitted and running jobs for completion before the upgrade takes place.

Please note that the reboot is prioritised over other jobs, and some nodes may be temporarily unavailable after the reboot.

Best regards,
Your HPC-Team@RWTH

You can track any disruptions or security advisories that may occur due to the aforementioned change in the Email category on our status reporting portal.

As the result of a recent security evaluation, we have decided to disable several methods in key exchange, message authentication codes and encryption ciphers classified insecure/weak which obsoletes the following methods and method groups as listed below. In general, we have disabled SHA-1-based methods since SHA-1 is broken since early 2017 (cf. Stevens et al.: “The first collision for Full SHA-1”).

We kindly ask you to update your client configuration accordingly since these methods cannot be used anymore to access the RWTH Aachen HPC Cluster until further notice: Read the rest of this entry »

We have reduced the per-user-resource limits for main memory on the HPC dialog systems (login18-1.hpc.itc.rwth-aachen.de etc.). A single user can now only use about 25% of the available main memory, i.e. 96GB for most of our servers. On login18-x-1 and login18-x-2, as before, only 16 GB are available to each user.