Categories
Pages
-

IT Center Events

Schlagwort: ‘HPC’

Newer Entries »

Training: Process Mining and Scientific Workflows Running on the HPC cluster

September 19th, 2022 | by

Name: Training: Process Mining and Scientific Workflows running on the HPC cluster

Event type: Workshop

Date: October 18, 2022

Time:  9am – 1pm

Format: Online

Target Audience: HPC users

Contact Person: Zahra Sadeghibogar

*More information* 

 

Desciption  

The goal of process mining is to turn event data into insights and actions. On the other side, there exist scientific workflows running on HPC clusters.

Now, how can we combine process mining and scientific workflows running on HPC clusters? The first idea is „Process mining on logs of execution of scientific workflows on HPC“. Process mining on previous executions of a workflow on an HPC system can be used to deduce certain parallel execution parameters such as the number of tasks, number of cores, dedicated memory, etc. So, here we can provide valuable insights and offer optimization ideas for running the tasks on the HPC cluster.

Currently, based on the analysis of the extracted event log, there is limited usage of SLURM as the workflow management system, which means that only a small fraction of accounts declare interdependencies between tasks. So here comes the second idea, which is implementing a workflow engine that runs workflow steps (jobs) on SLURM with correct interdependencies. The third idea could be enabling efficient and distributed execution of process mining operators on HPC clusters. That is to allow users to run process mining workflows on an HPC cluster.

 

Agenda 

8:45 – 9:00
Welcome

9:00 – 9:30
Talk 1: Introduction to Process Mining

9:30 – 10:00
Talk 2: Introduction to HPC challenges to process mining

10:00 – 10:30
Talk 3: Introduction to scientific workflows and workflow management systems

10:30 – 11:00
Break

11:00 – 11:30
Talk 3: The first idea: Process mining on logs of execution of scientific workflows on HPC

11:30 – 12:00
Talk 4: The second idea: Building a workflow engine to run workflow steps on the HPC cluster with correct interdependencies

12:00 – 12:30
Discussion

12:30 – 12:45
Conclusion

Kategorie:
Comments Closed

Training: Processing and Analyzing Micrographs with Artificial Intelligence

September 19th, 2022 | by

Name: Training: Processing and Analyzing Micrographs with Artificial Intelligence

Event type: Workshop

Date: October 2 to 7, 2022

Format: Hybrid

Target Audience: HPC users

Contact Person: Setareh Medghalchi

*More information* 

 

Desciption  

High performance materials like steels typically possess a heterogeneous microstructure. Owing to this fact, their properties exceed those of the individual components but collection of image data requires observation and analysis of relatively large areas to capture the heterogeneity reliably. High resolution scanning electron microscopy serves as a tool to unravel many of the physical mechanisms of deformation from the sub-micron to the millimeter scale. On the other hand, collection, and analysis of high resolution image data from large areas requires laborious efforts and considerable amount of time, which is why it is not yet performed routinely.

However, new image analysis-based tools in conjunction with the application of deep learning convolutional neural networks (CNN) allows us to handle these data collected from large areas.

In this workshop, we will go through several image analysis techniques using python libraries step by step within jupyter notebooks. In this way, we will introduce the participants to examples of statistical information about specific features of the real microstructures which can be obtained with these methods, including microstructural information like phase fraction and also insights into deformation mechanisms from damage site detection and classification.

Kategorie:
Comments Closed

Introduction to interactive HPC with JupyterHub at the RWTH

August 11th, 2022 | by

Abstract

The new HPC JupyterHub service at the RWTH Aachen Univerity allows all eligible users of the RWTH Compute Cluster to utilize the existing compute hardware interactively with the use of Jupyter Notebooks. This HPC JupyterHub provides customization of profiles with a variety of programming kernels, software packages, and hardware definitions. This workshop will introduce the HPC JupyterHub service and offer an interactive demo. There will also be a discussion section to explore new use cases.

Agenda

  • Introduction to JupyterHub @ RWTH (20 minutes)
  • Interactive Demo (40 minutes)
  • Q & A Session  (30 minutes)

Format

Online via Zooom.

Target Audience

HPC, Simulation Software and Machine Learning users.

Requirements

HPC and VPN account at the RWTH for the interactive demo.

Capacity

Unlimited for the presentation, and 96 for the interactive demo.

Contact Person

Alvaro Frank a.frank@itc.rwth-aachen.de

Registration

RWTH Single Sign-On (SSO)

Registration for RWTH externals

Registration closing date: September 18th, 2022

Kategorie:
No Comments »

PPCES 2021

February 4th, 2021 | by

About PPCES

This one week online event will continue the tradition of previous annual week-long events that take place in Aachen every spring since 2001. We will cover the basics of parallel programming using OpenMP and MPI in Fortran and C/C++ and a first step towards performance tuning as well as current topics in AI/machine learning. Hands-on exercises for each topic will be included.

The contents of the courses are generally applicable but will be specialized towards CLAIX the compute cluster which is the current system installed at the RWTH’s IT Center. It might be helpful to read through the information which was provided during the HPC introduction on March 12 this year. This is especially true if you want to actively use CLAIX after this event.

OpenMP is a widely used approach for programming shared memory architectures, supported by most compilers nowadays. We will cover the basics of the programming paradigm as well as some advanced topics such as programming NUMA machines. The nodes of the RWTH Compute Cluster contain an increasing number of cores and thus we consider shared memory programming a vital alternative for applications that cannot be easily parallelized with MPI. We also expect a growing number of application codes to combine MPI and OpenMP for clusters of nodes with a growing number of cores.

The Message Passing Interface (MPI) is the de-facto standard for programming large HPC systems. We will introduce the basic concepts and give an overview of some advanced features. Also covered is hybrid parallelization, i.e., the combination of MPI and shared memory programming, which is gaining popularity as the number of cores per cluster node grows.

Machine Learning: We provide an overview to end-to-end deep learning with the latest version of Tensorflow/Keras. It covers the basic concepts to define models with Keras  and data pipelines with Tensorflow’s “Dataset”, and to visualize the results with Tensorboard while training. If training on one node or GPU is not enough,  we show how to scale up/out distributed training onto multiple compute nodes  and GPUs with Horovod. Furthermore, we provide an introduction to scikit-learn, with an overview of  different machine learning algorithms it provides and how to utilize it on GPUs  with H2O4GPU. The training courses consist of a hands-on exercises to be run directly on  RWTH infrastructure.

Guest Speakers

We are very happy to present two guest speakers:

Agenda

Day 1+2: OpenMP 

Monday, March 22

Monday, March 22
Day 1: OpenMP Part I
09:00 – 10:30 OpenMP Basics part 1 Christian Terboven
11:00 – 12:00 OpenMP Basics part 2 (incl. Lab) Christian Terboven
14:00 – 15:30 OpenMP Basics part 3 (incl. Lab) Christian Terboven
16:00 – 17:00 OpenMP Basics part 4 (incl. Lab) Christian Terboven

Tuesday, March 23

Day 2: OpenMP Part II
09:00 – 10:30 Getting OpenMP up to speed Ruud van der Pas
11:00 – 12:00 OpenMP SIMD Tim Cramer
14:00 – 15:30 OpenMP Advanced Tasking (incl. Lab) Christian Terboven
16:00 – 17:00 OpenMP for Accelerators Christian Terboven

Day 3+4: MPI

Wednesday March 24

Day 3: MPI Part I
09:00 – 10:30 Introduction to MPI Marc-Andre Hermanns
11:00 – 12:00 Blocking Point-to-Point Communication I Marc-Andre Hermanns
14:00 – 15:30 Blocking Point-to-Point Communication II Marc-Andre Hermanns
16:00 – 17:00 Non-blocking Point-to-Point Communication Marc-Andre Hermanns

Thursday March 25

MPI Part II
09:00 – 10:30 Blocking Collective Communication Marc-Andre Hermanns
11:00 – 12:00 Communicator Basics Marc-Andre Hermanns
14:00 – 15:30 Hybrid Programming Marc-Andre Hermanns
16:00 – 17:00 Outlook on Advanced Topics & Wrap-Up Marc-Andre Hermanns

Day 5: Machine Learning 

Seminar times will be 9:00-12:00 and 13:00-15:00.

This event is partially supported by The [Czech] Ministry of Education, Youth and Sports from the Large Infrastructures for Research, Experimental Development and Innovations project  “e-Infrastruktura CZ – LM2018140”

Friday March 26

Day 5: Machine Learning
09:00 – 09:45 Introduction to scikit-learn Georg Zitzlsberger
09:45 – 10:00 Getting Started on the Cluster Jannis Klinkenberg
10:00 – 10:30 Hands-on scikit-learn examples Georg Zitzlsberger
11:00 – 12:00 Introduction to Deep Neural Networks Georg Zitzlsberger
13:00 – 14:00

Tensorflow/Keras Exercises (short intro + Hands-on exercise)

  • Define Data Pipeline with Dataset
  • Build a Model
  • Train & Visualize with Tensorboard
 Georg Zitzlsberger
14:00 – 14:45

 Multi-GPU with Horovod (incl. short Hands-on)

 Georg Zitzlsberger
14:45 – 15:00 Q&A Georg Zitzlsberger

Prerequisites

Attendees of part I and II should be comfortable with C/C++ or Fortran programming in a Linux environment and interested in learning more about the technical details of application tuning and parallelization.
Participants of part III – machine learning –  will need some basic knowledge of Python.

All presentations will be given in English.

This event will be an online presentation.
All all parts of the tutorials will be accompanied by exercises.

Participants who have access to the RWTH identity management can use their own HPC account.
Those members of RWTH who do not yet have such an account can provide an HPC account here (https://sso.rwth-aachen.de/idp/profile/SAML2/Redirect/SSO?execution=e1s1) using the selfservice (Choose: Accounts und Kennwörter – Account anlegen – Hochleistungsrechnen)

External participants must provide themselves a Linux environment that contains an OpenMP compiler, a MPI library, or respectively a singularity environment.
For parts I and II  a Linux virtual machine will be sufficient.

For example on Ubuntu 20.04 LTS  the following commands can be used to install the necessary software für OpenMP and MPI:

# g++ is  available by default
sudo apt install gfortran # install  Fortran Compiler - if necessary
sudo apt-get install libomp-dev # install OpenMP libraries
sudo apt install mpich # install MPI library

Simple program examples can be compiled and executed by

g++ -fopenmp openmp-hello-world.cc;   ./a.out
gfortran -fopenmp openmp-hello-world.f90;  ./a.out
mpicc mpi_hello_world.c -o  ./a.out;  mpirun -np 2  ./a.out

For part III (ML) participants need to run singularity containers with access to one or more NVIDIA GPUs.

Course Material of PPCES 2021

OpenMP

Presentations

Exercises

 

MPI

Presentations

Exercises

 

Machine Learning

Presentations

Exercises

Further Information

The OpenMP part is also available as online tutorial (including videos): https://hpc-wiki.info/hpc/OpenMP_in_Small_Bites

Kategorie:
No Comments »

Newer Entries »