- Diese Veranstaltung hat bereits stattgefunden.
PPCES 2021
Montag, 22. März 2021 - Freitag, 26. März 2021
About PPCES
This one week online event will continue the tradition of previous annual week-long events that take place in Aachen every spring since 2001. We will cover the basics of parallel programming using OpenMP and MPI in Fortran and C/C++ and a first step towards performance tuning as well as current topics in AI/machine learning. Hands-on exercises for each topic will be included.
The contents of the courses are generally applicable but will be specialized towards CLAIX the compute cluster which is the current system installed at the RWTH’s IT Center. It might be helpful to read through the information which was provided during the HPC introduction on March 12 this year. This is especially true if you want to actively use CLAIX after this event.
OpenMP is a widely used approach for programming shared memory architectures, supported by most compilers nowadays. We will cover the basics of the programming paradigm as well as some advanced topics such as programming NUMA machines. The nodes of the RWTH Compute Cluster contain an increasing number of cores and thus we consider shared memory programming a vital alternative for applications that cannot be easily parallelized with MPI. We also expect a growing number of application codes to combine MPI and OpenMP for clusters of nodes with a growing number of cores.
The Message Passing Interface (MPI) is the de-facto standard for programming large HPC systems. We will introduce the basic concepts and give an overview of some advanced features. Also covered is hybrid parallelization, i.e., the combination of MPI and shared memory programming, which is gaining popularity as the number of cores per cluster node grows.
Machine Learning: We provide an overview to end-to-end deep learning with the latest version of Tensorflow/Keras. It covers the basic concepts to define models with Keras and data pipelines with Tensorflow’s „Dataset“, and to visualize the results with Tensorboard while training. If training on one node or GPU is not enough, we show how to scale up/out distributed training onto multiple compute nodes and GPUs with Horovod. Furthermore, we provide an introduction to scikit-learn, with an overview of different machine learning algorithms it provides and how to utilize it on GPUs with H2O4GPU. The training courses consist of a hands-on exercises to be run directly on RWTH infrastructure.
Guest Speakers
We are very happy to present two guest speakers:
- Ruud van der Pas (Oracle) on OpenMP
- Georg Zitzlsberger (IT4Innovations National Supercomputing Center at VSB – Technical University of Ostrava) on Machine Learning
Agenda
Day 1+2: OpenMP
Monday, March 22
|
Monday, March 22 |
Day 1: OpenMP Part I
|
|
|---|---|---|
| 09:00 – 10:30 | OpenMP Basics part 1 | Christian Terboven |
| 11:00 – 12:00 | OpenMP Basics part 2 (incl. Lab) | Christian Terboven |
| 14:00 – 15:30 | OpenMP Basics part 3 (incl. Lab) | Christian Terboven |
| 16:00 – 17:00 | OpenMP Basics part 4 (incl. Lab) | Christian Terboven |
Tuesday, March 23 |
Day 2: OpenMP Part II |
|
|---|---|---|
| 09:00 – 10:30 | Getting OpenMP up to speed | Ruud van der Pas |
| 11:00 – 12:00 | OpenMP SIMD | Tim Cramer |
| 14:00 – 15:30 | OpenMP Advanced Tasking (incl. Lab) | Christian Terboven |
| 16:00 – 17:00 | OpenMP for Accelerators | Christian Terboven |
Day 3+4: MPI
Wednesday March 24 |
Day 3: MPI Part I |
|
|---|---|---|
| 09:00 – 10:30 | Introduction to MPI | Marc-Andre Hermanns |
| 11:00 – 12:00 | Blocking Point-to-Point Communication I | Marc-Andre Hermanns |
| 14:00 – 15:30 | Blocking Point-to-Point Communication II | Marc-Andre Hermanns |
| 16:00 – 17:00 | Non-blocking Point-to-Point Communication | Marc-Andre Hermanns |
Thursday March 25 |
MPI Part II |
|
|---|---|---|
| 09:00 – 10:30 | Blocking Collective Communication | Marc-Andre Hermanns |
| 11:00 – 12:00 | Communicator Basics | Marc-Andre Hermanns |
| 14:00 – 15:30 | Hybrid Programming | Marc-Andre Hermanns |
| 16:00 – 17:00 | Outlook on Advanced Topics & Wrap-Up | Marc-Andre Hermanns |
Day 5: Machine Learning
Seminar times will be 9:00-12:00 and 13:00-15:00.
This event is partially supported by The [Czech] Ministry of Education, Youth and Sports from the Large Infrastructures for Research, Experimental Development and Innovations project “e-Infrastruktura CZ – LM2018140”
Friday March 26 |
Day 5: Machine Learning |
|
|---|---|---|
| 09:00 – 09:45 | Introduction to scikit-learn | Georg Zitzlsberger |
| 09:45 – 10:00 | Getting Started on the Cluster | Jannis Klinkenberg |
| 10:00 – 10:30 | Hands-on scikit-learn examples | Georg Zitzlsberger |
| 11:00 – 12:00 | Introduction to Deep Neural Networks | Georg Zitzlsberger |
| 13:00 – 14:00 |
Tensorflow/Keras Exercises (short intro + Hands-on exercise)
|
Georg Zitzlsberger |
| 14:00 – 14:45 |
Multi-GPU with Horovod (incl. short Hands-on) |
Georg Zitzlsberger |
| 14:45 – 15:00 | Q&A | Georg Zitzlsberger |
Prerequisites
Attendees of part I and II should be comfortable with C/C++ or Fortran programming in a Linux environment and interested in learning more about the technical details of application tuning and parallelization.
Participants of part III – machine learning – will need some basic knowledge of Python.
All presentations will be given in English.
This event will be an online presentation.
All all parts of the tutorials will be accompanied by exercises.
Participants who have access to the RWTH identity management can use their own HPC account.
Those members of RWTH who do not yet have such an account can provide an HPC account here (https://sso.rwth-aachen.de/idp/profile/SAML2/Redirect/SSO?execution=e1s1) using the selfservice (Choose: Accounts und Kennwörter – Account anlegen – Hochleistungsrechnen)
External participants must provide themselves a Linux environment that contains an OpenMP compiler, a MPI library, or respectively a singularity environment.
For parts I and II a Linux virtual machine will be sufficient.
For example on Ubuntu 20.04 LTS the following commands can be used to install the necessary software für OpenMP and MPI:
# g++ is available by default sudo apt install gfortran # install Fortran Compiler - if necessary sudo apt-get install libomp-dev # install OpenMP libraries sudo apt install mpich # install MPI library
Simple program examples can be compiled and executed by
g++ -fopenmp openmp-hello-world.cc; ./a.out gfortran -fopenmp openmp-hello-world.f90; ./a.out mpicc mpi_hello_world.c -o ./a.out; mpirun -np 2 ./a.out
For part III (ML) participants need to run singularity containers with access to one or more NVIDIA GPUs.
Course Material of PPCES 2021
OpenMP
Presentations
- Organization_PPCES2021
- 00-openmp-CT-welcome
- 01-openmp-CT-overview
- 02-openmp-CT-parallel_region
- 03-openmp-CT-worksharing
- 04-openmp-CT-scoping
- 05-openmp-CT-compilers
- 06-openmp-CT-welcome
- 07-openmp-CT-tasking_motivation
- 08-openmp-CT-tasking_model
- 09-openmp-CT-taskloop
- 10-openmp-CT-tasking_cutoff
- 11-openmp-CT-tasking_dependencies
- 12-openmp-CT-welcome
- 13-openmp-CT-NUMA
- 14-openmp-CT-tasking_affinity
- 15-openmp-CT-hybrid
- 16-openmp-CT-SIMD
- 17-openmp-CT-offloading
Exercises
MPI
Presentations
- hpc.nrw-01-MPI-Overview
- hpc.nrw-02-MPI_Concepts
- hpc.nrw-03-Blocking_Point-to-Point_Communication
- hpc.nrw-04-Non-blocking-Point-to-Point_Communication
- hpc.nrw-05-Derived_Datatypes
- hpc.nrw-06-Blocking_Collective_Communication
- hpc.nrw-08-Hybrid_Programming
- hpc.nrw-09-Communicator_Handling
- hpc.nrw-XX-Advanced_Topics
Exercises
Machine Learning
Presentations
- 00_Agenda
- 01a_Introduction_to_scikit-learn
- 01b_scikit-learn_Optimizations
- 03_scikit-learn_Hands-On
- 04_Introduction_Deep_Neural_Networks
- 05_Tensorflow_Keras
- 06_Multi-GPU_Horovod
Exercises
Further Information
The OpenMP part is also available as online tutorial (including videos): https://hpc-wiki.info/hpc/OpenMP_in_Small_Bites
Sprache:
Follow Us
-
Events
