HPC Configuration Management

Thursday 19 May 2022, 09:30 → 16:00 Europe/Zurich

Akademie I (Empa)

Carlo Pignedoli (Empa), Christian Bolliger (ETH Zurich), Daniele Passerone (Empa), Edoardo Baldi (Empa), Michele De Lorenzi (CSCS)

Introduction

The often untold story about HPC is configuration management. When it comes to administering even small HPC clusters, configuration management is key. Since this happens in the background, most users are not aware of all the efforts to keep a cluster up-to-date, all nodes in sync, and to react to users' demands for specific settings.

There are almost as many ways to fulfill this task, as there are HPC systems. 

Key Questions

In the next hpc-ch forum we will discuss questions like:

• which configuration management tools have been proved to be useful?
• which caveats can be found in managing HPC clusters?
• is it preferable to use preconfigured configurations or to define the configuration management from the scratch? Is there a middle ground?
• which role are configuration management databases (CMDB) playing?
• which new aspects in configuration management have been brought-in by containerization and container orchestration (e.g. Kubernetes)? 
• what are your experiences with configuration management systems? 

 

Welcome Coffee & Registration

1 Welcome and Introduction

Speakers: Daniele Passerone (Empa), Michele De Lorenzi (CSCS)

2 Using SaltStack to Manage a Cluster Centrally

SaltStack is a way to apply "states" to a bunch of servers. It is easy to customize the states based on a central configuration files (pillar) or nodes spec (grains). We coupled this software with kickstart and clush to be able to re install a node or server untenanted. This permits to re-install the whole cluster on a regular basis instead of updating it.

Speaker: Yann Sagon (University of Geneva)

unige-hpc-ch-salt.pdf

3 Experiences Deploying Slurm Clusters on OpenStack using MagicCastle

Magic Castle is project that aims to replicate Compute Canada’s user experience in public clouds. Here we present the four different configuration management tools used in the project and what they are used for.

Speaker: Victor Holanda Rusu (CSCS)

hpc_ch-magiccastle.pdf

4 A Look at Some Scalable HPC Configuration Management Tools

Going through the history of recent CSCS’ large supercomputers, we look at how the CM tools have evolved and how we work with them to get our systems to be configured efficiently at scale.

Speaker: Miguel Gila (CSCS)

20220519.HPC-CH.A Look at Some Scalable HPC Configuration Management Tools.Miguel.Gila.pdf

5 Configuration Management on a Secure OpenStack Environment

Managing secure scientific infrastructure is challenging. We present the constraints encountered during deployment of secure OpenStack environment and configuration management tools introduced to address them.

Speaker: Jani Heikkinen (University of Basel)

Config management at sciCOREMed-2.pdf

12:15 Lunch and Networking

6 Community Development

The session will be dedicated to:

• Members Update after Two years of Pandemic: Latest Developments in our Organization.
  Members are kindly invited to give a 10-min presentation on the current activity status.

• Selection of topics of interest, themes and location for future forums.

Speakers: Christian Bollliger (ETH Zurich), Michele De Lorenzi (CSCS)

Empa_HPC_hpc-ch-forum-2022.pdf

Patrik_Burkhalter_Empa_Ansible.pdf

7 Guided Tour - Automated Driving Sensor Testing Vehicle

Autonomous vehicles have the potential to positively influence future traffic behavior. Nevertheless, the new technology opens social, legal, economical and technical questions. A significant technical question concerns the necessary minimum technical requirements for self-driving cars to make them street legal. We focus in testing the autonomous driving sensors in real world driving under different weather, light and contamination conditions. Storage and computational resources are important for our research, since we are producing a lot of data and we need computational power in order to execute the perception algorithms for AD and to analyze the performance of them.

Speakers: Christian Hohl (Empa), Dejan Milojevic (Empa)

8 Farewell and End of the Meeting