Access Abstraction to HPC Resources - Hybrid Event

Europe/Zurich
Akademie I (Empa)

Akademie I

Empa

Überlandstrasse 129 8600 Dübendorf
Carlo Pignedoli (Empa), Daniele Passerone (Empa), Edoardo Baldi (Empa), Michele De Lorenzi (ETH Zurich / CSCS)
Description

Introduction

For HPC centers it is essential to simplify the access to the computational resources for their user  communities. Personal computing did already introduce in the eighties of last century the concept of desktop computing to simplify the utilization by the users. At the same time we moved from mainframes to client-server architectures up to micro-services. Unfortunately supercomputers are still (mostly) used by manually submitting jobs in queues.

Some scientific communities did already work on such challenges and created web portals able to support complex workflow hiding from the end users details related to the HPC infrastructure. More and more scientists will be involved in the next few years in building such platforms.

For us it is getting more and more important to support those portals and offer new ways to use the computational and storage resources we manage. It shall be possible to use the supercomputers interactively or through defined programmatic interfaces (API) to cover from simple use cases up to complex workflows. This also includes access to data management tools to stage in (and out) everything that’s needed to run the simulation or analysis.

Key Questions

  • Scientific portals: What kind of scientific portals do exist today? What will be the future developments? What are their requirements respect to the HPC providers?

  • Interactive computing platforms: How can I easily run a calculation/analysis that requires some degree of interaction on a bigger and more powerful machine? Can I use an HPC resource as if it was my laptop?

  • Efficient management of data: Are there any optimal tools to manage data efficiently? E.g., avoiding copying non-essential info when moving data, or saving data for archiving purposes  Can I synchronize files between my computer, my home institution and my HPC resource without having to use too many different tools? Without having to monitor failures manually?

  • Computational resources abstraction: As a HPC provider, how can we build a “middle layer” allowing our scientific computing users to see the resources we are providing in a uniform way? (keywords: workflow, API, RESTful interfaces).

Registration
Registration Form
Participants
  • Alexander Kashev
  • Carlo Antonio Pignedoli
  • Christian Bolliger
  • Daniele Passerone
  • Edoardo Baldi
  • Guillermo Losilla
  • Hans-Rudolf Hotz
  • Kristjan Eimre
  • Martin Jacquot
  • Mei-Chih Chang
  • Michele De Lorenzi
  • Patrick Bleiziffer
  • Raluca Hodoroaba
  • Rene Windiks
  • Sofiane Sarni
  • Sébastien Moretti
  • Thomas Kramer
  • Thomas Wüst
    • 09:30 10:00
      Registration and Welcome Coffee 30m
    • 10:00 10:15
      Welcome and Introduction
      Conveners: Daniele Passerone (Empa), Michele De Lorenzi (CSCS)
    • 10:15 11:00
      Keynote Presentation
    • 11:00 11:30
      AiiDA Lab – an Ecosystem for Developing, Executing and Sharing Scientific Workflows 30m

      Computational simulations constitute an integral part of modern science. Running scientific software, however, requires expert knowledge not only in the modelled scientific phenomena but often also in accessing and operating compute resources, programming, installing and compiling software, and other topics related to computation. In the presentation I will introduce AiiDA lab, an open source platform that provides the means to develop, execute and share computational science workflows as an intuitive web service available to anyone who knows how to use a web browser. AiiDA lab is built on top of AiiDA, an infrastructure that provides the automatic workflow engine and provenance tracking, and Jupyter, which powers the graphical user interfaces.

      Speaker: Kristjan Eimre (Empa)
    • 11:30 12:00
      Real-Time Services for Large Volume Experiment-Data Analysis Utilizing Supercomputing and Cloud Technologies (SELVEDAS) 30m

      The ongoing developments in accelerators, detectors and experiment automation is leading to a rapid growth of data generated during experiments. A viable solution is utilizing suitable infrastructures that allow additional remote high performance capacity for processing and analysis of data from the experimental facilities with larger data volumes and higher processing needs.

      The SELVEDAS project proposes a hybrid cloud infrastructure, offering scalable and extensible services for data management and analysis to Swiss academic users by leveraging high performance computing (HPC), storage, networking as well as cloud technologies and orchestration. The on-demand services perform as a highly efficient remote data processing system providing fast feedback and analysis with the long time storage and archival of petabytes of data.

      Speaker: Mei-Chih Chang (PSI)
    • 12:00 12:30
      Community Development
    • 12:30 13:30
      Lunch and Networking 1h
    • 13:30 14:30
      Site Visit: Automated Driving Sensor Testing Vehicle

      Autonomous vehicles have the potential to positively influence future traffic behavior. Nevertheless, the new technology opens social, legal, economical and technical questions. A significant technical question concerns the necessary minimum technical requirements for self-driving cars to make them street legal. We focus in testing the autonomous driving sensors in real world driving under different weather, light and contamination conditions. Storage and computational resources are important for our research, since we are producing a lot of data and we need computational power in order to execute the perception algorithms for AD and to analyze the performance of them.

    • 13:30 14:30
      Site Visit: Virtual Acoustics

      Analogous to visualization, auralization allows creating virtual realities offering listening experiencing of non-existing situations. During this visit you will hear simulated environmental noise from cars, trains and aircraft in Empa's 3D audio laboratory AuraLab.

    • 14:30 15:00
      Community Development
    • 15:00 15:15
      Coffee Break 15m
    • 15:15 15:45
      Community Development
    • 15:45 16:15
      Community Development
    • 16:15 16:20
      Farewell and End of the Meeting