Asynchronous Improvements in AiiDA

by Dr Ali Khosravi (PSI/LMS)

Wednesday Jan 29, 2025, 3:00 PM → 4:00 PM Europe/Zurich

OVGA/200

In today’s era of supercomputing, securing resources and computation results has become increasingly important. Historically, HPC (High-Performance Computing) servers have interfaced with users via SSH (Secure Shell) to provide services. However, due to security vulnerabilities and the high costs of maintaining login servers, many supercomputer centers are exploring innovative alternatives.

One promising alternative is the use of REST (Representational State Transfer) APIs. While REST APIs are more cost-effective and offer enhanced security, they often suffer from significantly lower performance compared to SSH method. This performance limitation presents a challenge for developers of large-scale, high-performance workflow mangers, such as AiiDA, to redesign their systems in ways that handles server communication delays efficiently. Potential solutions include multi-threading, OS-level parallel programming, and asynchronous programming.

In this short talk, we will outline the problem, discuss the performance challenges we faced, and explain how asynchronous programming emerged as the best solution for our use case. The majority of the talk will focus on the concept of asynchronous programming, providing a brief and logical introduction to its principles. This talk is designed for an audience with very basic programming knowledge and emphasizes conceptual understanding rather than detailed coding.