Sep 19 – 22, 2022
Paul Scherer Institute, Villigen, Switzerland
Europe/Zurich timezone

The Blissdata Project: new perspectives on BLISS Data Management

Sep 20, 2022, 6:06 PM
2m
WHGA/Auditorium and online (Paul Scherer Institute, Villigen, Switzerland)

WHGA/Auditorium and online

Paul Scherer Institute, Villigen, Switzerland

Paul Scherrer Institute Forschungsstrasse 111 CH-5232-Villigen-PSI
Poster NOBUGS 2022

Speaker

Lucas Felix (ESRF)

Description

One of the main features of BLISS, the ESRF beamline control system in production since the ESRF-EBS upgrade, is to decouple data acquisition from online data storage. Redis, an online database, is used as a temporary buffer to store all the data produced by an acquisition or a reference to the data in the case of large data. This allows clients to consume data without perturbing the acquisition, and alleviates real-time constraints when clients want to access data for display (for example in the case of Flint, the BLISS data online visualization tool) or for saving in a HDF5 file (for example in the case of the BLISS Nexus Writer). In addition to display and saving, our goal is to enable online data analysis, with clients accessing the data buffer to produce intermediate results or to process data. The way data is structured within Redis is specific to BLISS and clients need to use the BLISS API. This means data analysis programs must import the whole BLISS library including beamline control parts and dependencies e.g. gevent. In order to provide a lightweight easy to use API library for online data analysis, the BLISS data management code is being refactored into a separate library called "blissdata". Clients with access to Redis can read data on the fly using an h5py-like API. The h5py compatible API will allow data processing code to access data in the online buffer as a file to process online and offline data in the same way.

Email address of presenting author lucas.felix@esrf.fr

Primary authors

Presentation materials

There are no materials yet.