Speaker
Description
One of the main features of BLISS, the ESRF beamline control system in production since the ESRF-EBS upgrade, is to decouple data acquisition from online data storage. Redis, an online database, is used as a temporary buffer to store all the data produced by an acquisition or a reference to the data in the case of large data. This allows clients to consume data without perturbing the acquisition, and alleviates real-time constraints when clients want to access data for display (for example in the case of Flint, the BLISS data online visualization tool) or for saving in a HDF5 file (for example in the case of the BLISS Nexus Writer). In addition to display and saving, our goal is to enable online data analysis, with clients accessing the data buffer to produce intermediate results or to process data. The way data is structured within Redis is specific to BLISS and clients need to use the BLISS API. This means data analysis programs must import the whole BLISS library including beamline control parts and dependencies e.g. gevent. In order to provide a lightweight easy to use API library for online data analysis, the BLISS data management code is being refactored into a separate library called "blissdata". Clients with access to Redis can read data on the fly using an h5py-like API. The h5py compatible API will allow data processing code to access data in the online buffer as a file to process online and offline data in the same way.
| Email address of presenting author | lucas.felix@esrf.fr |
|---|
