Towards Automated Interference Removal in Infrared Spectroscopy for Chemical Measurements: An Approach using Variational Empirical Bayes

by Dr Francois Kamper (Swiss Data Science Center)

Wednesday 18 Jun 2025, 11:30 → 13:00 Europe/Zurich

OHSA/B17

Abstract:

Extracting chemical measurements from infrared (IR) spectra is challenging, as they are often interfered with in a systematic manner so that a large contribution to the spectra do not contain any relevant chemical information.  Effectively removing this interference can significantly improve downstream tasks such as multivariate curve resolution, peak detection, and other quantitative analysis. While various interference removal techniques exist, their performance depends on hyperparameter tuning, which is difficult to perform in the absence of reliable ground truth. To address this, we propose to describe the process by which spectra are generated in a probabilistic manner, so that hyper-parameters can automatically calibrated via empirical Bayes (EB). To aid in the formulation of such a model, we consider the scenario where a few examples of the interference we can expect, or “measured blanks”, are available to form a prior, while the contribution of the chemical analytes to the spectra are modelled by the Gibbs distribution associated with a chosen asymmetric loss function. We demonstrate the benefits of our approach on laboratory-generated IR spectra of analytes collected on polytetrafluoroethylene (PTFE) filters and show that the pinball loss is preferable to the asymmetrically weighted squared error loss. We conclude with a discussion of limitations and open challenges.

Teams