Speaker
Description
Objective
Radiotherapy services routinely generate machine and dosimetric QA data, rarely analysed together. We propose a reproducible five-stage workflow for developing machine-learning models from such data, focusing on the step least often addressed: establishing whether exploitable signal exists, and distinguishing a genuine but weak physical association from useful predictive capability. We demonstrate it linking multileaf-collimator (MLC) technical controls with pre-treatment dosimetric QA across three linacs (727 temporally matched pairs, 2022–2025).
Methods
Five stages: (1) identify underused routine data; (2) integrate it preserving temporal structure; (3) model with physically interpretable features, a linear baseline and leakage-free chronological validation; (4) audit the signal rather than the error alone — explainability consistent with physics, unsupervised analysis complementing supervised models, apparent detectors tested against chance; (5) delimit validity and deployment conditions.
Weekly MLC controls (AQUA, Elekta) were matched by linac to QA records (Verisoft, PTW). Ridge and XGBoost regressors predicted the gamma passing rate at 3%/2 mm. Audits used SHAP, per-linac principal component analysis, and a robust Mahalanobis/Hotelling T² control chart benchmarked against frequency-matched random alarms (permutation test).
Results
Gamma passing rates concentrated in 92–100%, leaving little exploitable variation. Ridge yielded a mean absolute error of 1.3 percentage points (pp) but R² ≈ 0.03; XGBoost performed worse than predicting the mean (R² = −0.37).
The audits were more informative. SHAP importance was physically coherent; per-linac principal components correlated with gamma error (ρ = 0.24 and 0.20; p < 0.01). The control chart exposed the central difficulty. Flagged controls carried greater gamma error (1.4 versus 1.0 pp; p < 0.001), and Mahalanobis distance correlated with gamma error (ρ = 0.17; p < 0.001): the association was real. However, as a detector of individual episodes it did not outperform chance: depending on the lead-time window (14–60 days) it identified 5–8 of 11, matched by random alarms (p = 0.74).
The framework nevertheless correctly characterised a real jaw-mirror malfunction (gamma = 90%), jaw variables dominating both SHAP and the chart.
Outlook & Discussion
The main lesson is a second-order negative result: a statistically significant but weak association can masquerade as predictive capability. In a well-maintained service, events are rare and the signal too weak for reliable individual-episode prediction.
This yields a practical acceptance criterion: no QA prediction tool should be deployed until it outperforms an appropriate chance benchmark. Further: integrating the data required more effort than modelling them, and the workflow's demonstrated value lies in supporting diagnosis under medical-physicist supervision, not in autonomous prediction.