Speaker
Description
Objective: Generative models can create synthetic patient cohorts for radiotherapy research, but current multimodal generators primarily sample plausible patients without explicit control over the generated anatomy (Li, 2026). This limits their use for systematic investigation of radiotherapy scenarios. We investigate a conditioned multimodal head-and-neck model that generates anatomically consistent CT, T1-weighted MRI, T2-weighted MRI, and PET images from user-defined anatomical and tumor contours.
Methods: The proposed framework combines complementary generative priors through a teacher–student approach. A pretrained multimodal generator supplies a prior for realistic cross-modality appearance and correlations (Li, 2026), while Medical AI for Synthetic Imaging (MAISI) provides anatomical conditioning through segmentation maps (Pengfei, et al., 2025). MAISI-v2 addresses condition fidelity using rectified flow and region-specific contrastive learning (Zhao, et al., 2026). The student learns from both teachers to preserve multimodal consistency while enabling control over anatomy, tumor location, and anatomical configuration.
Results: Initial experiments successful reconstructed relevant three-dimensional anatomy using the MAISI encoder–decoder. Preliminary segmentation-conditioned CT generations shows that user-defined anatomical masks influence the generated image content, providing initial evidence for controllable image synthesis.
Outlook & Discussion: The principal challenge is to transfer controllability without sacrificing multimodal realism. Training is complicated by the limited availability of consistently acquired multimodal datasets and institutional domain shifts caused by differences in scanners, protocols, immobilization, registration, patient populations, and clinical workflows. The proposed teacher–student framework may reduce dependence on fully paired, densely annotated multimodal training data by learning directly from model priors. Ultimately, we will investigate whether controlled generation can produce sufficiently realistic and internally consistent digital patients for systematic radiotherapy research.