Fusing imaging and metabolic modeling via multimodal deep learning in ovarian cancer

Integrating genotype (e.g., transcriptomics), phenotype (e.g., imaging), and tumor microenvironment (e.g., metabolomics) is crucial to elucidating the molecular basis of ovarian cancer. However, there is a lack of robust multimodal integration methods when only a limited number of common samples is available. Here, we generate patient-specific metabolic models starting from transcriptomics data and integrate them with imaging data. We show that this multimodal integration—never attempted before—improves survival estimation and enables a mechanistic interpretation of the predictions. We assess the robustness of our approach with different combinations of transcriptomics, fluxomics, and 3D computerized tomography (CT) imaging data, correctly stratifying patients based on risk. Fusing metabolic modeling with imaging and transcriptomics significantly improves model accuracy compared with widely used transcriptomics-imaging approaches and elucidates critical metabolic reactions. Our approach is general and can be applied to other cancer types where coupled imaging-transcriptomics data are available. A record of this paper's transparent peer review process is included in the supplemental information.