Artificial Intelligence-Driven Multimodal Integration of Genomic, Pathological, and Clinical Data in Breast Cancer: A Narrative Review of Clinical Applications and Translational Challenges Multimodal AI in Breast Cancer
Abstract
Background: Breast cancer remains the most frequently diagnosed malignancy in women worldwide, and its clinical heterogeneity imposes formidable challenges on treatment standardization and outcome prediction. The emergence of artificial intelligence (AI) and multimodal data integration strategies is fundamentally reshaping how genomic, histopathological, radiological, and electronic health record (EHR) data are synthesized into actionable clinical intelligence.
Methods: We conducted a narrative review of peer-reviewed literature published between 2015 and 2026, focusing on AI-driven approaches to genomic profiling, digital pathology, radiomics, natural language processing (NLP) of clinical text, and multimodal data fusion in the context of breast cancer diagnosis, prognosis, and treatment planning.
Results: AI-based frameworks have demonstrated superior performance compared with conventional single-modality approaches in subtype classification, treatment response prediction, and biomarker discovery. Deep learning architectures applied to whole-slide imaging, convolutional neural networks in radiomics, graph-based genomic models, and transformer-based NLP pipelines each contribute distinct and complementary dimensions to precision breast oncology. Multimodal fusion models integrating 2 or more data streams consistently outperform unimodal counterparts in prognostic accuracy.
Conclusions: Multimodal AI represents a convergent approach capable of operationalizing the biological complexity of breast cancer at the point of clinical care. Realizing this potential requires rigorous prospective validation, harmonized data standards, explainability frameworks, and equitable implementation strategies. Future models must be adaptive, continuously learning, and grounded in real-world clinical evidence.
References
Bray F, Laversanne M, Sung H, Ferlay J, Siegel RL, Soerjomataram I, et al. Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA A Cancer J Clinicians. 2024;74(3):229-63. doi: 10.3322/caac.21834
Perou CM, Sørlie T, Eisen MB, van de Rijn M, Jeffrey SS, Rees CA, et al. Molecular portraits of human breast tumours. Nature. 2000;406(6797):747-52. doi: 10.1038/35021093
Cardoso F, Kyriakides S, Ohno S, Penault-Llorca F, Poortmans P, Rubio IT, et al. Early breast cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Annals of Oncology. 2019;30(8):1194-220. doi: 10.1093/annonc/mdz173
Litton JK, Rugo HS, Ettl J, Hurvitz SA, Gonçalves A, Lee KH, et al. Talazoparib in Patients with Advanced Breast Cancer and a Germline BRCA Mutation. N Engl J Med. 2018;379(8):753-63. doi: 10.1056/nejmoa1802905
Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44-56. doi: 10.1038/s41591-018-0300-7
McKinney SM, Sieniek M, Godbole V, Godwin J, Antropova N, Ashrafian H, et al. International evaluation of an AI system for breast cancer screening. Nature. 2020;577(7788):89-94. doi: 10.1038/s41586-019-1799-6
Abu Abeelh E, Abuabeileh Z. Screening Mammography and Artificial Intelligence: A Comprehensive Systematic Review. Cureus. 2025. doi: 10.7759/cureus.79353
Boehm KM, Khosravi P, Vanguri R, Gao J, Shah SP. Harnessing multimodal data integration to advance precision oncology. Nat Rev Cancer. 2022;22(2):114-26. doi: 10.1038/s41568-021-00408-3
Acosta JN, Falcone GJ, Rajpurkar P, Topol EJ. Multimodal biomedical AI. Nat Med. 2022;28(9):1773-84. doi: 10.1038/s41591-022-01981-2
Cheerla A, Gevaert O. Deep learning with multimodal representation for pancancer prognosis prediction. Bioinformatics. 2019;35(14):i446-i454. doi: 10.1093/bioinformatics/btz342
Sparano JA, Gray RJ, Makower DF, Pritchard KI, Albain KS, Hayes DF, et al. Adjuvant Chemotherapy Guided by a 21-Gene Expression Assay in Breast Cancer. N Engl J Med. 2018;379(2):111-21. doi: 10.1056/nejmoa1804710
Rhee S, Seo S, Kim S. Hybrid Approach of Relation Network and Localized Graph Convolutional Filtering for Breast Cancer Subtype Classification. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. 2018;3527-34. doi: 10.24963/ijcai.2018/490
Szklarczyk D, Gable AL, Lyon D, Junge A, Wyder S, Huerta-Cepas J, et al. STRING v11: protein–protein association networks with increased coverage, supporting functional discovery in genome-wide experimental datasets. Nucleic Acids Res. 2019;47(D1):D607-D613. doi: 10.1093/nar/gky1131
Mackenzie CC, Dawood M, Graham S, Eastwood M, ul Amir Afsar Minhas F. Neural graph modelling of whole slide images for survival ranking. Proceedings of the First Learning on Graphs Conference [Internet]. PMLR; 2022 Oct 8;198:48:1-48:10. [cited 8 10 2026]. Available from: https://proceedings.mlr.press/v198/mackenzie22a.html
Wu SZ, Al-Eryani G, Roden DL, Junankar S, Harvey K, Andersson A, et al. A single-cell and spatially resolved atlas of human breast cancers. Nat Genet. 2021;53(9):1334-47. doi: 10.1038/s41588-021-00911-1
Ciriello G, Gatza ML, Beck AH, Wilkerson MD, Rhie SK, Pastore A, et al. Comprehensive Molecular Portraits of Invasive Lobular Breast Cancer. Cell. 2015;163(2):506-19. doi: 10.1016/j.cell.2015.09.033
Alexandrov LB, Kim J, Haradhvala NJ, Huang MN, Tian Ng AW, Wu Y, et al. The repertoire of mutational signatures in human cancer. Nature. 2020;578(7793):94-101. doi: 10.1038/s41586-020-1943-3
Pancotti C, Rollo C, Codicè F, Birolo G, Fariselli P, Sanavia T. MUSE-XAE: MUtational Signature Extraction with eXplainable AutoEncoder enhances tumour types classification. Bioinformatics. 2024;40(5):btae320. doi: 10.1093/bioinformatics/btae320
Telli ML, Timms KM, Reid J, Hennessy B, Mills GB, Jensen KC, et al. Homologous Recombination Deficiency (HRD) Score Predicts Response to Platinum-Containing Neoadjuvant Chemotherapy in Patients with Triple-Negative Breast Cancer. Clinical Cancer Research. 2016;22(15):3764-73. doi: 10.1158/1078-0432.ccr-15-2477
Cortes J, Cescon DW, Rugo HS, Nowecki Z, Im SA, Yusof MM, et al. Pembrolizumab plus chemotherapy versus placebo plus chemotherapy for previously untreated locally recurrent inoperable or metastatic triple-negative breast cancer (KEYNOTE-355): a randomised, placebo-controlled, double-blind, phase 3 clinical trial. The Lancet. 2020;396(10265):1817-28. doi: 10.1016/s0140-6736(20)32531-9
Relling MV, Evans WE. Pharmacogenomics in the clinic. Nature. 2015;526(7573):343-50. doi: 10.1038/nature15817
Bray J, Sludden J, Griffin MJ, Cole M, Verrill M, Jamieson D, et al. Influence of pharmacogenetics on response and toxicity in breast cancer patients treated with doxorubicin and cyclophosphamide. Br J Cancer. 2010;102(6):1003-9. doi: 10.1038/sj.bjc.6605587
Relling MV, Klein TE. CPIC: Clinical Pharmacogenetics Implementation Consortium of the Pharmacogenomics Research Network. Clin Pharmacol Ther. 2011;89(3):464-7. doi: 10.1038/clpt.2010.279
Whirl‐Carrillo M, Huddart R, Gong L, Sangkuhl K, Thorn CF, Whaley R, et al. An Evidence‐Based Framework for Evaluating Pharmacogenomics Knowledge for Personalized Medicine. Clin Pharmacol Ther. 2021;110(3):563-72. doi: 10.1002/cpt.2350
Helland T, Alsomairy S, Lin C, Søiland H, Mellgren G, Hertz DL. Generating a Precision Endoxifen Prediction Algorithm to Advance Personalized Tamoxifen Treatment in Patients with Breast Cancer. JPM. 2021;11(3):201. doi: 10.3390/jpm11030201
Hertz DL, Freedman RA. Pharmacogenomics in breast cancer. Pharmacogenomics. 2018;19(2):77-80. doi: 10.2217/pgs-2017-0167
Abdollahi J, Mehrpour O. Opportunities and challenges: artificial intelligence in pharmacology. 2024 11th International Symposium on Telecommunications (IST). Tehran, Iran: IEEE; 2024;28-37. doi: 10.1109/IST64061.2024.10843462
Kather JN, Heij LR, Grabsch HI, Loeffler C, Echle A, Muti HS, et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat Cancer. 2020;1(8):789-99. doi: 10.1038/s43018-020-0087-6
Lu MY, Williamson DFK, Chen TY, Chen RJ, Barbieri M, Mahmood F. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng. 2021;5(6):555-70. doi: 10.1038/s41551-020-00682-w
Schmauch B, Romagnoni A, Pronier E, Saillard C, Maillé P, Calderaro J, et al. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat Commun. 2020;11(1):3877. doi: 10.1038/s41467-020-17678-4
Salgado R, Denkert C, Demaria S, Sirtaine N, Klauschen F, Pruneri G, et al. The evaluation of tumor-infiltrating lymphocytes (TILs) in breast cancer: recommendations by an International TILs Working Group 2014. Annals of Oncology. 2015;26(2):259-71. doi: 10.1093/annonc/mdu450
Marx V. Method of the Year: spatially resolved transcriptomics. Nat Methods. 2021;18(1):9-14. doi: 10.1038/s41592-020-01033-y
Croizer H, Mhaidly R, Kieffer Y, Gentric G, Djerroudi L, Leclere R, et al. Deciphering the spatial landscape and plasticity of immunosuppressive fibroblasts in breast cancer. Nat Commun. 2024;15(1):2806. doi: 10.1038/s41467-024-47068-z
Chen RJ, Lu MY, Williamson DFK, Chen TY, Lipkova J, Noor Z, et al. Pan-cancer integrative histology-genomic analysis via multimodal deep learning. Cancer Cell. 2022;40(8):865-878.e6. doi: 10.1016/j.ccell.2022.07.004
Lambin P, Leijenaar RTH, Deist TM, Peerlings J, de Jong EEC, van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14(12):749-62. doi: 10.1038/nrclinonc.2017.141
Salim M, Wåhlin E, Dembrower K, Azavedo E, Foukakis T, Liu Y, et al. External Evaluation of 3 Commercial Artificial Intelligence Algorithms for Independent Assessment of Screening Mammograms. JAMA Oncol. 2020;6(10):1581-8. doi: 10.1001/jamaoncol.2020.3321
Dembrower K, Wåhlin E, Liu Y, Salim M, Smith K, Lindholm P, et al. Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer detection and radiologist workload: a retrospective simulation study. The Lancet Digital Health. 2020;2(9):e468-e474. doi: 10.1016/s2589-7500(20)30185-0
Li W, Newitt DC, Gibbs J, Wilmes LJ, Jones EF, Arasu VA, et al. Predicting breast cancer response to neoadjuvant treatment using multi-feature MRI: results from the I-SPY 2 TRIAL. npj Breast Cancer. 2020;6(1):63. doi: 10.1038/s41523-020-00203-7
Ha R, Mutasa S, Karcich J, Gupta N, Pascual Van Sant E, Nemer J, et al. Predicting Breast Cancer Molecular Subtype with MRI Dataset Utilizing Convolutional Neural Network Algorithm. J Digit Imaging. 2019;32(2):276-82. doi: 10.1007/s10278-019-00179-2
Akin M, Erdemli S, Dulgeroglu O, Taskin F, Tokat F. Potential role of AI decision support in reducing unnecessary ultrasound-guided breast biopsies. Egypt J Radiol Nucl Med. 2026;57(1):22. doi: 10.1186/s43055-026-01684-5
Weitz M, Pfeiffer JR, Patel S, Biancalana M, Pekis A, Kannan V, et al. Performance of an AI-powered visualization software platform for precision surgery in breast cancer patients. npj Breast Cancer. 2024;10(1):98. doi: 10.1038/s41523-024-00696-6
Chen KA, Kirchoff KE, Butler LR, Holloway AD, Kapadia MR, Kuzmiak CM, et al. Analysis of Specimen Mammography with Artificial Intelligence to Predict Margin Status. Ann Surg Oncol. 2023;30(12):7107-15. doi: 10.1245/s10434-023-14083-1
Shia WC, Kuo YH, Hsu FR, Lin J, Wu WP, Wu HK, et al. Evaluating the Margins of Breast Cancer Tumors by Using Digital Breast Tomosynthesis with Deep Learning: A Preliminary Assessment. Diagnostics. 2024;14(10):1032. doi: 10.3390/diagnostics14101032
Alsentzer E, Murphy J, Boag W, Weng WH, Jindi D, Naumann T, McDermott M. Publicly Available Clinical BERT Embeddings. Proceedings of the 2nd Clinical Natural Language Processing Workshop. 2019;72-8. doi: 10.18653/v1/W19-1909
Kehl KL, Elmarakeby H, Nishino M, Van Allen EM, Lepisto EM, Hassett MJ, et al. Assessment of Deep Natural Language Processing in Ascertaining Oncologic Outcomes From Radiology Reports. JAMA Oncol. 2019;5(10):1421-9. doi: 10.1001/jamaoncol.2019.1800
Di Maio M, Perrone F, Conte P. Real-world evidence in oncology: opportunities and limitations. Oncologist. 2020;25(5):e746-e752. doi: 10.1634/theoncologist.2019-0647
Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172-80. doi: 10.1038/s41586-023-06291-2
Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Küttler H, Lewis M, Yih WT, Rocktäschel T, Riedel S, Kiela D. Retrieval-augmented generation for knowledge-intensive NLP tasks. Adv Neural Inf Process Syst [Internet]. NeurIPS; 2020 Oct 8;33:9459-74. [cited 8 10 2026]. Available from: https://papers.neurips.cc/paper_files/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
Park H, Ok S, Kang T, Park M. Integrating Large Language Models with Deep Learning for Breast Cancer Treatment Decision Support. Diagnostics. 2026;16(3):394. doi: 10.3390/diagnostics16030394
Lipkova J, Chen RJ, Chen B, Lu MY, Barbieri M, Shao D, et al. Artificial intelligence for multimodal data integration in oncology. Cancer Cell. 2022;40(10):1095-110. doi: 10.1016/j.ccell.2022.09.012
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Process Syst [Internet]. NeurIPS; 2017 Oct 8;30:5998-6008. [cited 8 10 2026]. Available from: https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
Polikar R. Ensemble based systems in decision making. IEEE Circuits Syst. Mag. 2006;6(3):21-45. doi: 10.1109/mcas.2006.1688199
Akinlade O, Abdollahi J, Paul Etchie M, Ajagbe SA, Alabi OO, Ayo Adeyanju B. Ensuring effective cervical cancer diseases diagnosis system using ensemble machine learning model. Int J Artif Intell. 2026;15(3):2410-22. doi: 10.11591/ijai.v15.i3.pp2410-2422
Chen RJ, Lu MY, Weng WH, Chen TY, Williamson DFK, Manz T, Shady M, Mahmood F. Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 2021;4015-25. doi: 10.1109/ICCV48922.2021.00398
Steyaert S, Pizurica M, Nagaraj D, Khandelwal P, Hernandez-Boussard T, Gentles AJ, et al. Multimodal data fusion for cancer biomarker discovery with deep learning. Nat Mach Intell. 2023;5(4):351-62. doi: 10.1038/s42256-023-00633-5
Rieke N, Hancox J, Li W, Milletarì F, Roth HR, Albarqouni S, et al. The future of digital health with federated learning. npj Digit. Med. 2020;3(1):119. doi: 10.1038/s41746-020-00323-1
Ogier du Terrail J, Leopold A, Joly C, Béguier C, Andreux M, Maussion C, et al. Federated learning for predicting histological response to neoadjuvant chemotherapy in triple-negative breast cancer. Nat Med. 2023;29(1):135-46. doi: 10.1038/s41591-022-02155-w
US Food and Drug Administration. Artificial Intelligence-Enabled Medical Devices [Internet]. Silver Spring, MD: U.S. Food and Drug Administration; 2026 Mar 8. [cited 8 10 2026]. Available from: https://www.fda.gov/medical-devices/digital-health-center-excellence/artificial-intelligence-enabled-medical-devices
US Food and Drug Administration. Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles [Internet]. Silver Spring, MD: U.S. Food and Drug Administration; 2023 Oct 8. [cited 8 10 2026]. Available from: https://www.fda.gov/medical-devices/artificial-intelligence-enabled-medical-devices/predetermined-change-control-plans-machine-learning-enabled-medical-devices-guiding-principles
Abdollahi J, Nouri-Moghaddam B, Mohajer A, Mirzaei A, Derakhshanfard N. Bridging global semantic representation and inference efficiency via hybrid knowledge distillation for clinically deployable cardiac MRI classification. Intelligence-Based Medicine. 2026;14:100366. doi: 10.1016/j.ibmed.2026.100366
Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Adv Neural Inf Process Syst [Internet]. NeurIPS; 2017 Oct 8;30:4765-74. [cited 8 10 2026]. Available from: https://papers.nips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html
Klauschen F, Dippel J, Keyl P, Jurmeister M, Bockmayr M, Mock A, et al. Toward explainable artificial intelligence for precision pathology. Annu Rev Pathol. 2024;19:541-70. doi: 10.1146/annurev-pathmechdis-051222-113147
Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-53. doi: 10.1126/science.aax2342
Seyyed-Kalantari L, Zhang H, McDermott MBA, Chen IY, Ghassemi M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat Med. 2021;27(12):2176-82. doi: 10.1038/s41591-021-01595-0
Chen RJ, Ding T, Lu MY, Williamson DFK, Jaume G, Song AH, et al. Towards a general-purpose foundation model for computational pathology. Nat Med. 2024;30(3):850-62. doi: 10.1038/s41591-024-02857-3
Al Amri WS, Al Jabri M, Al Abri A, Hughes TA. Cancer Genetics in the Arab World. Technol Cancer Res Treat. 2025;24:15330338251336829. doi: 10.1177/15330338251336829
Al-Jumaan M, Chu H, Alsulaiman A, Camp SY, Han S, Gillani R, et al. Interplay of Mendelian and polygenic risk factors in Arab breast cancer patients. Genome Med. 2023;15(1):65. doi: 10.1186/s13073-023-01220-4
Wan JCM, Massie C, Garcia-Corbacho J, Mouliere F, Brenton JD, Caldas C, et al. Liquid biopsies come of age: towards implementation of circulating tumour DNA. Nat Rev Cancer. 2017;17(4):223-38. doi: 10.1038/nrc.2017.7
Coombes RC, Page K, Salari R, Hastings RK, Armstrong A, Ahmed S, et al. Personalized Detection of Circulating Tumor DNA Antedates Breast Cancer Metastatic Recurrence. Clinical Cancer Research. 2019;25(14):4255-63. doi: 10.1158/1078-0432.ccr-18-3663
Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17(1):195. doi: 10.1186/s12916-019-1426-2
Bates DW, Auerbach A, Schulam P, Wright A, Saria S. Reporting and Implementing Interventions Involving Machine Learning and Artificial Intelligence. Ann Intern Med. 2020;172(11 Suppl):S137-S144. doi: 10.7326/M19-0872
Howard FM, Dolezal J, Kochanny S, Schulte J, Chen H, Heij L, et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nat Commun. 2021;12(1):4423. doi: 10.1038/s41467-021-24698-1
Hsu W, Hippe DS, Nakhaei N, Wang PC, Zhu B, Siu N, et al. External Validation of an Ensemble Model for Automated Mammography Interpretation by Artificial Intelligence. JAMA Netw Open. 2022;5(11):e2242343. doi: 10.1001/jamanetworkopen.2022.42343
Authors
Copyright (c) 2026 Jamshad Taslimi, Alireza Minagar
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright©. This is an open-access article distributed under the terms of the Creative Commons Attribution-Non-Commercial 4.0 International License, which permits copy and redistribution of the material in any medium or format or adapt, remix, transform, and build upon the material for any purpose, except for commercial purposes.