<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "https://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ali="http://www.niso.org/schemas/ali/1.0/" article-type="research-article" dtd-version="1.2" xml:lang="en">
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">abc</journal-id>
      <journal-title-group>
        <journal-title>Archives of Breast Cancer</journal-title>
      </journal-title-group>
      <issn publication-format="electronic">2383-0433</issn>
      <publisher>
        <publisher-name>Archives of Breast Cancer</publisher-name>
      </publisher>
    </journal-meta>
    <article-meta>
      <article-id pub-id-type="doi">10.32768/abc.2024113255-261</article-id>
      <article-id pub-id-type="manuscript">910</article-id>
      <article-categories>
        <subj-group subj-group-type="heading">
          <subject>Original Article</subject>
        </subj-group>
      </article-categories>
      <title-group>
        <article-title>The Value of Radiomic Features of Primary Breast Tumor in STIR Sequences in Predicting Axilla Metastasis</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author" corresp="yes">
          <name>
            <surname>Rona</surname>
            <given-names>Günay</given-names>
          </name>
          <xref ref-type="aff" rid="aff1">a</xref>
          <xref ref-type="corresp" rid="cor1">*</xref>
        </contrib>
        <contrib contrib-type="author">
          <name>
            <surname>Arifoğlu</surname>
            <given-names>Meral</given-names>
          </name>
          <xref ref-type="aff" rid="aff1">a</xref>
        </contrib>
        <contrib contrib-type="author">
          <name>
            <surname>Serel</surname>
            <given-names>Ahmet Tekin</given-names>
          </name>
          <xref ref-type="aff" rid="aff2">b</xref>
        </contrib>
        <contrib contrib-type="author">
          <name>
            <surname>Baysal</surname>
            <given-names>Tamer</given-names>
          </name>
          <xref ref-type="aff" rid="aff1">a</xref>
        </contrib>
      </contrib-group>
      <aff id="aff1">
        <label>a</label>
        <institution>Department of Radiology, Kartal Doktor Lütfi Kırdar Training and Research Hospital, University of Health Sciences</institution>, <city>Istanbul</city>, <country country="TR">Turkey</country>
      </aff>
      <aff id="aff2">
        <label>b</label>
        <institution>Department of Urology, Suleyman Demirel University School of Medicine</institution>, <city>Istanbul</city>, <country country="TR">Turkey</country>
      </aff>
      <author-notes>
        <corresp id="cor1">
          <label>*</label>Address for correspondence: Günay Rona, MD, Cevizli Neighbourhood, Semsi Denizer Road. E-5 Highway District, 34890 Kartal, İstanbul/Turkey Tel +902164583000 Email: <email>gunayrona@gmail.com</email>
        </corresp>
        <fn fn-type="coi-statement">
          <p>No conflicts of interest exist regarding the publication of the present study.</p>
        </fn>
      </author-notes>
      <pub-date date-type="pub" publication-format="print">
        <year>2024</year>
      </pub-date>
      <volume>11</volume>
      <issue>3</issue>
      <fpage>255</fpage>
      <lpage>261</lpage>
      <history>
        <date date-type="received" iso-8601-date="2024-03-09">
          <day>09</day>
          <month>03</month>
          <year>2024</year>
        </date>
        <date date-type="rev-recd" iso-8601-date="2024-04-20">
          <day>20</day>
          <month>04</month>
          <year>2024</year>
        </date>
        <date date-type="accepted" iso-8601-date="2024-05-02">
          <day>02</day>
          <month>05</month>
          <year>2024</year>
        </date>
      </history>
      <permissions>
        <copyright-statement>Copyright &#x00A9; 2024 Archives of Breast Cancer</copyright-statement>
        <copyright-year>2024</copyright-year>
        <copyright-holder>Archives of Breast Cancer</copyright-holder>
        <license license-type="open-access">
          <license-p>This is an open-access article distributed under the terms of the Creative Commons Attribution-Non-Commercial 4.0 International License, which permits copy and redistribution of the material in any medium or format or adapt, remix, transform, and build upon the material for any purpose, except for commercial purposes.</license-p>
          <ali:license_ref>https://creativecommons.org/licenses/by-nc/4.0/</ali:license_ref>
        </license>
      </permissions>
      <self-uri xlink:href="https://www.archbreastcancer.com/index.php/abc/article/view/910" content-type="pdf" xlink:title="PDF Full Text"/>
      <abstract>
        <sec id="abs-s1">
          <title>Background</title>
          <p id="Pabs1">Detection of axillary metastases in breast cancer is critical for determining treatment options and prognosis. The aim of this study was to investigate the value of radiomic features obtained from short tau inversion recovery (STIR) sequences in magnetic resonance imaging (MRI) of the primary tumor in predicting axillary lymph node metastasis (ALNM).</p>
        </sec>
        <sec id="abs-s2">
          <title>Methods</title>
          <p id="Pabs2">Lesions of 165 patients with a mean (SD) age of 51.1 (11) years (range, 28–82 years) with newly diagnosed invasive breast cancer who underwent breast MRI before treatment were manually segmented from STIR sequences in the 3D Slicer program in all sections. Machine learning (ML) analysis was performed using the extracted 851 features, Python 3.11, Pycaret library program. Datasets were randomly divided into training (132, 80%) and independent test (33, 20%) sets. The performances of ML algorithms were compared with the area under the curve (AUC), accuracy, recall, precision, and F1 scores.</p>
        </sec>
        <sec id="abs-s3">
          <title>Results</title>
          <p id="Pabs3">Accuracy and AUC in the training set were in the range of 57% to 86% and 0.50 to 0.95, respectively. The best model in the training set was the CatBoost classifier, with an AUC of 0.95 and an accuracy of 83.9%. After tuning, the CatBoost classifier achieved an AUC of 0.92, an accuracy of 84%, a recall of 92.9%, a precision of 82%, and an F1 score of 86.7% on the independent test set, respectively.</p>
        </sec>
        <sec id="abs-s4">
          <title>Conclusion</title>
          <p id="Pabs4">Radiomic features obtained from primary tumors on STIR sequences have the potential to predict ALNM in invasive breast cancer.</p>
        </sec>
      </abstract>
      <kwd-group>
        <title>Keywords</title>
        <kwd>breast cancer</kwd>
        <kwd>lymphatic metastasis</kwd>
        <kwd>magnetic resonance imaging</kwd>
        <kwd>radiomics</kwd>
        <kwd>machine learning</kwd>
      </kwd-group>
      <funding-group>
        <funding-statement>This research did not receive any funding grants from public, commercial, or nonprofit agencies.</funding-statement>
      </funding-group>
    </article-meta>
  </front>
  <body>
    <sec sec-type="intro" id="S1">
      <title>Introduction</title>
      <p id="P1">Axillary lymph node metastasis (ALNM) is one of the most important prognostic factors determining survival in breast cancer.<xref ref-type="bibr" rid="R1">1</xref> The status of the axilla determines the need for axillary lymph node dissection (ALND), axillary radiotherapy, and neoadjuvant or adjuvant chemotherapy.<xref ref-type="bibr" rid="R2">2</xref> Accurate determination of the axillary status before treatment is critical for defining individualized treatment options.<xref ref-type="bibr" rid="R3">3</xref> Age, tumor size, tumor quadrant, multifocality, histological grade, pathological type, receptor status, and molecular subtype are associated with ALNM.<xref ref-type="bibr" rid="R4">4</xref>–<xref ref-type="bibr" rid="R9">9</xref></p>
      <p id="P2">Radiomic analysis aims to contribute to diagnosis, treatment, and follow-up by extracting specific quantitative information about diseases that the human eye cannot perceive from medical images. With radiomic analysis, the aim is to maximize the information obtained from images by obtaining quantitative data about signal intensities and the spatial distribution of interpixel relationships.<xref ref-type="bibr" rid="R10">10</xref></p>
      <p id="P3">Recently, radiomics has attracted considerable attention in the medical field, especially in oncology. Successful results have been achieved in the diagnosis, treatment, and classification of breast cancer.<xref ref-type="bibr" rid="R11">11</xref> Also, promising results were acquired in the differentiation of malignant and benign breast masses, in the estimation of the grade, receptor status, and subtypes of malignant tumors using radiomic features extracted from magnetic resonance imaging (MRI). This method is also promising in the prediction of neoadjuvant chemotherapy response in breast cancer. In studies performed with MRI, it was found that radiomics successfully helped even in the prediction of breast cancer recurrence .<xref ref-type="bibr" rid="R12">12</xref>–<xref ref-type="bibr" rid="R16">16</xref></p>
      <p id="P4">Radiomic features of axillary lymph nodes from T2-weighted (T2W) MR images were not successful in predicting ALNM.<xref ref-type="bibr" rid="R17">17</xref> However, ALNM could be predicted with radiomic features obtained from T2W, diffusion-weighted (DW), and contrast-enhanced T1-weighted (T1+C) images of the primary tumor.<xref ref-type="bibr" rid="R18">18</xref>–<xref ref-type="bibr" rid="R24">24</xref></p>
      <p id="P5">Our aim in this study was to investigate the performance of radiomic features obtained from short tau inversion recovery (STIR) sequences of the primary tumor in predicting axillary metastasis.</p>
    </sec>
    <sec sec-type="methods" id="S2">
      <title>Methods</title>
      <sec id="S2-1">
        <title>Participants</title>
        <p id="P6">Patients diagnosed with invasive breast cancer by core biopsy between August 2017 and February 2023 were evaluated retrospectively. Patients who underwent sentinel lymph node biopsy (SLNB) or ALND and who underwent MRI before treatment were included in the study. Patients who received neoadjuvant chemotherapy, had unknown pathology, had another malignancy, had recurrent disease, or had artifacts in MRI images were excluded from the study.</p>
      </sec>
      <sec id="S2-2">
        <title>MRI acquisition protocol</title>
        <p id="P7">MRI examinations were performed with a 1.5-T MRI device (Philips Ingenia, Philips Healthcare, Best, The Netherlands) using a dedicated 16-channel phased-array breast coil. The following images were obtained: non-fat-saturated turbo-spin-echo T1-weighted (field of view [FOV], 302 × 302 mm; matrix, 199 × 203; flip angle [FA], 90°; repetition time [TR], 547 ms; echo time [TE], 8 ms; slice thickness, 3.0 mm; slice gap, 3.3 mm), spin-echo STIR (FOV, 341 × 341 mm; matrix, 263 × 223; FA, 90°; TR, 4040 ms; TE, 65/175.000 ms; slice thickness, 3.0 mm; slice gap, 3.3 mm), and 3-dimensional fat-saturated ultrafast spoiled gradient-echo dynamic (FOV, 342 × 342 mm; matrix, 342 × 340; FA, 10°; TR, 5 ms; TE, 3 ms; slice thickness, 2.0 mm; slice gap, 1.0 mm).</p>
        <p id="P8">Dynamic sequences were acquired at 90, 142, 194, 246, and 298 seconds after contrast injection. A single dose of 0.1 mmol/kg of body weight gadolinium chelate was administered to the patients with an automatic injector.</p>
      </sec>
      <sec id="S2-3">
        <title>Segmentation and feature extraction</title>
        <p id="P9">STIR sequences in DICOM format were transferred to 3D Slicer software (version 4.10.2; <ext-link ext-link-type="uri" xlink:href="https://www.slicer.org">3D Slicer</ext-link>). Images were resampled to a size of 1 × 1 × 1 mm and normalized. Manual segmentation was performed independently by 2 radiologists with 8 and 10 years of experience in breast imaging, who were blinded to the axillary condition of the patients. Segmentation was performed on all axial STIR sequences with the tumor.</p>
        <p id="P10">A total of 851 texture features, including first-order, second-order, and wavelet-based features, were extracted with SlicerRadiomics (PyRadiomics v3.0.1) (<xref ref-type="fig" rid="F1">Figure 1</xref>). One month later, 30 randomly selected patients were independently segmented by the same 2 radiologists, and radiomic features were extracted again to evaluate interobserver agreement.</p>
        <fig id="F1">
          <label>Figure 1</label>
          <caption>
            <p>Workflow for Extraction of Radiomic Features from STIR Sequences and Machine Learning Analysis.</p>
          </caption>
          <graphic xlink:href="2383-0433-11-03-255-g001.tif">
            <alt-text>Figure 1</alt-text>
          </graphic>
        </fig>
      </sec>
      <sec id="S2-4">
        <title>Machine learning analysis</title>
        <p id="P11">Python (version 3.11) with the PyCaret library in a Jupyter Notebook environment was used for data processing and machine learning analysis. The synthetic minority oversampling technique (SMOTE) was used to avoid imbalanced datasets. Data normalization was performed before model development.</p>
        <p id="P12">The data sets were randomly divided into training and independent test sets. We used 10-fold cross-validation of the trained models to avoid data overfitting.</p>
        <p id="P13">Overall, 15 machine learning (ML) algorithms were used. The area under the receiver operating characteristic curve (AUC), accuracy, recall, precision, and F1 scores were compared with the performances of the ML algorithms. The best model for accuracy and AUC was selected and evaluated on the test set. The AUC, accuracy, recall, precision, and F1 scores were derived from the confusion matrix. The best model was tuned and finalized.</p>
      </sec>
      <sec id="S2-5">
        <title>Statistical analysis</title>
        <p id="P14">The data were analyzed by the Statistical Package for Social Sciences (SPSS) version 25.0 software (IBM Corp, Armonk, NY, USA). Percentage, mean, and standard deviation were used to present descriptive results.</p>
        <p id="P15">The 1-sample Kolmogorov-Smirnov test was used to assess if the groups had a normal distribution. Continuous variables with a normal distribution were reported as mean (SD). Interobserver agreement was evaluated using the intraclass correlation coefficient (ICC) values. Features with an ICC value greater than 0.70 were further analyzed with ML.</p>
      </sec>
    </sec>
    <sec sec-type="results" id="S3">
      <title>Results</title>
      <p id="P16">In this study, 421 patients were evaluated retrospectively. Overall, 106 patients who did not undergo MRI before treatment, 48 patients whose SLNB or ALND data were not available, 52 patients who received neoadjuvant chemotherapy, 6 patients with recurrent disease, 3 patients with concurrent malignancy, and 41 patients with artifacts were excluded from the study. Thus, 165 patients with a mean age of 51.12 ± 11 (range, 28–82) were included in the study. While 92 (55.76 %) patients had axillary metastases, 73 (44.24 %) did not have axillary metastases. The mean lesion size was 24.78 ± 15 (6–120) mm.</p>
      <p id="P17">Altogether, 667 features with ICC values above 0.7 were evaluated with ML. Wavelet filtered texture features, maximum 3D diameter, skewness, kurtosis, and maximum signal features showed a high correlation with ALNM. The features selected by the ML algorithms and their importance are presented in <xref ref-type="fig" rid="F2">Figure 2</xref>.</p>
      <fig id="F2">
        <label>Figure 2</label>
        <caption>
          <p>Features Selected by the Machine Learning Algorithms</p>
        </caption>
        <graphic xlink:href="2383-0433-11-03-255-g002.eps">
          <alt-text>Figure 2</alt-text>
        </graphic>
      </fig>
      <p id="P18">The accuracy and AUC of the ML algorithms on the training set ranged from 57% to 86% and 0.50 to 0.95, respectively (<xref ref-type="table" rid="T1">Table 1</xref>). Among the ML algorithms, the best model was the CatBoost classifier (AUC, 0.95; accuracy, 84%). The receiver operating characteristic curve showing the success of the CatBoost classifier in predicting ALNM is presented in <xref ref-type="fig" rid="F3">Figure 3</xref>.</p>
      <fig id="F3">
        <label>Figure 3</label>
        <caption>
          <p>Receiver Operating Characteristics Curve of the CatBoost Classifier in Predicting Axillary Lymph Node Metastasis</p>
        </caption>
        <graphic xlink:href="2383-0433-11-03-255-g003.tif">
          <alt-text>Figure 3</alt-text>
        </graphic>
      </fig>
      <table-wrap id="T1" position="float">
        <label>Table 1</label>
        <caption>
          <p>Performance of Machine Learning Models in Differentiating Axillary Metastases. Evaluation was performed on the training set of patients with invasive breast cancer using features from STIR sequences.</p>
        </caption>
        <table>
          <thead>
            <tr>
              <th>Model</th>
              <th>Accuracy</th>
              <th>AUC</th>
              <th>Recall</th>
              <th>Precision</th>
              <th>F1 Score</th>
            </tr>
          </thead>
          <tbody>
            <tr>
              <td>CatBoost Classifier</td>
              <td>0.8391</td>
              <td>0.9518</td>
              <td>0.9288</td>
              <td>0.8197</td>
              <td>0.8675</td>
            </tr>
            <tr>
              <td>Extra Trees Classifier</td>
              <td>0.8391</td>
              <td>0.9517</td>
              <td>0.8981</td>
              <td>0.8326</td>
              <td>0.8618</td>
            </tr>
            <tr>
              <td>Light Gradient Boosting Machine</td>
              <td>0.8696</td>
              <td>0.9510</td>
              <td>0.9212</td>
              <td>0.8632</td>
              <td>0.8870</td>
            </tr>
            <tr>
              <td>Gradient Boosting Classifier</td>
              <td>0.8652</td>
              <td>0.9480</td>
              <td>0.9058</td>
              <td>0.8687</td>
              <td>0.8834</td>
            </tr>
            <tr>
              <td>Random Forest Classifier</td>
              <td>0.8522</td>
              <td>0.9461</td>
              <td>0.9058</td>
              <td>0.8447</td>
              <td>0.8720</td>
            </tr>
            <tr>
              <td>Extreme Gradient Boosting</td>
              <td>0.8652</td>
              <td>0.9379</td>
              <td>0.9141</td>
              <td>0.8626</td>
              <td>0.8842</td>
            </tr>
            <tr>
              <td>Ada Boost Classifier</td>
              <td>0.8261</td>
              <td>0.8644</td>
              <td>0.8679</td>
              <td>0.8406</td>
              <td>0.8480</td>
            </tr>
            <tr>
              <td>Logistic Regression</td>
              <td>0.8174</td>
              <td>0.8479</td>
              <td>0.8372</td>
              <td>0.8478</td>
              <td>0.8299</td>
            </tr>
            <tr>
              <td>Quadratic Discriminant Analysis</td>
              <td>0.8261</td>
              <td>0.8427</td>
              <td>0.7045</td>
              <td>0.9798</td>
              <td>0.8091</td>
            </tr>
            <tr>
              <td>Decision Tree Classifier</td>
              <td>0.8348</td>
              <td>0.8338</td>
              <td>0.8449</td>
              <td>0.8654</td>
              <td>0.8479</td>
            </tr>
            <tr>
              <td>Linear Discriminant Analysis</td>
              <td>0.7696</td>
              <td>0.7738</td>
              <td>0.7660</td>
              <td>0.8083</td>
              <td>0.7820</td>
            </tr>
            <tr>
              <td>K Neighbors Classifier</td>
              <td>0.7435</td>
              <td>0.7348</td>
              <td>0.8263</td>
              <td>0.7443</td>
              <td>0.7815</td>
            </tr>
            <tr>
              <td>Naive Bayes</td>
              <td>0.5739</td>
              <td>0.5393</td>
              <td>0.8641</td>
              <td>0.5767</td>
              <td>0.6897</td>
            </tr>
            <tr>
              <td>Dummy Classifier</td>
              <td>0.5565</td>
              <td>0.5000</td>
              <td>1.0000</td>
              <td>0.5565</td>
              <td>0.7149</td>
            </tr>
            <tr>
              <td>SVM - Linear Kernel</td>
              <td>0.7696</td>
              <td>N/A<xref ref-type="table-fn" rid="tfn1">a</xref></td>
              <td>0.7814</td>
              <td>0.8094</td>
              <td>0.7856</td>
            </tr>
            <tr>
              <td>Ridge Classifier</td>
              <td>0.7957</td>
              <td>N/A<xref ref-type="table-fn" rid="tfn1">a</xref></td>
              <td>0.8212</td>
              <td>0.8274</td>
              <td>0.8116</td>
            </tr>
          </tbody>
        </table>
        <table-wrap-foot>
          <fn id="tfn1">
            <label>a</label>
            <p>For the Linear SVM and Ridge Classifier, the AUC could not be calculated reliably because the models did not provide valid probability estimates. Therefore, the AUC values are reported as N/A</p>
          </fn>
          <fn>
            <p>AUC, area under the receiver operating characteristic curve; SVM, support vector machine.</p>
          </fn>
        </table-wrap-foot>
      </table-wrap>
      <p id="P19">The CatBoost classifier was evaluated on the test set. After tuning, the AUC, accuracy, recall, precision, and F1 score were 0.92, 84%, 89%, 85%, and 86%, respectively. The CatBoost classifier model had a sensitivity of 92.9% and a specificity of 86.4% in detecting ALNM. The classification report and confusion matrix showing the performance of the CatBoost model are presented in <xref ref-type="fig" rid="F4">Figure 4</xref>.</p>
      <fig id="F4">
        <label>Figure 4</label>
        <caption>
          <p>Classification Report (A) and Confusion Matrix (B) for CatBoost Classifier in Predicting Axillary Lymph Node Metastasis</p>
        </caption>
        <graphic xlink:href="2383-0433-11-03-255-g004.eps">
          <alt-text>Figure 4</alt-text>
        </graphic>
      </fig>
    </sec>
    <sec sec-type="discussion" id="S4">
      <title>Discussion</title>
      <p id="P20">Recently, less invasive treatment approaches for the axilla have been accepted. SLNB is preferred instead of ALND in early-stage breast cancer. Determining the condition of the axilla in the preoperative period is important in the development of surgical plans. For individualized and minimally invasive treatment options, it is important to determine the condition of the axilla before treatment.<xref ref-type="bibr" rid="R25">25</xref> Among imaging methods, ultrasound (US) is the primary tool in the evaluation of the axilla. However, approximately 15% to 20% of patients with negative US findings have metastases in the SLNB. In mammography, 50% of level 1 axillary lymph nodes can be visualized, and levels 2 and 3 cannot be evaluated.<xref ref-type="bibr" rid="R26">26</xref> It has been reported that MRI has similar sensitivity to US but less specificity in detecting nodal metastases.<xref ref-type="bibr" rid="R27">27</xref></p>
      <p id="P21">In previous studies, ALNM could be predicted by the radiomic features of the primary tumor in MRI. Yu et al. successfully predicted axillary lymph node status on T1+C, T2W, and DW MRI with a multiomic signature including radiomic features and clinicopathologic features obtained from lymph nodes and the primary tumor.<xref ref-type="bibr" rid="R19">19</xref> Qiu et al. were successful in predicting ALNM using clinicopathological features, morphological features of lymph nodes from MR images, and radiomic features of the primary mass. Radiomic features obtained from DWI, T2W, and T1+C images of the primary mass were successful in predicting ALNM with an AUC of 0.806.<xref ref-type="bibr" rid="R20">20</xref> Wang et al. were able to predict axillary metastases with an AUC of more than 0.80 with the radiomic and deep learning features obtained from dynamic contrast-enhanced (DCE) MRI.<xref ref-type="bibr" rid="R21">21</xref> Chen et al. were able to predict ALNM with a nomogram created from the radiomic and clinicopathological features of the primary tumor obtained from DW and DCE MRI (AUC in the training and test sets, 0.80 and 0.71, respectively).<xref ref-type="bibr" rid="R22">22</xref> Using DCE MRI, Liu et al. predicted ALNM with a tumoral and peritumoral radiomic signature (AUCs in the training and test sets were 0.872 and 0.863, respectively).<xref ref-type="bibr" rid="R23">23</xref> Cui et al. predicted axillary lymph node status with radiomic features obtained from second postcontrast images on MRI with an AUC of 0.86 and an accuracy of 89%.<xref ref-type="bibr" rid="R24">24</xref> However, none of these studies presented the performance of T2W images alone in predicting ALNM.</p>
      <p id="P22">Dong et al. were successful in predicting sentinel lymph node metastasis with radiomic features obtained from fat-suppressed T2W and DW MRI images. With T2W alone, AUCs were 0.847 in the training set and 0.770 in the validation set. With DW MRI, they obtained AUCs of 0.847 in the training set and 0.787 in the validation set. When they combined the features obtained from T2W and DW images, the AUCs were 0.863 in the training set and 0.805 in the validation set.<xref ref-type="bibr" rid="R18">18</xref></p>
      <p id="P23">In our study, we predicted ALNM with an AUC of 0.92 and 84% accuracy in the test set with the radiomic features obtained from STIR sequences by modeling using machine learning analysis. The ML models used, except for Naive Bayes, Dummy Classifier, SVM - Linear Kernel, and Ridge Classifier, showed successful performance with an AUC of at least 0.73 and 74% accuracy. The CatBoost Classifier, Extra Trees Classifier, Light Gradient Boosting Machine, Gradient Boosting Classifier, and Extreme Gradient Boosting had AUCs above 0.93 and accuracy values above 86%. Among the models, the CatBoost Classifier, Extra Trees Classifier, Light Gradient Boosting, Gradient Boosting Classifier, and Random Forest Classifier had the highest performance, with AUCs of 0.95 and accuracy values of 84% to 86%. Among these models, the CatBoost Classifier had the highest recall and precision values, at 93% and 82%, respectively.</p>
      <p id="P24">Radiomic features derived from T2W sequences detected ALNM with an AUC of up to 0.85. When DWI and DCE were combined with T2W, the AUC increased to 0.86.<xref ref-type="bibr" rid="R18">18</xref> In our study, only STIR sequences reached an AUC of 0.92. Since STIR sequences are used in routine breast MRI in some centers, they may contribute to the prediction of ALNM. To our knowledge, STIR radiomics has not been used in studies investigating ALNM. STIR sequences stand out compared with other fat suppression techniques by providing more uniform fat suppression without being affected by magnetic field inhomogeneity. Although the signal-to-noise ratio is poor, it is useful because it includes both T1 and T2 contrast.</p>
      <p id="P25">There is a correlation between tumor size and lymph node metastasis.<xref ref-type="bibr" rid="R9">9</xref> In our study, the maximum diameter of the tumor and axillary metastases showed a high correlation. Kurtosis and skewness features, which evaluate intralesional homogeneity, were also correlated with ALNM.</p>
      <p id="P26">Our study has some limitations. Its retrospective nature, small number of patients, and choosing the largest lesion in patients with more than one lesion are among the limitations of our study. Another limitation is that we did not divide the patients according to the number of metastatic lymph nodes.</p>
    </sec>
    <sec sec-type="conclusions" id="S5">
      <title>Conclusion</title>
      <p id="P27">In conclusion, radiomic features obtained from the primary tumor on STIR sequences have the potential to predict ALNM. STIR sequences are noninvasive and are currently used as a routine component of breast MRI in some centers. However, some lesions cannot be segmented because they have a signal intensity close to that of the parenchyma in STIR sequences.</p>
    </sec>
    <sec id="S6">
      <title>Ethical considerations</title>
      <p id="P28">Ethics committee approval was obtained from our institution for this retrospective study (Approval No. 202351425036).</p>
    </sec>
  </body>
  <back>
    <ack>
      <title>Acknowledgment</title>
      <p id="P29">None.</p>
    </ack>
    <ref-list>
      <title>References</title>
      <ref id="R1">
        <label>1</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Beenken</surname>
              <given-names>SW</given-names>
            </name>
            <name>
              <surname>Urist</surname>
              <given-names>MM</given-names>
            </name>
            <name>
              <surname>Zhang</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Desmond</surname>
              <given-names>R</given-names>
            </name>
            <name>
              <surname>Krontiras</surname>
              <given-names>H</given-names>
            </name>
            <name>
              <surname>Medina</surname>
              <given-names>H</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Axillary lymph node status, but not tumor size, predicts locoregional recurrence and overall survival after mastectomy for breast cancer</article-title>. <source>Ann Surg</source>
          <year>2003</year>;<volume>237</volume>:<fpage>732</fpage>-<lpage>739</lpage>. doi: <pub-id pub-id-type="doi">10.1097/01.SLA.0000065289.06765.71</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R2">
        <label>2</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Rao</surname>
              <given-names>R</given-names>
            </name>
            <name>
              <surname>Euhus</surname>
              <given-names>D</given-names>
            </name>
            <name>
              <surname>Mayo</surname>
              <given-names>HG</given-names>
            </name>
            <name>
              <surname>Balch</surname>
              <given-names>C</given-names>
            </name>
          </person-group>
          <article-title>Axillary node interventions in breast cancer: a systematic review</article-title>. <source>JAMA</source>
          <year>2013</year>;<volume>310</volume>:<fpage>1385</fpage>-<lpage>1394</lpage>.doi: <pub-id pub-id-type="doi">10.1001/jama.2013.277804</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R3">
        <label>3</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Diessner</surname>
              <given-names>J</given-names>
            </name>
            <name>
              <surname>Anders</surname>
              <given-names>L</given-names>
            </name>
            <name>
              <surname>Herbert</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Kiesel</surname>
              <given-names>M</given-names>
            </name>
            <name>
              <surname>Bley</surname>
              <given-names>T</given-names>
            </name>
            <name>
              <surname>Schlaiss</surname>
              <given-names>T</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Evaluation of different imaging modalities for axillary lymph node staging in breast cancer patients to provide a personalized and optimized therapy algorithm</article-title>. <source>J Cancer Res Clin Oncol</source>. <year>2023</year> Jul;<volume>149</volume>(<issue>7</issue>):<fpage>3457</fpage>-<lpage>3467</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s00432-022-04221-9</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R4">
        <label>4</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Chua</surname>
              <given-names>B</given-names>
            </name>
            <name>
              <surname>Ung</surname>
              <given-names>O</given-names>
            </name>
            <name>
              <surname>Taylor</surname>
              <given-names>R</given-names>
            </name>
            <name>
              <surname>Boyages</surname>
              <given-names>J</given-names>
            </name>
          </person-group>
          <article-title>Frequency and predictors of axillary lymph node metastases in invasive breast cancer</article-title>. <source>ANZ J Surg</source>. <year>2001</year> Dec;<volume>71</volume>(<issue>12</issue>):<fpage>723</fpage>-<lpage>728</lpage>. doi: <pub-id pub-id-type="doi">10.1046/j.1445-1433.2001.02266.x</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R5">
        <label>5</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Van Zee</surname>
              <given-names>KJ</given-names>
            </name>
            <name>
              <surname>Manasseh</surname>
              <given-names>DM</given-names>
            </name>
            <name>
              <surname>Bevilacqua</surname>
              <given-names>JL</given-names>
            </name>
            <name>
              <surname>Boolbol</surname>
              <given-names>SK</given-names>
            </name>
            <name>
              <surname>Fey</surname>
              <given-names>JV</given-names>
            </name>
            <name>
              <surname>Tan</surname>
              <given-names>LK</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>A nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy</article-title>. <source>Ann Surg Oncol</source>. <year>2003</year> Dec;<volume>10</volume>(<issue>10</issue>):<fpage>1140</fpage>-<lpage>1151</lpage>. doi: <pub-id pub-id-type="doi">10.1245/aso.2003.03.015</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R6">
        <label>6</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Zhang</surname>
              <given-names>J</given-names>
            </name>
            <name>
              <surname>Li</surname>
              <given-names>X</given-names>
            </name>
            <name>
              <surname>Huang</surname>
              <given-names>R</given-names>
            </name>
            <name>
              <surname>Feng</surname>
              <given-names>WL</given-names>
            </name>
            <name>
              <surname>Kong</surname>
              <given-names>YN</given-names>
            </name>
            <name>
              <surname>Xu</surname>
              <given-names>F</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>A nomogram to predict the probability of axillary lymph node metastasis in female patients with breast cancer in China: A nationwide, multicenter, 10-year epidemiological study</article-title>. <source>Oncotarget</source>
          <year>2017</year>;<volume>8</volume>:<fpage>35311</fpage>-<lpage>35325</lpage>. doi: <pub-id pub-id-type="doi">10.18632/oncotarget.13330</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R7">
        <label>7</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Cutuli</surname>
              <given-names>B</given-names>
            </name>
            <name>
              <surname>Velten</surname>
              <given-names>M</given-names>
            </name>
            <name>
              <surname>Martin</surname>
              <given-names>C</given-names>
            </name>
          </person-group>
          <article-title>Assessment of axillary lymph node involvement in small breast cancer: analysis of 893 cases</article-title>. <source>Clin Breast Cancer</source>
          <year>2001</year>;<volume>2</volume>:<fpage>59</fpage>-<lpage>66</lpage>. doi: <pub-id pub-id-type="doi">10.3816/CBC.2001.n.012</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R8">
        <label>8</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Andea</surname>
              <given-names>AA</given-names>
            </name>
            <name>
              <surname>Bouwman</surname>
              <given-names>D</given-names>
            </name>
            <name>
              <surname>Wallis</surname>
              <given-names>T</given-names>
            </name>
            <name>
              <surname>Visscher</surname>
              <given-names>DW</given-names>
            </name>
          </person-group>
          <article-title>Correlation of tumor volume and surface area with lymph node status in patients with multifocal/multicentric breast carcinoma</article-title>. <source>Cancer</source>
          <year>2004</year>;<volume>100</volume>:<fpage>20</fpage>-<lpage>27</lpage>. doi: <pub-id pub-id-type="doi">10.1002/cncr.11880</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R9">
        <label>9</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Chen</surname>
              <given-names>W</given-names>
            </name>
            <name>
              <surname>Wang</surname>
              <given-names>C</given-names>
            </name>
            <name>
              <surname>Fu</surname>
              <given-names>F</given-names>
            </name>
            <name>
              <surname>Yang</surname>
              <given-names>B</given-names>
            </name>
            <name>
              <surname>Chen</surname>
              <given-names>C</given-names>
            </name>
            <name>
              <surname>Sun</surname>
              <given-names>Y</given-names>
            </name>
          </person-group>
          <article-title>A Model to Predict the Risk of Lymph Node Metastasis in Breast Cancer Based on Clinicopathological Characteristics</article-title>. <source>Cancer Manag Res</source>
          <year>2020</year>;<volume>12</volume>:<fpage>10439</fpage>-<lpage>10447</lpage>. doi: <pub-id pub-id-type="doi">10.2147/CMAR.S272420</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R10">
        <label>10</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Van Timmeren</surname>
              <given-names>JE</given-names>
            </name>
            <name>
              <surname>Cester</surname>
              <given-names>D</given-names>
            </name>
            <name>
              <surname>Tanadini-Lang</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Alkadhi</surname>
              <given-names>H</given-names>
            </name>
            <name>
              <surname>Baessler</surname>
              <given-names>B</given-names>
            </name>
          </person-group>
          <article-title>Radiomics in medical imaging-"how-to" guide and critical reflection</article-title>. <source>Insights Imaging</source>
          <year>2020</year>;<volume>11</volume>:<fpage>91</fpage>. doi: <pub-id pub-id-type="doi">10.1186/s13244-020-00887-2</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R11">
        <label>11</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Conti</surname>
              <given-names>A</given-names>
            </name>
            <name>
              <surname>Duggento</surname>
              <given-names>A</given-names>
            </name>
            <name>
              <surname>Indovina</surname>
              <given-names>I</given-names>
            </name>
            <name>
              <surname>Guerrisi</surname>
              <given-names>M</given-names>
            </name>
            <name>
              <surname>Toschi</surname>
              <given-names>N</given-names>
            </name>
          </person-group>
          <article-title>Radiomics in breast cancer classification and prediction</article-title>. <source>Semin Cancer Biol</source>
          <year>2021</year>;<volume>72</volume>:<fpage>238</fpage>-<lpage>250</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.semcancer.2020.04.002</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R12">
        <label>12</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Parekh</surname>
              <given-names>VS</given-names>
            </name>
            <name>
              <surname>Jacobs</surname>
              <given-names>MA</given-names>
            </name>
          </person-group>
          <article-title>Integrated radiomic framework for breast cancer and tumor biology using advanced machine learning and multiparametric</article-title>. <source>MRI NPJ Breast Cancer</source>
          <year>2017</year>;<volume>3</volume>:<fpage>43</fpage>. doi: <pub-id pub-id-type="doi">10.1038/s41523-017-0045-3</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R13">
        <label>13</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Whitney</surname>
              <given-names>HM</given-names>
            </name>
            <name>
              <surname>Taylor</surname>
              <given-names>NS</given-names>
            </name>
            <name>
              <surname>Drukker</surname>
              <given-names>K</given-names>
            </name>
            <name>
              <surname>Edwards</surname>
              <given-names>AV</given-names>
            </name>
            <name>
              <surname>Papaioannou</surname>
              <given-names>J</given-names>
            </name>
            <name>
              <surname>Schacht</surname>
              <given-names>D</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Additive Benefit of Radiomics Over Size Alone in the Distinction Between Benign Lesions and Luminal A Cancers on a Large Clinical Breast MRI Dataset</article-title>. <source>Acad Radiol</source>. <year>2019</year>;<volume>26</volume>:<fpage>202</fpage>-<lpage>209</lpage>. doi: <pub-id pub-id-type="doi">10.1016/j.acra.2018.04.019</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R14">
        <label>14</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Bickelhaupt</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Paech</surname>
              <given-names>D</given-names>
            </name>
            <name>
              <surname>Kickingereder</surname>
              <given-names>P</given-names>
            </name>
            <name>
              <surname>Steudle</surname>
              <given-names>F</given-names>
            </name>
            <name>
              <surname>Lederer</surname>
              <given-names>W</given-names>
            </name>
            <name>
              <surname>Daniel</surname>
              <given-names>H</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Prediction of malignancy by a radiomic signature from contrast agent-free diffusion MRI in suspicious breast lesions found on screening mammography</article-title>. <source>J Magn Reson Imaging</source>
          <year>2017</year>;<volume>46</volume>:<fpage>604</fpage>-<lpage>616</lpage>. doi: <pub-id pub-id-type="doi">10.1002/jmri.25606</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R15">
        <label>15</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Bickelhaupt</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Jaeger</surname>
              <given-names>PF</given-names>
            </name>
            <name>
              <surname>Laun</surname>
              <given-names>FB</given-names>
            </name>
            <name>
              <surname>Lederer</surname>
              <given-names>W</given-names>
            </name>
            <name>
              <surname>Daniel</surname>
              <given-names>H</given-names>
            </name>
            <name>
              <surname>Kuder</surname>
              <given-names>TA</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Radiomics Based on Adapted Diffusion Kurtosis Imaging Helps to Clarify Most Mammographic Findings Suspicious for Cancer</article-title>. <source>Radiology</source>
          <year>2018</year>;<volume>287</volume>:<fpage>761</fpage>-<lpage>770</lpage>. doi: <pub-id pub-id-type="doi">10.1148/radiol.2017170273</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R16">
        <label>16</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Antropova</surname>
              <given-names>N</given-names>
            </name>
            <name>
              <surname>Huynh</surname>
              <given-names>BQ</given-names>
            </name>
            <name>
              <surname>Giger</surname>
              <given-names>ML</given-names>
            </name>
          </person-group>
          <article-title>A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets</article-title>. <source>Med Phys</source>
          <year>2017</year>;<volume>44</volume>:<fpage>5162</fpage>-<lpage>5171</lpage>. doi: <pub-id pub-id-type="doi">10.1002/mp.12453</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R17">
        <label>17</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Samiei</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Granzier</surname>
              <given-names>RWY</given-names>
            </name>
            <name>
              <surname>Ibrahim</surname>
              <given-names>A</given-names>
            </name>
            <name>
              <surname>Primakov</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Lobbes</surname>
              <given-names>MBI</given-names>
            </name>
            <name>
              <surname>Beets-Tan</surname>
              <given-names>RGH</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Dedicated Axillary MRI-Based Radiomics Analysis for the Prediction of Axillary Lymph Node Metastasis in Breast Cancer</article-title>. <source>Cancers (Basel)</source>
          <year>2021</year> 12;<volume>13</volume>:<fpage>757</fpage>. doi: <pub-id pub-id-type="doi">10.3390/cancers13040757</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R18">
        <label>18</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Dong</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Feng</surname>
              <given-names>Q</given-names>
            </name>
            <name>
              <surname>Yang</surname>
              <given-names>W</given-names>
            </name>
            <name>
              <surname>Lu</surname>
              <given-names>Z</given-names>
            </name>
            <name>
              <surname>Deng</surname>
              <given-names>C</given-names>
            </name>
            <name>
              <surname>Zhang</surname>
              <given-names>L</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Preoperative prediction of sentinel lymph node metastasis in breast cancer based on radiomics of T2-weighted fat-suppression and diffusion-weighted MRI</article-title>. <source>Eur Radiol</source>. <year>2018</year>;<volume>28</volume>:<fpage>582</fpage>-<lpage>591</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s00330-017-5005-7</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R19">
        <label>19</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Yu</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>He</surname>
              <given-names>Z</given-names>
            </name>
            <name>
              <surname>Ouyang</surname>
              <given-names>J</given-names>
            </name>
            <name>
              <surname>Tan</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Chen</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Gu</surname>
              <given-names>Y</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Magnetic resonance imaging radiomics predicts preoperative axillary lymph node metastasis to support surgical decisions and is associated with tumor microenvironment in invasive breast cancer: A machine learning, multicenter study</article-title>. <source>EBioMedicine</source>
          <year>2021</year>;<volume>69</volume>:<fpage>103460</fpage>. doi: <pub-id pub-id-type="doi">10.1016/j.ebiom.2021.103460</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R20">
        <label>20</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Qiu</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Zhang</surname>
              <given-names>X</given-names>
            </name>
            <name>
              <surname>Wu</surname>
              <given-names>Z</given-names>
            </name>
            <name>
              <surname>Wu</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Yang</surname>
              <given-names>Z</given-names>
            </name>
            <name>
              <surname>Wang</surname>
              <given-names>D</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>MRI-Based Radiomics Nomogram: Prediction of Axillary Non-Sentinel Lymph Node Metastasis in Patients With Sentinel Lymph Node-Positive Breast Cancer</article-title>. <source>Front Oncol</source>
          <year>2022</year>;<volume>12</volume>:<fpage>811347</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fonc.2022.811347</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R21">
        <label>21</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Wang</surname>
              <given-names>D</given-names>
            </name>
            <name>
              <surname>Hu</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Zhan</surname>
              <given-names>C</given-names>
            </name>
            <name>
              <surname>Zhang</surname>
              <given-names>Q</given-names>
            </name>
            <name>
              <surname>Wu</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Ai</surname>
              <given-names>T</given-names>
            </name>
          </person-group>
          <article-title>A nomogram based on radiomics signature and deep-learning signature for preoperative prediction of axillary lymph node metastasis in breast cancer</article-title>. <source>Front Oncol</source>
          <year>2022</year>;<volume>12</volume>:<fpage>940655</fpage>. doi: <pub-id pub-id-type="doi">10.3389/fonc.2022.940655</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R22">
        <label>22</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Chen</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Wang</surname>
              <given-names>L</given-names>
            </name>
            <name>
              <surname>Dong</surname>
              <given-names>X</given-names>
            </name>
            <name>
              <surname>Luo</surname>
              <given-names>R</given-names>
            </name>
            <name>
              <surname>Ge</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Liu</surname>
              <given-names>H</given-names>
            </name>
            <name>
              <surname>Zhang</surname>
              <given-names>Y</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Deep Learning Radiomics of Preoperative Breast MRI for Prediction of Axillary Lymph Node Metastasis in Breast Cancer</article-title>. <source>J Digit Imaging</source>
          <year>2023</year>:<fpage>1</fpage>-<lpage>9</lpage>. doi: <pub-id pub-id-type="doi">10.1007/s10278-023-00818-9</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R23">
        <label>23</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Liu</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Li</surname>
              <given-names>X</given-names>
            </name>
            <name>
              <surname>Zhu</surname>
              <given-names>L</given-names>
            </name>
            <name>
              <surname>Zhao</surname>
              <given-names>Z</given-names>
            </name>
            <name>
              <surname>Wang</surname>
              <given-names>T</given-names>
            </name>
            <name>
              <surname>Zhang</surname>
              <given-names>X</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Preoperative Prediction of Axillary Lymph Node Metastasis in Breast Cancer Based on Intratumoral and Peritumoral DCE-MRI Radiomics Nomogram</article-title>. <source>Contrast Media Mol Imaging</source>
          <year>2022</year> 18;<volume>2022</volume>:<fpage>6729473</fpage>. doi: <pub-id pub-id-type="doi">10.1155/2022/6729473</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R24">
        <label>24</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Cui</surname>
              <given-names>X</given-names>
            </name>
            <name>
              <surname>Wang</surname>
              <given-names>N</given-names>
            </name>
            <name>
              <surname>Zhao</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Chen</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Li</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Xu</surname>
              <given-names>M</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Preoperative Prediction of Axillary Lymph Node Metastasis in Breast Cancer using Radiomics Features of DCE-MRI</article-title>. <source>Sci Rep</source>
          <year>2019</year>;<volume>9</volume>(<issue>1</issue>):<fpage>2240</fpage>. doi: <pub-id pub-id-type="doi">10.1038/s41598-019-38502-0</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R25">
        <label>25</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Morrow</surname>
              <given-names>M</given-names>
            </name>
          </person-group>
          <article-title>Management of the Node-Positive Axilla in Breast Cancer in 2017: Selecting the Right Option</article-title>. <source>JAMA Oncol</source>
          <year>2018</year>;<volume>4</volume>:<fpage>250</fpage>-<lpage>251</lpage>. doi: <pub-id pub-id-type="doi">10.1001/jamaoncol.2017.3625</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R26">
        <label>26</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Chang</surname>
              <given-names>JM</given-names>
            </name>
            <name>
              <surname>Leung</surname>
              <given-names>JWT</given-names>
            </name>
            <name>
              <surname>Moy</surname>
              <given-names>L</given-names>
            </name>
            <name>
              <surname>Ha</surname>
              <given-names>SM</given-names>
            </name>
            <name>
              <surname>Moon</surname>
              <given-names>WK</given-names>
            </name>
          </person-group>
          <article-title>Axillary Nodal Evaluation in Breast Cancer: State of the Art</article-title>. <source>Radiology</source>
          <year>2020</year>;<volume>295</volume>:<fpage>500</fpage>-<lpage>515</lpage>. doi: <pub-id pub-id-type="doi">10.1148/radiol.2020192534</pub-id>.
        </mixed-citation>
      </ref>
      <ref id="R27">
        <label>27</label>
        <mixed-citation publication-type="journal">
          <person-group person-group-type="author">
            <name>
              <surname>Cooper</surname>
              <given-names>KL</given-names>
            </name>
            <name>
              <surname>Meng</surname>
              <given-names>Y</given-names>
            </name>
            <name>
              <surname>Harnan</surname>
              <given-names>S</given-names>
            </name>
            <name>
              <surname>Ward</surname>
              <given-names>SE</given-names>
            </name>
            <name>
              <surname>Fitzgerald</surname>
              <given-names>P</given-names>
            </name>
            <name>
              <surname>Papaioannou</surname>
              <given-names>D</given-names>
            </name>
            <etal/>
          </person-group>
          <article-title>Positron emission tomography (PET) and magnetic resonance imaging (MRI) for the assessment of axillary lymph node metastases in early breast cancer: systematic review and economic evaluation</article-title>. <source>Health Technol Assess</source>
          <year>2011</year>;<volume>15</volume>:<fpage>iii</fpage>-<lpage>134</lpage>. doi: <pub-id pub-id-type="doi">10.3310/hta15040</pub-id>.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>


