skip to main content


This content will become publicly available on December 1, 2024

Title: 3DGAUnet: 3D Generative Adversarial Networks with a 3D U-Net Based Generator to Achieve the Accurate and Effective Synthesis of Clinical Tumor Image Data for Pancreatic Cancer

Pancreatic ductal adenocarcinoma (PDAC) presents a critical global health challenge, and early detection is crucial for improving the 5-year survival rate. Recent medical imaging and computational algorithm advances offer potential solutions for early diagnosis. Deep learning, particularly in the form of convolutional neural networks (CNNs), has demonstrated success in medical image analysis tasks, including classification and segmentation. However, the limited availability of clinical data for training purposes continues to represent a significant obstacle. Data augmentation, generative adversarial networks (GANs), and cross-validation are potential techniques to address this limitation and improve model performance, but effective solutions are still rare for 3D PDAC, where the contrast is especially poor, owing to the high heterogeneity in both tumor and background tissues. In this study, we developed a new GAN-based model, named 3DGAUnet, for generating realistic 3D CT images of PDAC tumors and pancreatic tissue, which can generate the inter-slice connection data that the existing 2D CT image synthesis models lack. The transition to 3D models allowed the preservation of contextual information from adjacent slices, improving efficiency and accuracy, especially for the poor-contrast challenging case of PDAC. PDAC’s challenging characteristics, such as an iso-attenuating or hypodense appearance and lack of well-defined margins, make tumor shape and texture learning challenging. To overcome these challenges and improve the performance of 3D GAN models, our innovation was to develop a 3D U-Net architecture for the generator, to improve shape and texture learning for PDAC tumors and pancreatic tissue. Thorough examination and validation across many datasets were conducted on the developed 3D GAN model, to ascertain the efficacy and applicability of the model in clinical contexts. Our approach offers a promising path for tackling the urgent requirement for creative and synergistic methods to combat PDAC. The development of this GAN-based model has the potential to alleviate data scarcity issues, elevate the quality of synthesized data, and thereby facilitate the progression of deep learning models, to enhance the accuracy and early detection of PDAC tumors, which could profoundly impact patient outcomes. Furthermore, the model has the potential to be adapted to other types of solid tumors, hence making significant contributions to the field of medical imaging in terms of image processing models.

 
more » « less
Award ID(s):
1853636
NSF-PAR ID:
10478755
Author(s) / Creator(s):
; ; ; ; ; ; ;
Publisher / Repository:
Cancers
Date Published:
Journal Name:
Cancers
Volume:
15
Issue:
23
ISSN:
2072-6694
Page Range / eLocation ID:
5496
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ABSTRACT

    Pancreatic ductal adenocarcinoma (PDAC) is among the most lethal of human malignancies. PDAC is characterized by dense fibrous stroma which obstructs drug delivery and plays complex tumor‐promoting roles. Photodynamic therapy (PDT) is a light‐based modality which has been demonstrated to be clinically feasible and effective for tumors of the pancreas. Here, we usein vitroheterocellular 3D co‐culture models in conjunction with imaging, bulk rheology and microrheology to investigate photodegradation of non‐cellular components of PDAC stroma (photodynamic stromal depletion, PSD). By measuring the rheology of extracellular matrix (ECM) before and after PDT we find that softening of ECM is concomitant with increased transport of nanoparticles (NPs). At the same time, as shown by us previously, photodestruction of stromal fibroblasts leads to enhanced tumor response to PDT. Here we specifically evaluate the capability of PSD to enhance RNA nanomedicine delivery, using a NP carrying an inhibitor of miR‐21‐5P, a PDAC oncomiR. We confirm improved delivery of this therapeutic NP after PSD by observation of increased expression of PDCD4, a protein target of miR‐21‐5P. Collectively, these results in 3D tumor models suggest that PSD could be developed to enhance delivery of other cancer therapeutics and improve tumor response to treatment.

     
    more » « less
  2. Early intervention in kidney cancer helps to improve survival rates. Abdominal computed tomography (CT) is often used to diagnose renal masses. In clinical practice, the manual segmentation and quantification of organs and tumors are expensive and time-consuming. Artificial intelligence (AI) has shown a significant advantage in assisting cancer diagnosis. To reduce the workload of manual segmentation and avoid unnecessary biopsies or surgeries, in this paper, we propose a novel end-to-end AI-driven automatic kidney and renal mass diagnosis framework to identify the abnormal areas of the kidney and diagnose the histological subtypes of renal cell carcinoma (RCC). The proposed framework first segments the kidney and renal mass regions by a 3D deep learning architecture (Res-UNet), followed by a dual-path classification network utilizing local and global features for the subtype prediction of the most common RCCs: clear cell, chromophobe, oncocytoma, papillary, and other RCC subtypes. To improve the robustness of the proposed framework on the dataset collected from various institutions, a weakly supervised learning schema is proposed to leverage the domain gap between various vendors via very few CT slice annotations. Our proposed diagnosis system can accurately segment the kidney and renal mass regions and predict tumor subtypes, outperforming existing methods on the KiTs19 dataset. Furthermore, cross-dataset validation results demonstrate the robustness of datasets collected from different institutions trained via the weakly supervised learning schema. 
    more » « less
  3. null (Ed.)
    Abstract Radiogenomics uses machine-learning (ML) to directly connect the morphologic and physiological appearance of tumors on clinical imaging with underlying genomic features. Despite extensive growth in the area of radiogenomics across many cancers, and its potential role in advancing clinical decision making, no published studies have directly addressed uncertainty in these model predictions. We developed a radiogenomics ML model to quantify uncertainty using transductive Gaussian Processes (GP) and a unique dataset of 95 image-localized biopsies with spatially matched MRI from 25 untreated Glioblastoma (GBM) patients. The model generated predictions for regional EGFR amplification status (a common and important target in GBM) to resolve the intratumoral genetic heterogeneity across each individual tumor—a key factor for future personalized therapeutic paradigms. The model used probability distributions for each sample prediction to quantify uncertainty, and used transductive learning to reduce the overall uncertainty. We compared predictive accuracy and uncertainty of the transductive learning GP model against a standard GP model using leave-one-patient-out cross validation. Additionally, we used a separate dataset containing 24 image-localized biopsies from 7 high-grade glioma patients to validate the model. Predictive uncertainty informed the likelihood of achieving an accurate sample prediction. When stratifying predictions based on uncertainty, we observed substantially higher performance in the group cohort (75% accuracy, n = 95) and amongst sample predictions with the lowest uncertainty (83% accuracy, n = 72) compared to predictions with higher uncertainty (48% accuracy, n = 23), due largely to data interpolation (rather than extrapolation). On the separate validation set, our model achieved 78% accuracy amongst the sample predictions with lowest uncertainty. We present a novel approach to quantify radiogenomics uncertainty to enhance model performance and clinical interpretability. This should help integrate more reliable radiogenomics models for improved medical decision-making. 
    more » « less
  4. Background

    Although pancreatic ductal adenocarcinoma (PDAC) has one of the lowest 5‐year survival rates of all cancers, differences in survival exist between patients with clinically identical characteristics. The authors previously demonstrated that keratin 17 (K17) expression in PDAC, measured by RNA sequencing or immunohistochemistry (IHC), is an independent negative prognostic biomarker. Only 20% of cases are candidates for surgical resection, but most patients are diagnosed by needle aspiration biopsy (NAB). The aims of this study were to determine whether there was a correlation in K17 scores detected in matched NABs and surgical resection tissue sections and whether K17 IHC in NAB cell block specimens could be used as a negative prognostic biomarker in PDAC.

    Methods

    K17 IHC was performed for a cohort of 70 patients who had matched NAB cell block and surgical resection samples to analyze the correlation of K17 expression levels. K17 IHC was also performed in cell blocks from discovery and validation cohorts. Kaplan‐Meier and Cox proportional hazards regression models were analyzed to determine survival differences in cases with different levels of K17 IHC expression.

    Results

    K17 IHC expression correlated in matched NABs and resection tissues. NAB samples were classified as high for K17 when ≥80% of tumor cells showed strong (2+) staining. High‐K17 cases, including stage‐matched cases, had shorter survival.

    Conclusions

    K17 has been identified as a robust and independent prognostic biomarker that stratifies clinical outcomes for cases that are diagnosed by NAB. Testing for K17 also has the potential to inform clinical decisions for optimization of chemotherapeutic interventions.

     
    more » « less
  5. Abstract

    Purpose: Paracrine activation of pro-fibrotic hedgehog (HH) signaling in pancreatic ductal adenocarcinoma (PDAC) results in stromal amplification that compromises tumor drug delivery, efficacy, and patient survival. Interdiction of HH-mediated tumor-stroma crosstalk with smoothened (SMO) inhibitors (SHHi) ‘primes’ PDAC patient-derived xenograft (PDX) tumors for increased drug delivery by transiently increasing vascular patency/permeability, and thereby macromolecule delivery. However, patient tumor isolates vary in their responsiveness, and responders show co-induction of epithelial-mesenchymal transition (EMT). We aimed to identify the signal derangements responsible for EMT induction and reverse them, and devise approaches to stratify SHHi-responsive tumors non-invasively based on clinically-quantifiable parameters. Experimental design: Animals underwent diffusion-weighted magnetic resonance (DW-MR) imaging for measurement of intra-tumor diffusivity. In parallel, tissue-level deposition of nanoparticle probes was quantified as a marker of vascular permeability/perfusion. Transcriptomic and bioinformatic analysis was employed to investigate SHHi-induced gene reprogramming and identify key ‘nodes’ responsible for EMT induction. Results: multiple patient tumor isolates responded to short-term SHH inhibitor exposure with increased vascular patency and permeability, with proportionate increases in tumor diffusivity. Non-responding PDXs did not. SHHi-treated tumors showed elevated FGF drive and distinctly higher nuclear localization of fibroblast growth factor receptor (FGFR1) in EMT-polarized tumor cells. Pan-FGFR inhibitor NVP-BGJ398 (Infigratinib) reversed the SHHi-induced EMT marker expression and nuclear FGFR1 accumulation without compromising the enhanced permeability effect. Conclusion: This dual-hit strategy of SMO and FGFR inhibition provides a clinically-translatable approach to compromise the profound impermeability of PDAC tumors. Furthermore, clinical deployment of DW-MR imaging could fulfill the essential clinical-translational requirement for patient stratification.

     
    more » « less