An official website of the United States government
Magnetic particle imaging (MPI) is an emerging noninvasive molecular imaging modality with high sensitivity and specificity, exceptional linear quantitative ability, and potential for successful applications in clinical settings. Computed tomography (CT) is typically combined with the MPI image to obtain more anatomical information. Herein, a deep learning‐based approach for MPI‐CT image segmentation is presented. The dataset utilized in training the proposed deep learning model is obtained from a transgenic mouse model of breast cancer following administration of indocyanine green (ICG)‐conjugated superparamagnetic iron oxide nanoworms (NWs‐ICG) as the tracer. The NWs‐ICG particles progressively accumulate in tumors due to the enhanced permeability and retention (EPR) effect. The proposed deep learning model exploits the advantages of the multihead attention mechanism and the U‐Net model to perform segmentation on the MPI‐CT images, showing superb results. In addition, the model is characterized with a different number of attention heads to explore the optimal number for our custom MPI‐CT dataset.
more »
« less
Recurrent and convolutional neural networks for sequential multispectral optoacoustic tomography ( MSOT ) imaging
Abstract Multispectral optoacoustic tomography (MSOT) is a beneficial technique for diagnosing and analyzing biological samples since it provides meticulous details in anatomy and physiology. However, acquiring high through‐plane resolution volumetric MSOT is time‐consuming. Here, we propose a deep learning model based on hybrid recurrent and convolutional neural networks to generate sequential cross‐sectional images for an MSOT system. This system provides three modalities (MSOT, ultrasound, and optoacoustic imaging of a specific exogenous contrast agent) in a single scan. This study used ICG‐conjugated nanoworms particles (NWs‐ICG) as the contrast agent. Instead of acquiring seven images with a step size of 0.1 mm, we can receive two images with a step size of 0.6 mm as input for the proposed deep learning model. The deep learning model can generate five other images with a step size of 0.1 mm between these two input images meaning we can reduce acquisition time by approximately 71%.
more »
« less
- Award ID(s):
- 2237142
- PAR ID:
- 10433652
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Journal of Biophotonics
- ISSN:
- 1864-063X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Histological staining is a vital step in diagnosing various diseases and has been used for more than a century to provide contrast in tissue sections, rendering the tissue constituents visible for microscopic analysis by medical experts. However, this process is time consuming, labour intensive, expensive and destructive to the specimen. Recently, the ability to virtually stain unlabelled tissue sections, entirely avoiding the histochemical staining step, has been demonstrated using tissue-stain-specific deep neural networks. Here, we present a new deep-learning-based framework that generates virtually stained images using label-free tissue images, in which different stains are merged following a micro-structure map defined by the user. This approach uses a single deep neural network that receives two different sources of information as its input: (1) autofluorescence images of the label-free tissue sample and (2) a “digital staining matrix”, which represents the desired microscopic map of the different stains to be virtually generated in the same tissue section. This digital staining matrix is also used to virtually blend existing stains, digitally synthesizing new histological stains. We trained and blindly tested this virtual-staining network using unlabelled kidney tissue sections to generate micro-structured combinations of haematoxylin and eosin (H&E), Jones’ silver stain, and Masson’s trichrome stain. Using a single network, this approach multiplexes the virtual staining of label-free tissue images with multiple types of stains and paves the way for synthesizing new digital histological stains that can be created in the same tissue cross section, which is currently not feasible with standard histochemical staining methods.more » « less
-
Abstract An inexpensive and compact underwater digital camera imaging system was developed to collect in situ high resolution images of flocculated suspended sediment at depths of up to 60 meters. The camera has a field of view of 3.7 × 2.8 mm and can resolve particles down to 5 . Depending on the degree of flocculation, the system is capable of accurately sizing particles to concentrations up to 500 mg/L. The system is fast enough to allow for profiling whereby size distributions of suspended particles and flocs can be provided at multiple verticals within the water column over a relatively short amount of time (approximately 15 min for a profile of 15 m). Using output from image processing routines, methods are introduced to estimate the mass suspended sediment concentration (SSC) from the images and to separate identified particles into sand and mud floc populations. The combination of these two methods allows for the size and concentration estimates of each fraction independently. The camera and image analysis methods are used in both the laboratory and the Mississippi River for development and testing. Output from both settings are presented in this study.more » « less
-
This paper proposes a phase-to-depth deep learning model to repair shadow-induced errors for fringe projection profilometry (FPP). The model comprises two hourglass branches that extract information from texture images and phase maps and fuses the information from the two branches by concatenation and weights. The input of the proposed model contains texture images, masks, and unwrapped phase maps, and the ground truth is the depth map from CAD models. A loss function was chosen to consider image details and structural similarity. The training data contain 1200 samples in the verified virtual FPP system. After training, we conduct experiments on the virtual and real-world scanning data, and the results support the model’s effectiveness. The mean absolute error and the root mean squared error are 1.0279 mm and 1.1898 mm on the validation dataset. In addition, we analyze the influence of ambient light intensity on the model’s performance. Low ambient light limits the model’s performance as the model cannot extract valid information from the completely dark shadow regions in texture images. The contribution of each branch network is also investigated. Features from the texture-dominant branch are leveraged as guidance to remedy shadow-induced errors. Information from the phase-dominant branch network makes accurate predictions for the whole object. Our model provides a good reference for repairing shadow-induced errors in the FPP system.more » « less
-
Abstract We propose a binary classification model rooted in state‐of‐the‐art deep learning techniques to predict whether or not complete‐interface rupture is imminent along a numerical megathrust fault. The models are trained on labeled 2D space‐time input features taken from the synthetic fault system. We contrast the performance of two neural networks trained on three types of data, to determine the relative predictive power of each. The neural networks are able to discriminate imminent complete rupture precursors from everything else, thus providing a relative size and time forecast. Vertical displacements along the fault demonstrate relatively good predictive power. The results confirm previous qualitative observations that precursory deformation scales with upcoming event size, consistent with the preslip model for earthquake nucleation. The methods we propose are adaptable and can be modified to use 3D data in the future.more » « less
