skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: A Physics-Informed Deep Learning Deformable Medical Image Registration Method Based on Neural ODEs
Abstract An unsupervised machine learning method is introduced to align medical images in the context of the large deformation elasticity coupled with growth and remodeling biophysics. The technique, which stems from the principle of minimum potential energy in solid mechanics, consists of two steps: Firstly, in the predictor step, the geometric registration is achieved by minimizing a loss function composed of a dissimilarity measure and a regularizing term. Secondly, the physics of the problem, including the equilibrium equations along with growth mechanics, are enforced in a corrector step by minimizing the potential energy corresponding to a Dirichlet problem, where the predictor solution defines the boundary condition and is maintained by distance functions. The features of the new solution procedure, as well as the nature of the registration problem, are highlighted by considering several examples. In particular, registration problems containing large non-uniform deformations caused by extension, shearing, and bending of multiply-connected regions are used as benchmarks. In addition, we analyzed a benchmark biological example (registration for brain data) to showcase that the new deep learning method competes with available methods in the literature. We then applied the method to various datasets. First, we analyze the regrowth of the zebrafish embryonic fin from confocal imaging data. Next, we evaluate the quality of the solution procedure for two examples related to the brain. For one, we apply the new method for 3D image registration of longitudinal magnetic resonance images of the brain to assess cerebral atrophy, where a first-order ODE describes the volume loss mechanism. For the other, we explore cortical expansion during early fetal brain development by coupling the elastic deformation with morphogenetic growth dynamics. The method and examples show the ability of our framework to attain high-quality registration and, concurrently, solve large deformation elasticity balance equations and growth and remodeling dynamics.  more » « less
Award ID(s):
2227232
PAR ID:
10598790
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
Springer Science + Business Media
Date Published:
Journal Name:
International Journal of Computer Vision
ISSN:
0920-5691
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. G., Renata Raidou; Sommer, Björn; W., Torsten Kuhlen; Krone, Michael; Schultz, Thomas; Wu, Hsiang-Yun (Ed.)
    We introduce a novel multi-modal 3D image registration framework based on 3D user-guided deformation of both volume's shape and intensity values. Being able to apply deformations in 3D gives access to a wide new range of interactions allowing for the registration of images from any acquisition method and of any organ, complete or partial. Our framework uses a state of the art 3D volume rendering method for real-time feedback on the registration accuracy as well as the image deformation. We propose a novel methodological variation to accurately display 3D segmented voxel grids, which is a requirement in a registration context for visualizing a segmented atlas. Our pipeline is implemented in an open-source software (available via GitHub) and was directly used by biologists for registration of mouse brain model autofluorescence acquisition on the Allen Brain Atlas. The latter mapping allows them to retrieve regions of interest properly identified on the segmented atlas in acquired brain datasets and therefore extract only high-resolution images of those areas, avoiding the creation of images too large to be processed. 
    more » « less
  2. Abstract Solar images observed in different channels with different instruments are crucial to the study of solar activity. However, the images have different fields of view, causing them to be misaligned. It is essential to accurately register the images for studying solar activity from multiple perspectives. Image registration is described as an optimizing problem from an image to be registered to a reference image. In this paper, we proposed a novel coarse-to-fine solar image registration method to register the multichannel solar images. In the coarse registration step, we used the regular step gradient descent algorithm as an optimizer to maximize the normalized cross correlation metric. The fine registration step uses the Powell–Brent algorithms as an optimizer and brings the Mattes mutual information similarity metric to the minimum. We selected five pairs of images with different resolutions, rotation angles, and shifts to compare and evaluate our results to those obtained by scale-invariant feature transform and phase correlation. The images are observed by the 1.6 m Goode Solar Telescope at Big Bear Solar Observatory and the Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory. Furthermore, we used the mutual information and registration time criteria to quantify the registration results. The results prove that the proposed method not only reaches better registration precision but also has better robustness. Meanwhile, we want to highlight that the method can also work well for the time-series solar image registration. 
    more » « less
  3. null (Ed.)
    There is a growing importance in characterizing 3D shape quality in additive manufacturing (a.k.a. 3D printing). To accurately define the shape deviation between the designed product and actual build, shape registration of scanned point cloud data serves as a prerequisite for a reliable measurement. However, manual registration is currently heavily involved, for example, in obtaining initial matching of the design and the scanned product based on landmark features. The procedure can be inefficient, and more importantly, introduce potentially large operator-to-operator variations for complex geometries and deformation. Finding a sparse shape correspondence before refined registration would be meaningful to address this problem. In that case, automatic landmark selection has been a challenging issue, particularly for complicate geometric shapes like teeth. In this work we present an automatic landmark selection method for complicated 3D shapes. By incorporating subject matter knowledge (e.g., dental biometric information), a 3D shape will be first segmented through a new density-based clustering method. The geodesic distance is proposed as the distance metric in the revised clustering procedure. Geometrically informative features in each segment are automatically selected through the principal component analysis and Hotelling's T2 statistic. The proposed method is demonstrated in dental 3D printing application and could serve as a basis of sparse shape correspondence. 
    more » « less
  4. After a theory of morphogenesis in chemical cells was introduced in the 1950s, much attention had been devoted to the numerical solution of reaction-diffusion (RD) partial differential equations (PDEs). The Crank–Nicolson (CN) method has been a common second-order time-stepping procedure. However, the CN method may introduce spurious oscillations for nonsmooth data unless the time step size is sufficiently small. This article studies a nonoscillatory second-order time-stepping procedure for RD equations, called a variable- θ method , as a perturbation of the CN method. In each time level, the new method detects points of potential oscillations to implicitly resolve the solution there. The proposed time-stepping procedure is nonoscillatory and of a second-order temporal accuracy. Various examples are given to show effectiveness of the method. The article also performs a sensitivity analysis for the numerical solution of biological pattern forming models to conclude that the numerical solution is much more sensitive to the spatial mesh resolution than the temporal one. As the spatial resolution becomes higher for an improved accuracy, the CN method may produce spurious oscillations, while the proposed method results in stable solutions. 
    more » « less
  5. Mechanical forces generated by dynamic cellular activities play a crucial role in the morphogenesis and growth of biological tissues. While the influence of mechanics is clear, many questions arise regarding the way by which mechanical forces communicate with biological processes at the level of a confluent cell population. Some answers may be found in the development of mathematical models that are capable of describing the emerging behavior of a large population of active agents based on individualistic rules (single-cell response). In this perspective, the present work presents a continuum-scale model that can capture, in an average sense, the active mechanics and evolution of a confluent tissue with or without external mechanical constraints. For this, we conceptualize a confluent cell population (in a monolayer) as a deformable dynamic network, where a single cell can modify the topology of its neighborhood by swapping neighbors or dividing. With this description, we use concepts from statistical mechanics and the transient network theory to derive an equivalent active visco-elastic continuum model, which can recapitulate some of the salient features of the underlying network at the macroscale. Without loss of generality, the cell network is here assumed to follow well-known rules used in vertex model simulations, which are: (a) cell elasticity based on its bulk and cortical elasticity, (b) cell intercalation (or T1 transition), and (c) cell proliferation (expansion and division). We show, through examples and illustrations, that the model is able to characterize complex cross-talk between mechanical forces and biological processes, which are likely to drive the emergent growth and deformation of cell aggregates. 
    more » « less