skip to main content

Title: Spatial Variability Aware Deep Neural Networks (SVANN): A General Approach
Spatial variability is a prominent feature of various geographic phenomena such as climatic zones, USDA plant hardiness zones, and terrestrial habitat types (e.g., forest, grasslands, wetlands, and deserts). However, current deep learning methods follow a spatial-one-size-fits-all (OSFA) approach to train single deep neural network models that do not account for spatial variability. Quantification of spatial variability can be challenging due to the influence of many geophysical factors. In preliminary work, we proposed a spatial variability aware neural network (SVANN-I, formerly called SVANN ) approach where weights are a function of location but the neural network architecture is location independent. In this work, we explore a more flexible SVANN-E approach where neural network architecture varies across geographic locations. In addition, we provide a taxonomy of SVANN types and a physics inspired interpretation model. Experiments with aerial imagery based wetland mapping show that SVANN-I outperforms OSFA and SVANN-E performs the best of all.  more » « less
Award ID(s):
1916252 1737633
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
ACM Transactions on Intelligent Systems and Technology
Page Range / eLocation ID:
1 to 21
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Point-set classification for multiplexed pathology images aims to distinguish between the spatial configurations of cells within multiplexed immuno-fluorescence (mIF) images of different diseases. This problem is important towards aiding pathologists in diag- nosing diseases (e.g., chronic pancreatitis and pancreatic ductal adenocarcinoma). This problem is challenging because crucial spa- tial relationships are implicit in point sets and the non-uniform distribution of points makes the relationships complex. Manual morphologic or cell-count based methods, the conventional clinical approach for studying spatial patterns within mIF images, is limited by inter-observer variability. The current deep neural network methods for point sets (e.g., PointNet) are limited in learning the representation of implicit spatial relationships between categorical points. To overcome the limitation, we propose a new deep neural network (DNN) architecture, namely spatial-relationship aware neural networks (SRNet), with a novel design of representation learning layers. Experimental results with a University of Michigan mIF dataset show that the proposed method significantly outperforms the competing DNN methods, by 80%, reaching 95% accuracy. 
    more » « less
  2. The brain has long been divided into distinct areas based upon its local microstructure, or patterned composition of cells, genes, and proteins. While this taxonomy is incredibly useful and provides an essential roadmap for comparing two brains, there is also immense anatomical variability within areas that must be incorporated into models of brain architecture. In this work we leverage the expressive power of deep neural networks to create a data-driven model of intra- and inter-brain area variability. To this end, we train a convolutional neural network that learns relevant microstructural features directly from brain imagery. We then extract features from the network and fit a simple classifier to them, thus creating a simple, robust, and interpretable model of brain architecture. We further propose and show preliminary results for the use of features from deep neural networks in conjunction with unsupervised learning techniques to find fine-grained structure within brain areas. We apply our methods to micron-scale X-ray microtomography images spanning multiple regions in the mouse brain and demonstrate that our deep feature-based model can reliably discriminate between brain areas, is robust to noise, and can be used to reveal anatomically relevant patterns in neural architecture that the network wasn't trained to find. 
    more » « less
  3. Abstract

    The Southern Ocean (SO) is one of the most energetic regions in the world, where strong air‐sea fluxes, oceanic instabilities, and flow‐topography interactions yield complex dynamics. The Kerguelen Plateau (KP) region in the Indian sector of the SO is a hot spot for these energetic dynamics, which result in large spatiotemporal variability of physical and biogeochemical properties throughout the water column. Data from Argo floats (including biogeochemical) are used to investigate the spatial variability of intermediate and deep water physical and biogeochemical properties. An unsupervised machine learning classification approach is used to organize the float profiles into five SO frontal zones based on their temperature and salinity structure between 300 and 900 m, revealing not only the location of frontal zones and their boundaries but also the variability of water mass properties relative to the zonal mean state. We find that the variability is property dependent and can be more than twice as large as the mean zonal variability in intense eddy fields. In particular, we observe this intense variability in the intermediate and deep waters of the Subtropical Zone; in the Subantarctic Zone just west of and at KP; east of KP in the Polar Frontal Zone, associated with intense eddy variability that enhances deep waters convergence and mixing; and, as the deep waters upwell to the upper 500 m and mix with the surface waters in the southernmost regimes, each property shows a large variability.

    more » « less
  4. null (Ed.)
    Non-Rigid Structure from Motion (NRSfM) refers to the problem of reconstructing cameras and the 3D point cloud of a non-rigid object from an ensemble of images with 2D correspondences. Current NRSfM algorithms are limited from two perspectives: (i) the number of images, and (ii) the type of shape variability they can handle. These difficulties stem from the inherent conflict between the condition of the system and the degrees of freedom needing to be modeled – which has hampered its practical utility for many applications within vision. In this paper we propose a novel hierarchical sparse coding model for NRSFM which can overcome (i) and (ii) to such an extent, that NRSFM can be applied to problems in vision previously thought too ill posed. Our approach is realized in practice as the training of an unsupervised deep neural network (DNN) auto-encoder with a unique architecture that is able to disentangle pose from 3D structure. Using modern deep learning computational platforms allows us to solve NRSfM problems at an unprecedented scale and shape complexity. Our approach has no 3D supervision, relying solely on 2D point correspondences. Further, our approach is also able to handle missing/occluded 2D points without the need for matrix completion. Extensive experiments demonstrate the impressive performance of our approach where we exhibit superior precision and robustness against all available state-of-the-art works in some instances by an order of magnitude. We further propose a new quality measure (based on the network weights) which circumvents the need for 3D ground-truth to ascertain the confidence we have in the reconstructability. We believe our work to be a significant advance over state of-the-art in NRSFM. 
    more » « less
  5. Abstract

    We develop and demonstrate a new interpretable deep learning model specifically designed for image analysis in Earth system science applications. The neural network is designed to be inherently interpretable, rather than explained via post hoc methods. This is achieved by training the network to identify parts of training images that act as prototypes for correctly classifying unseen images. The new network architecture extends the interpretable prototype architecture of a previous study in computer science to incorporate absolute location. This is useful for Earth system science where images are typically the result of physics-based processes, and the information is often geolocated. Although the network is constrained to only learn via similarities to a small number of learned prototypes, it can be trained to exhibit only a minimal reduction in accuracy relative to noninterpretable architectures. We apply the new model to two Earth science use cases: a synthetic dataset that loosely represents atmospheric high and low pressure systems, and atmospheric reanalysis fields to identify the state of tropical convective activity associated with the Madden–Julian oscillation. In both cases, we demonstrate that considering absolute location greatly improves testing accuracies when compared with a location-agnostic method. Furthermore, the network architecture identifies specific historical dates that capture multivariate, prototypical behavior of tropical climate variability.

    Significance Statement

    Machine learning models are incredibly powerful predictors but are often opaque “black boxes.” The how-and-why the model makes its predictions is inscrutable—the model is not interpretable. We introduce a new machine learning model specifically designed for image analysis in Earth system science applications. The model is designed to be inherently interpretable and extends previous work in computer science to incorporate location information. This is important because images in Earth system science are typically the result of physics-based processes, and the information is often map based. We demonstrate its use for two Earth science use cases and show that the interpretable network exhibits only a small reduction in accuracy relative to black-box models.

    more » « less