skip to main content

This content will become publicly available on June 1, 2024

Title: SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundaries
Current Deep Network (DN) visualization and inter-pretability methods rely heavily on data space visualizations such as scoring which dimensions of the data are responsible for their associated prediction or generating new data features or samples that best match a given DN unit or representation. In this paper, we go one step further by developing the first provably exact method for computing the geometry of a DN's mapping - including its decision boundary - over a specified region of the data space. By lever-aging the theory of Continuous Piece- Wise Linear (CPWL) spline DNs, SplineCam exactly computes a DN's geometry without resorting to approximations such as sampling or architecture simplification. SplineCam applies to any DN architecture based on CPWL activation nonlinearities, including (leaky) ReLU, absolute value, maxout, and max-pooling and can also be applied to regression DNs such as implicit neural representations. Beyond decision boundary visualization and characterization, SplineCam enables one to compare architectures, measure generalizability, and sample from the decision boundary on or off the data manifold. Project website:  more » « less
Award ID(s):
1911094 1838177 1730574
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Page Range / eLocation ID:
3789 to 3798
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Computing or approximating the convex hull of a dataset plays a role in a wide range of applications, including economics, statistics, and physics, to name just a few. However, convex hull computation and approximation is exponentially complex, in terms of both memory and computation, as the ambient space dimension increases. In this paper, we propose DeepHull, a new convex hull approximation algorithm based on convex deep networks (DNs) with continuous piecewise-affine nonlinearities and nonnegative weights. The idea is that binary classification between true data samples and adversarially generated samples with such a DN naturally induces a polytope decision boundary that approximates the true data convex hull. A range of exploratory experiments demonstrates that DeepHull efficiently produces a meaningful convex hull approximation, even in a high-dimensional ambient space. 
    more » « less
  2. In this video, we show high-fidelity numerical results of supersonic spatially-developing turbulent boundary layers (SDTBL) under strong concave and concave curvatures and Mach = 2.86. The selected numerical tool is Direct Numerical Simulation (DNS) with high spatial/temporal resolution. The prescribed concave geometry is based on the experimental study by Donovan et al. (J. Fluid Mech., 259, 1-24, 1994). Turbulent inflow conditions are based on extracted data from a previous DNS over a flat plate (i.e., turbulence precursors). The comprehensive DNS information sheds important light on the transport phenomena inside turbulent boundary layers subject to strong deceleration or Adverse Pressure Gradient (APG) caused by concave walls as well as to strong acceleration or Favorable Pressure Gradient (FPG) caused by convex walls at different wall thermal conditions (i.e., cold, adiabatic and hot walls). In this opportunity, the selected scientific visualization tool is Virtual Reality (VR) by extracting vortex core iso-surfaces via the Q-criterion to convert them to a file format readable by the HTC Vive VR toolkit. The reader is invited to visit our Virtual Wind Tunnel (VWT) under a fully immersive environment for further details. The video is available at: 
    more » « less
  3. Advisor: Dr. Guillermo Araya (Ed.)
    The present study provides fundamental knowledge on an issue in fluid dynamics that is not well understood: flow separation and its association with heat and contaminant transport. In the separated region, a swirling motion increases the fluid drag force on the object. Very often, this is undesirable because it can seriously reduce the performance of engineered devices such as aircraft and turbines. Furthermore, Computational Fluid Dynamics (CFD) has gained ground due to its relatively low cost, high accuracy, and versatility. The principal aim of this study is to numerically elucidate the details behind momentum and passive scalar transport phenomena during turbulent boundary layer separation resulting from a wall-curvature-driven pressure gradient. With Open- FOAM CFD software, the numerical discretization of Reynolds-Averaged Navier-Stokes and passive scalar transport equations will be described in two-dimensional domains via the assessment of two popular turbulence models (i.e., the Spalart-Allmaras and the K-w SST model). The computational domain reproduces a wind tunnel geometry from previously performed experiments by Baskaran et al. (JFM, vol. 182 and 232 “A turbulent flow over a curved hill.” Part 1 and Part 2). Only the velocity and pressure distribution were measured there, which will be used for validation purposes in the present study. A second aim in the present work is the scientific visualization of turbulent events and coherent structures via the ParaView toolkit and Unity game engine. Thus, fully immersive visualization approaches will be used via virtual reality (VR) and augmented reality (AR) technologies. A Virtual Wind Tunnel (VWT), developed for the VR approach, emulates the presence in a wind tunnel laboratory and has already employed fluid flow visualization from an existing numerical database with high temporal/spatial resolution, i.e., Direct Numeric Simulation (DNS). In terms of AR, a FlowVisXR app for smartphones and HoloLens has been developed for portability. It allows the user to see virtual 3D objects (i.e., turbulent coherent structures) invoked into the physical world using the device as the lens. 
    more » « less
  4. Pinniped vibrissae possess a unique and complex three-dimensional topography, which has beneficial fluid flow characteristics such as substantial reductions in drag, lift, and vortex induced vibration. To understand and leverage these effects, the downstream vortex dynamics must be studied. Dye visualization is a traditional qualitative method of capturing these downstream effects, specifically in comparative biological investigations where complex equipment can be prohibitive. High-fidelity numerical simulations or experimental particle image velocimetry are commonplace for quantitative high-resolution flow measurements, but are computationally expensive, require costly equipment, and can have limited measurement windows. This study establishes a method for extracting quantitative data from standard dye visualization experiments on seal whisker geometries by leveraging novel but intuitive computer vision techniques, which maintain simplicity and an advantageous large experimental viewing window while automating the extraction of vortex frequency, position, and advection. Results are compared to direct numerical simulation (DNS) data for comparable geometries. Power spectra and Strouhal numbers show consistent behavior between methods for a Reynolds number of 500, with minima at the canonical geometry wavelength of 3.43 and a peak frequency of 0.2 for a Reynolds number of 250. The vortex tracking reveals a clear increase in velocity from roll-up to 3.5 whisker diameters downstream, with a strong overlap with the DNS data but shows steady results beyond the limited DNS window. This investigation provides insight into a valuable bio-inspired engineering model while advancing an analytical methodology that can readily be applied to a broad range of comparative biological studies. 
    more » « less
  5. Spatially invariant feature detection is a property of many visual systems that rely on visual information provided by two eyes. However, how information across both eyes is integrated for invariant feature detection is not fully understood. Here we investigate spatial invariance of looming responses in descending neurons (DNs) of Drosophila melanogaster. We find multiple looming responsive DNs integrate looming information across both eyes, even though their dendrites are restricted to a single visual hemisphere. One DN, the giant fiber (GF), responds invariantly to looming stimuli across tested azimuthal locations. We confirm visual information propagates to the GF from the contralateral eye through an unidentified pathway and demonstrate that the absence of this pathway alters GF responses to looming stimuli presented to the ipsilateral eye. Our data highlight a role for bilateral visual integration in generating consistent, looming-evoked escape responses that are robust across different stimulus locations and parameters.

    more » « less