skip to main content

This content will become publicly available on December 5, 2024

Title: Discontinuity-Aware 2D Neural Fields

Neural image representations offer the possibility of high fidelity, compact storage, and resolution-independent accuracy, providing an attractive alternative to traditional pixel- and grid-based representations. However, coordinate neural networks fail to capture discontinuities present in the image and tend to blur across them; we aim to address this challenge. In many cases, such as rendered images, vector graphics, diffusion curves, or solutions to partial differential equations, the locations of the discontinuities are known. We take those locations as input, represented as linear, quadratic, or cubic Bézier curves, and construct a feature field that is discontinuous across these locations and smooth everywhere else. Finally, we use a shallow multi-layer perceptron to decode the features into the signal value. To construct the feature field, we develop a new data structure based on a curved triangular mesh, with features stored on the vertices and on a subset of the edges that are marked as discontinuous. We show that our method can be used to compress a 100, 0002-pixel rendered image into a 25MB file; can be used as a new diffusion-curve solver by combining with Monte-Carlo-based methods or directly supervised by the diffusion-curve energy; or can be used for compressing 2D physics simulation data.

more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
Date Published:
Journal Name:
ACM Transactions on Graphics
Page Range / eLocation ID:
1 to 11
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Iron overload, a complication of repeated blood transfusions, can cause tissue damage and organ failure. The body has no regulatory mechanism to excrete excess iron, so iron overload must be closely monitored to guide therapy and measure treatment response. The concentration of iron in the liver is a reliable marker for total body iron content and is now measured noninvasively with magnetic resonance imaging (MRI). MRI produces a diagnostic image by measuring the signals emitted from the body in the presence of a constant magnetic field and radiofrequency pulses. At each pixel, the signal decay constant, T2*, can be calculated, providing insight about the structure of each tissue. Liver iron content can be quantified based on this T2* value because signal decay accelerates with increasing iron concentration. We developed a method to automatically segment the liver from the MRI image to accurately calculate iron content. Our current algorithm utilizes the active contour model for image segmentation, which iteratively evolves a curve until it reaches an edge or a boundary. We applied this algorithm to each MRI image in addition to a map of pixelwise T2* values, combining basic image processing with imaging physics. One of the limitations of this segmentation model is how it handles noise in the MRI data. Recent advancements in deep learning have enabled researchers to utilize convolutional neural networks to denoise and reconstruct images. We used the Trainable Nonlinear Reaction Diffusion network architecture to denoise the MRI images, allowing for smoother segmentation while preserving fine details. 
    more » « less
  2. Convolutional Neural Network (CNN) uses convolutional layers to explore spatial/temporal adjacency to construct new feature representations. So, CNN is commonly used for data with strong temporal/spatial correlations, but cannot be directly applied to generic learning tasks. In this paper, we propose to enable CNN for learning from generic data to improve classification accuracy. To take the full advantage of CNN’s feature learning power, we propose to convert each instance of the original dataset into a synthetic matrix/image format. To maximize the correlation in the constructed matrix/image, we use 0/1 optimization to reorder features and ensure that the ones with strong correlations are adjacent to each other. By using a feature reordering matrix, we are able to create a synthetic image to represent each instance. Because the constructed synthetic image preserves the original feature values and correlation, CNN can be applied to learn effective features for classification. Experiments and comparisons, on 22 benchmark datasets, demonstrate clear performance gain of applying CNN to generic datasets, compared to conventional machine learning methods. Furthermore, our method consistently outperforms approaches which directly apply CNN to generic datasets in naive ways. This research allows deep learning to be broadly applied to generic datasets. 
    more » « less
  3. Abstract

    Given its large plate scale of 21″ pixel−1, analyses of data from the Transiting Exoplanet Survey Satellite (TESS) space telescope must be wary of source confusion from blended light curves, which creates the potential to attribute observed photometric variability to the wrong astrophysical source. We explore the impact of light curve contamination on the detection of fast yellow pulsating supergiant (FYPS) stars as a case study to demonstrate the importance of confirming the source of detected signals in the TESS pixel data. While some of the FYPS signals have already been attributed to contamination from nearby eclipsing binaries, others are suggested to be intrinsic to the supergiant stars. In this work, we carry out a detailed analysis of the TESS pixel data to fit the source locations of the dominant signals reported for 17 FYPS stars with the Python packageTESS_localize. We are able to reproduce the detections of these signals for 14 of these sources, obtaining consistent source locations for four. Three of these originate from contaminants, while the signal reported for BZ Tuc is likely a spurious frequency introduced to the light curve of this 127 day Cepheid by the data processing pipeline. Other signals are not significant enough to be localized with our methods, or have long periods that are difficult to analyze given other TESS systematics. Since no localizable signals hold up as intrinsic pulsation frequencies of the supergiant targets, we argue that unambiguous detection of pulsational variability should be obtained before FYPS are considered a new class of pulsator.

    more » « less
  4. Medical image segmentation is one of the most challenging tasks in medical image analysis and widely developed for many clinical applications. While deep learning-based approaches have achieved impressive performance in semantic segmentation, they are limited to pixel-wise settings with imbalanced-class data problems and weak boundary object segmentation in medical images. In this paper, we tackle those limitations by developing a new two-branch deep network architecture which takes both higher level features and lower level features into account. The first branch extracts higher level feature as region information by a common encoder-decoder network structure such as Unet and FCN, whereas the second branch focuses on lower level features as support information around the boundary and processes in parallel to the first branch. Our key contribution is the second branch named Narrow Band Active Contour (NB-AC) attention model which treats the object contour as a hyperplane and all data inside a narrow band as support information that influences the position and orientation of the hyperplane. Our proposed NB-AC attention model incorporates the contour length with the region energy involving a fixed-width band around the curve or surface. The proposed network loss contains two fitting terms: (i) a high level feature (i.e., region) fitting term from the first branch; (ii) a lower level feature (i.e., contour) fitting term from the second branch including the (ii1) length of the object contour and (ii2) regional energy functional formed by the homogeneity criterion of both the inner band and outer band neighboring the evolving curve or surface. The proposed NB-AC loss can be incorporated into both 2D and 3D deep network architectures. The proposed network has been evaluated on different challenging medical image datasets, including DRIVE, iSeg17, MRBrainS18 and Brats18. The experimental results have shown that the proposed NB-AC loss outperforms other mainstream loss functions: Cross Entropy, Dice, Focal on two common segmentation frameworks Unet and FCN. Our 3D network which is built upon the proposed NB-AC loss and 3DUnet framework achieved state-of-the-art results on multiple volumetric datasets. 
    more » « less
  5. Abstract

    With the wide deployment of rechargeable batteries, battery degradation prediction has emerged as a challenging issue. However, battery life defined by capacity loss provides limited information regarding battery degradation. In this article, we explore the prediction of voltage‐capacity curves over battery lifetime based on a sequence to sequence (seq2seq) model. We demonstrate that the data of one present voltage‐capacity curve can be used as the input of the seq2seq model to accurately predict the voltage‐capacity curves at 100, 200, and 300 cycles ahead. This offers an opportunity to update battery management strategies in response to the predicted consequences. Besides, the model avoids feature engineering and is flexible to incorporate different numbers of input and output cycles. Therefore, it can be easily transplanted to other battery systems or electrochemical components. Furthermore, the model features data generation, that is, we can use the data of only one cycle to generate a large spectrum of aging data at the future cycles for developing other battery diagnosis or prognosis methods. In this way, the time and energy consuming battery degradation tests can be sharply reduced.image

    more » « less