An official website of the United States government
Deep neural networks are susceptible to shortcut learning, using simple features to achieve low training loss without discovering essential semantic structure. Contrary to prior belief, we show that generative models alone are not sufficient to prevent shortcut learning, despite an incentive to recover a more comprehensive representation of the data than discriminative approaches. However, we observe that shortcuts are preferentially encoded with minimal information, a fact that generative models can exploit to mitigate shortcut learning. In particular, we propose Chroma-VAE, a two-pronged approach where a VAE classifier is initially trained to isolate the shortcut in a small latent subspace, allowing a secondary classifier to be trained on the complementary, shortcut-free latent subspace. In addition to demonstrating the efficacy of Chroma-VAE on benchmark and real-world shortcut learning tasks, our work highlights the potential for manipulating the latent space of generative classifiers to isolate or interpret specific correlations.
more »
« less
Generative Modeling of Atmospheric Convection
While cloud-resolving models can explicitly simulate the details of small-scale storm formation and morphology, these details are often ignored by climate models for lack of computational resources. Here, we explore the potential of generative modeling to cheaply recreate small-scale storms by designing and implementing a Variational Autoencoder (VAE) that performs structural replication, dimension- ality reduction, and clustering of high-resolution vertical velocity fields. Trained on ∼ 6 · 106 samples spanning the globe, the VAE successfully reconstructs the spatial structure of convection, per- forms unsupervised clustering of convective organization regimes, and identifies anomalous storm activity, confirming the potential of generative modeling to power stochastic parameterizations of convection in climate models.
more »
« less
- PAR ID:
- 10299896
- Date Published:
- Journal Name:
- CI2020
- Page Range / eLocation ID:
- 98 to 105
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Pervasive cirrus clouds in the upper troposphere and tropical tropopause layer (TTL) influence the climate by altering the top‐of‐atmosphere radiation balance and stratospheric water vapor budget. These cirrus are often associated with deep convection, which global climate models must parameterize and struggle to accurately simulate. By comparing high‐resolution global storm‐resolving models from the Dynamics of the Atmospheric general circulation Modeled On Non‐hydrostatic Domains (DYAMOND) intercomparison that explicitly simulate deep convection to satellite observations, we assess how well these models simulate deep convection, convectively generated cirrus, and deep convective injection of water into the TTL over representative tropical land and ocean regions. The DYAMOND models simulate deep convective precipitation, organization, and cloud structure fairly well over land and ocean regions, but with clear intermodel differences. All models produce frequent overshooting convection whose strongest updrafts humidify the TTL and are its main source of frozen water. Intermodel differences in cloud properties and convective injection exceed differences between land and ocean regions in each model. We argue that, with further improvements, global storm‐resolving models can better represent tropical cirrus and deep convection in present and future climates than coarser‐resolution climate models. To realize this potential, they must use available observations to perfect their ice microphysics and dynamical flow solvers.more » « less
-
This paper presents a multiscale modeling framework (MMF) to model moist at- mospheric limited-area weather. The MMF resolves large-scale convection using a coarse grid while simultaneously resolving local features through numerous fine local grids and coupling them seamlessly. Both large- and small-scale processes are modeled using the compressible Navier-Stokes equations within the Nonhydrostatic Unified Model of the Atmosphere (NUMA), and they are discretized using a continuous element-based Galerkin method (spectral elements) with high-order basis functions. Consequently, the large-scale and small-scale models share the same dynamical core but have the flexibility to be ad- justed individually. The proposed MMF method is tested in 2D and 3D idealized limited- area weather problems involving storm clouds produced by squall line and supercell sim- ulations. The MMF numerical results showed enhanced representation of cloud processes compared to the coarse model.more » « less
-
Abstract. The tropical tropopause layer (TTL) is a sea of vertical motions. Convectively generated gravity waves create vertical winds on scales of a few to thousands of kilometers as they propagate in a stable atmosphere. Turbulence from gravity wave breaking, radiatively driven convection, and Kelvin–Helmholtz instabilities stirs up the TTL on the kilometer scale. TTL cirrus clouds, which moderate the water vapor concentration in the TTL and stratosphere, form in the cold phases of large-scale (> 100 km) wave activity. It has been proposed in several modeling studies that small-scale (< 100 km) vertical motions control the ice crystal number concentration and the dehydration efficiency of TTL cirrus clouds. Here, we present the first observational evidence for this. High-rate vertical winds measured by aircraft are a valuable and underutilized tool for constraining small-scale TTL vertical wind variability, examining its impacts on TTL cirrus clouds, and evaluating atmospheric models. We use 20 Hz data from five National Aeronautics and Space Administration (NASA) campaigns to quantify small-scale vertical wind variability in the TTL and to see how it varies with ice water content, distance from deep convective cores, and height in the TTL. We find that 1 Hz vertical winds are well represented by a normal distribution, with a standard deviation of 0.2–0.4 m s−1. Consistent with a previous observational study that analyzed two out of the five aircraft campaigns that we analyze here, we find that turbulence is enhanced over the tropical west Pacific and within 100 km of convection and is most common in the lower TTL (14–15.5 km), closer to deep convection, and in the upper TTL (15.5–17 km), further from deep convection. An algorithm to classify turbulence and long-wavelength (5 km < λ < 100 km) and short-wavelength (λ < 5 km) gravity wave activity during level flight legs is applied to data from the Airborne Tropical TRopopause EXperiment (ATTREX). The most commonly sampled conditions are (1) a quiescent atmosphere with negligible small-scale vertical wind variability, (2) long-wavelength gravity wave activity (LW GWA), and (3) LW GWA with turbulence. Turbulence rarely occurs in the absence of gravity wave activity. Cirrus clouds with ice crystal number concentrations exceeding 20 L−1 and ice water content exceeding 1 mg m−3 are rare in a quiescent atmosphere but about 20 times more likely when there is gravity wave activity and 50 times more likely when there is also turbulence, confirming the results of the aforementioned modeling studies. Our observational analysis shows that small-scale gravity waves strongly influence the ice crystal number concentration and ice water content within TTL cirrus clouds. Global storm-resolving models have recently been run with horizontal grid spacing between 1 and 10 km, which is sufficient to resolve some small-scale gravity wave activity. We evaluate simulated vertical wind spectra (10–100 km) from four global storm-resolving simulations that have horizontal grid spacing of 3–5 km with aircraft observations from ATTREX. We find that all four models have too little resolved vertical wind at horizontal wavelengths between 10 and 100 km and thus too little small-scale gravity wave activity, although the bias is much less pronounced in global SAM than in the other models. We expect that deficient small-scale gravity wave activity significantly limits the realism of simulated ice microphysics in these models and that improved representation requires moving to finer horizontal and vertical grid spacing.more » « less
-
Semi-implicit graph variational auto-encoder (SIG-VAE) is proposed to expand the flexibility of variational graph auto-encoders (VGAE) to model graph data. SIG-VAE employs a hierarchical variational framework to enable neighboring node sharing for better generative modeling of graph dependency structure, together with a Bernoulli-Poisson link decoder. Not only does this hierarchical construction provide a more flexible generative graph model to better capture real-world graph properties, but also does SIG-VAE naturally lead to semi-implicit hierarchical variational inference that allows faithful modeling of implicit posteriors of given graph data, which may exhibit heavy tails, multiple modes, skewness, and rich dependency structures. SIG-VAE integrates a carefully designed generative model, well suited to model real-world sparse graphs, and a sophisticated variational inference network, which propagates the graph structural information and distribution uncertainty to capture complex posteriors. SIG-VAE clearly outperforms a simple combination of VGAE with variational inference, including semi-implicit variational inference~(SIVI) or normalizing flow (NF), which does not propagate uncertainty in its inference network, and provides more interpretable latent representations than VGAE does. Extensive experiments with a variety of graph data show that SIG-VAE significantly outperforms state-of-the-art methods on several different graph analytic tasks.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3429309.3429324
