An official website of the United States government
Abstract Total ice water content (IWC) derived from an isokinetic evaporator probe and ice crystal particle size distributions (PSDs) measured by a two-dimensional stereo probe and precipitation imaging probe installed on an aircraft during the 2014 European High Altitude Ice Crystals–North American High IWC field campaign (HAIC/HIWC) were used to characterize regions of high IWC consisting mainly of small ice crystals (HIWC_S) with IWC ≥ 1.0 g m−3and median mass diameter (MMD) < 0.5 mm. A novel fitting routine developed to automatically determine whether a unimodal, bimodal, or trimodal gamma distribution best fits a PSD was used to compare characteristics of HIWC_S and other PSDs (e.g., multimodality, gamma fit parameters) for HIWC_S simulations. The variation of these characteristics and bulk properties (MMD, IWC) was regressed with temperature, IWC, and vertical velocity. HIWC_S regions were most pronounced in updraft cores. The three modes of the PSD reveal different dominant processes contributing to ice growth: nucleation for maximum dimensionD< 0.15 mm, diffusion for 0.15 <D< 1.0 mm, and aggregation forD> 1.0 mm. The frequency of trimodal distributions increased with temperature. The volumes of equally plausible parameters derived in the phase space of gamma fit parameters increased with temperature for unimodal distributions and, for temperatures less than −27°C, for multimodal distributions. Bimodal distributions with 0.4 mm in the larger mode were most common in updraft cores and HIWC_S regions; bimodal distributions with 0.4 mm in the smaller mode were least common in convective cores.
more »
« less
Beyond Additive Fusion: Learning Non-Additive Multimodal Interactions
Multimodal fusion addresses the problem of analyzing spoken words in the multimodal context, including visual expressions and prosodic cues. Even when multimodal models lead to performance improvements, it is often unclear whether bimodal and trimodal interactions are learned or whether modalities are processed independently of each other. We propose Multimodal Residual Optimization (MRO) to separate unimodal, bimodal, and trimodal interactions in a multimodal model. This improves interpretability as the multimodal interaction can be quantified. Inspired by Occam’s razor, the main intuition of MRO is that (simpler) unimodal contributions should be learned before learning (more complex) bimodal and trimodal interactions. For example, bimodal predictions should learn to correct the mistakes (residuals) of unimodal predictions, thereby letting the bimodal predictions focus on the remaining bimodal interactions. Empirically, we observe that MRO successfully separates unimodal, bimodal, and trimodal interactions while not degrading predictive performance. We complement our empirical results with a human perception study and observe that MRO learns multimodal interactions that align with human judgments.
more »
« less
- Award ID(s):
- 1750439
- PAR ID:
- 10404844
- Date Published:
- Journal Name:
- Findings of the Association for Computational Linguistics: EMNLP 2022
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract This paper reports a study on the effects of particle size distribution (tuned by mixing different-sized powders) on density of a densely packed powder, powder bed density, and sintered density in binder jetting additive manufacturing. An analytical model was used first to study the mixture packing density. Analytical results showed that multimodal (bimodal or trimodal) mixtures could achieve a higher packing density than their component powders and there existed an optimal mixing fraction to achieve the maximum mixture packing density. Both a lower component particle size ratio (fine to coarse) and a larger component packing density ratio (fine to coarse) led to a larger maximum mixture packing density. A threshold existed for the component packing density ratio, below which the mixing method was not effective for density improvement. Its relationship to the component particle size ratio was calculated and plotted. In addition, the dependence of the optimal mixing fraction and maximum mixture packing density on the component particle size ratio and component packing density ratio was calculated and plotted. These plots can be used as theoretical tools to select parameters for the mixing method. Experimental results of tap density were consistent with the above-mentioned analytical predictions. Also, experimental measurements showed that powders with multimodal particle size distributions achieved a higher tap density, powder bed density, and sintered density in most cases.more » « less
-
An inherent strength of hydrogen/deuterium exchange coupled to mass spectrometry (HDX-MS) is its ability to detect the presence of multiple conformational states of a protein, which often manifest as multimodal isotopic envelopes. However, the statistical considerations for accurate analysis of multimodal spectra have yet to be established. Here we outline an unrestrained binomial distribution fitting approach with the corresponding statistical tests to accurately detect and, when possible, deconvolute isotopic distributions that contain multiple subpopulations. The algorithms have been incorporated into an updated version of the freely available software, HX-Express, and validated using known mixtures of peptides deuterated to varying degrees. This approach presents a readily accessible tool to fit and interpret bimodal and trimodal behavior in HDX-MS data for mixed populations, EX1 kinetics, and pulse labeling data.more » « less
-
In this paper, we argue that, as HCI becomes more multimodal with the integration of gesture, gaze, posture, and other nonverbal behavior, it is important to understand the role played by affordances and their associated actions in human-object interactions (HOI), so as to facilitate reasoning in HCI and HRI environments. We outline the requirements and challenges involved in developing a multimodal semantics for human-computer and human-robot interactions. Unlike unimodal interactive agents (e.g., text-based chatbots or voice-based personal digital assistants), multimodal HCI and HRI inherently require a notion of embodiment, or an understanding of the agent’s placement within the environment and that of its interlocutor. We present a dynamic semantics of the language, VoxML, to model human-computer, human-robot, and human-human interactions by creating multimodal simulations of both the communicative content and the agents’ common ground, and show the utility of VoxML information that is reified within the environment on computational understanding of objects for HOI.more » « less
-
Audio-visual representation learning aims to develop systems with human-like perception by utilizing correlation between auditory and visual information. However, current models often focus on a limited set of tasks, and generalization abilities of learned representations are unclear. To this end, we propose the AV-SUPERB benchmark that enables general-purpose evaluation of unimodal audio/visual and bimodal fusion representations on 7 datasets covering 5 audio-visual tasks in speech and audio processing. We evaluate 5 recent self-supervised models and show that none of these models generalize to all tasks, emphasizing the need for future study on improving universal model performance. In addition, we show that representations may be improved with intermediate-task fine-tuning and audio event classification with AudioSet serves as a strong intermediate task. We release our benchmark with evaluation code and a model submission platform to encourage further research in audio-visual learning.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
