skip to main content


This content will become publicly available on October 1, 2024

Title: Diverse and faithful knowledge-grounded dialogue generation via sequential posterior inference
The capability to generate responses with diversity and faithfulness using factual knowledge is paramount for creating a human-like, trustworthy dialogue system. Common strategies either adopt a two-step paradigm, which optimizes knowledge selection and response generation separately and may overlook the inherent correlation between these two tasks, or leverage conditional variational method to jointly optimize knowledge selection and response generation by employing an inference network. In this paper, we present an end-to-end learning framework, termed Sequential Posterior Inference (SPI), capable of se- lecting knowledge and generating dialogues by approximately sampling from the posterior distribution. Unlike other methods, SPI does not require the inference network or assume a simple geometry of the posterior distribution. This straightforward and intuitive inference procedure of SPI directly queries the response generation model, allowing for accurate knowledge selection and generation of faithful responses. In addition to modeling contributions, our experimental results on two common dialogue datasets (Wizard of Wikipedia and Holl-E) demonstrate that SPI outperforms previous strong baselines according to both automatic and human evaluation metrics.  more » « less
Award ID(s):
2015577
NSF-PAR ID:
10469435
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
International Conference on Machine Learning (ICML 2023)
Date Published:
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    As the pressures on water resources are ever increasing, the organization of complex disparate data and scientific information to inform the actions to protect and enhance the resilience of freshwater resources is key for sustainable development and implementation of integrated water resource management (IWRM). Methodologies supporting IWRM implementation have largely focused on water management and governance, with less attention to evaluation methods of ecologic, economic, and social conditions. To assist in assessing water resource sustainability, the Integrated Hydro‐Environment Assessment Tool (IHEAT) has been developed to create a framework for different disciplines and interests to engage in structured dialogue. The IHEAT builds on the considerable body of knowledge developed around IWRM and seeks to place this information into a single framework that facilitates the cogeneration of knowledge between managers, stakeholders, and the communities affected by management decisions with the understanding that there is a need to merge expert analysis with traditional knowledge and the lived experience of communities. IHEAT merges the driver‐pressure‐state‐impact‐response (DPSIR) framework, the Millennium Ecosystem Assessment's ecosystem services and human well‐being (HWB) framework, sustainability criteria for water resource systems, and water resources indexes and sets of indicators to better understand spatiotemporal interactions between hydrologic, socioeconomic, and ecologic systems and evaluate impacts of disturbances on ecological goods and services and HWB. IHEAT consists of a Conceptual Template (IHEAT‐CT) which provides a systematic framework for assessing basin conditions and guiding indicator selection as well as an Assessment Interface (IHEAT‐AI) for organizing, processing, and assessing analytical results. The IHEAT‐CT, presented herein, is a rapid screening tool that connects water use directly, or through ecosystem goods and services (EGS), to constituents of HWB. Disturbance Templates for eight pressure types, such as land‐use change, climate change, and population growth, are provided to guide practitioners regarding potential changes to landscape elements in the hydrological cycle, impacts on EGS, and societal implications on HWB. The basin screening results in a summary report card illuminating key freshwater ecosystems, the EGS they provide, and potential responses to drivers and pressures acting on the hydrologic system. This screening provides a common understanding by technical and nontechnical parties and provides the foundation for more complex conceptual models should they be required. An indicator list guides the selection of hydrologic, ecologic, economic, and social analytical methods to support IWRM technical input.

     
    more » « less
  2. Abstract

    Bayesian data analysis is increasingly used in ecology, but prior specification remains focused on choosing non‐informative priors (e.g., flat or vague priors). One barrier to choosing more informative priors is that priors must be specified on model parameters (e.g., intercepts, slopes, and sigmas), but prior knowledge often exists on the level of the response variable. This is particularly true for common models in ecology, like generalized linear mixed models that have a link function and potentially dozens of parameters, each of which needs a prior distribution. We suggest that this difficulty can be overcome by simulating from the prior predictive distribution and visualizing the results on the scale of the response variable. In doing so, some common choices for non‐informative priors on parameters can easily be seen to produce biologically impossible values of response variables. Such implications of prior choices are difficult to foresee without visualization. We demonstrate a workflow for prior selection using simulation and visualization with two ecological examples (predator–prey body sizes and spider responses to food competition). This approach is not new, but its adoption by ecologists will help to better incorporate prior information in ecological models, thereby maximizing one of the benefits of Bayesian data analysis.

     
    more » « less
  3. Abstract

    Meta‐analytic techniques for mining the neuroimaging literature continue to exert an impact on our conceptualization of functional brain networks contributing to human emotion and cognition. Traditional theories regarding the neurobiological substrates contributing to affective processing are shifting from regional‐ towards more network‐based heuristic frameworks. To elucidate differential brain network involvement linked to distinct aspects of emotion processing, we applied an emergent meta‐analytic clustering approach to the extensive body of affective neuroimaging results archived in the BrainMap database. Specifically, we performed hierarchical clustering on the modeled activation maps from 1,747 experiments in the affective processing domain, resulting in five meta‐analytic groupings of experiments demonstrating whole‐brain recruitment. Behavioral inference analyses conducted for each of these groupings suggested dissociable networks supporting: (1) visual perception within primary and associative visual cortices, (2) auditory perception within primary auditory cortices, (3) attention to emotionally salient information within insular, anterior cingulate, and subcortical regions, (4) appraisal and prediction of emotional events within medial prefrontal and posterior cingulate cortices, and (5) induction of emotional responses within amygdala and fusiform gyri. These meta‐analytic outcomes are consistent with a contemporary psychological model of affective processing in which emotionally salient information from perceived stimuli are integrated with previous experiences to engender a subjective affective response. This study highlights the utility of using emergent meta‐analytic methods to inform and extend psychological theories and suggests that emotions are manifest as the eventual consequence of interactions between large‐scale brain networks.

     
    more » « less
  4. In recent years, the field of machine learning has made phenomenal progress in the pursuit of simulating real-world data generation processes. One notable example of such success is the variational autoencoder (VAE). In this work, with a small shift in perspective, we leverage and adapt VAEs for a different purpose: uncertainty quantification in scientific inverse problems. We introduce UQ-VAE: a flexible, adaptive, hybrid data/model-informed framework for training neural networks capable of rapid modelling of the posterior distribution representing the unknown parameter of interest. Specifically, from divergence-based variational inference, our framework is derived such that most of the information usually present in scientific inverse problems is fully utilized in the training procedure. Additionally, this framework includes an adjustable hyperparameter that allows selection of the notion of distance between the posterior model and the target distribution. This introduces more flexibility in controlling how optimization directs the learning of the posterior model. Further, this framework possesses an inherent adaptive optimization property that emerges through the learning of the posterior uncertainty. 
    more » « less
  5. Joan Bruna, Jan S (Ed.)
    In recent years, the field of machine learning has made phenomenal progress in the pursuit of simulating real-world data generation processes. One notable example of such success is the variational autoencoder (VAE). In this work, with a small shift in perspective, we leverage and adapt VAEs for a different purpose: uncertainty quantification in scientific inverse problems. We introduce UQ-VAE: a flexible, adaptive, hybrid data/model-constrained framework for training neural networks capable of rapid modelling of the posterior distribution representing the unknown parameter of interest. Specifically, from divergence-based variational inference, our framework is derived such that most of the information usually present in scientific inverse problems is fully utilized in the training procedure. Additionally, this framework includes an adjustable hyperparameter that allows selection of the notion of distance between the posterior model and the target distribution. This introduces more flexibility in controlling how optimization directs the learning of the posterior model. Further, this framework possesses an inherent adaptive optimization property that emerges through the learning of the posterior uncertainty. Numerical results for an elliptic PDE-constrained Bayesian inverse problem are provided to verify the proposed framework. 
    more » « less