Natural language generators for taskoriented
dialogue must effectively realize
system dialogue actions and their associated
semantics. In many applications,
it is also desirable for generators to control
the style of an utterance. To date,
work on task-oriented neural generation
has primarily focused on semantic fidelity
rather than achieving stylistic goals, while
work on style has been done in contexts
where it is difficult to measure content
preservation. Here we present three different
sequence-to-sequence models and
carefully test how well they disentangle
content and style. We use a statistical generator,
PERSONAGE, to synthesize a new
corpus of over 88,000 restaurant domain
utterances whose style varies according to
models of personality, giving us total control
over both the semantic content and the
stylistic variation in the training data. We
then vary the amount of explicit stylistic
supervision given to the three models. We
show that our most explicit model can simultaneously
achieve high fidelity to both
semantic and stylistic goals: this model
adds a context vector of 36 stylistic parameters
as input to the hidden state of the encoder
at each time step, showing the benefits
of explicit stylistic supervision, even
when the amount of training data is large.
more »
« less
Can Neural Generators for Dialogue Learn Sentence Planning and Discourse Structuring?
Responses in task-oriented dialogue systems
often realize multiple propositions
whose ultimate form depends on the use
of sentence planning and discourse structuring
operations. For example a recommendation
may consist of an explicitly
evaluative utterance e.g. Chanpen
Thai is the best option, along with content
related by the justification discourse
relation, e.g. It has great food and service,
that combines multiple propositions
into a single phrase. While neural generation
methods integrate sentence planning
and surface realization in one endto-end
learning framework, previous work
has not shown that neural generators can:
(1) perform common sentence planning
and discourse structuring operations; (2)
make decisions as to whether to realize
content in a single sentence or over multiple
sentences; (3) generalize sentence
planning and discourse relation operations
beyond what was seen in training. We
systematically create large training corpora
that exhibit particular sentence planning
operations and then test neural models
to see what they learn. We compare
models without explicit latent variables for
sentence planning with ones that provide
explicit supervision during training. We
show that only the models with additional
supervision can reproduce sentence planning
and discourse operations and generalize
to situations unseen in training.
more »
« less
- Award ID(s):
- 1748056
- NSF-PAR ID:
- 10079403
- Date Published:
- Journal Name:
- Proceedings of the 11th International Conference on Natural Language Generation
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Gender bias in language technologies has been widely studied, but research has mostly been restricted to a binary paradigm of gender. It is essential also to consider non-binary gender identities, as excluding them can cause further harm to an already marginalized group. In this paper, we comprehensively evaluate popular language models for their ability to correctly use English gender-neutral pronouns (e.g., singular they, them) and neo-pronouns (e.g., ze, xe, thon) that are used by individuals whose gender identity is not represented by binary pronouns. We introduce Misgendered, a framework for evaluating large language models’ ability to correctly use preferred pronouns, consisting of (i) instances declaring an individual’s pronoun, followed by a sentence with a missing pronoun, and (ii) an experimental setup for evaluating masked and auto-regressive language models using a unified method. When prompted out-of-the-box, language models perform poorly at correctly predicting neo-pronouns (averaging 7.6% accuracy) and gender-neutral pronouns (averaging 31.0% accuracy). This inability to generalize results from a lack of representation of non-binary pronouns in training data and memorized associations. Few-shot adaptation with explicit examples in the prompt improves the performance but plateaus at only 45.4% for neo-pronouns. We release the full dataset, code, and demo at https://tamannahossainkay.github.io/misgendered/.more » « less
-
Extractive text summarization aims at extract- ing the most representative sentences from a given document as its summary. To extract a good summary from a long text document, sen- tence embedding plays an important role. Re- cent studies have leveraged graph neural net- works to capture the inter-sentential relation- ship (e.g., the discourse graph) to learn con- textual sentence embedding. However, those approaches neither consider multiple types of inter-sentential relationships (e.g., semantic similarity & natural connection), nor model intra-sentential relationships (e.g, semantic & syntactic relationship among words). To ad- dress these problems, we propose a novel Mul- tiplex Graph Convolutional Network (Multi- GCN) to jointly model different types of rela- tionships among sentences and words. Based on Multi-GCN, we propose a Multiplex Graph Summarization (Multi-GraS) model for extrac- tive text summarization. Finally, we evaluate the proposed models on the CNN/DailyMail benchmark dataset to demonstrate the effec- tiveness of our method.more » « less
-
null (Ed.)Several works have aimed to explain why overparameterized neural networks generalize well when trained by Stochastic Gradient Descent (SGD). The consensus explanation that has emerged credits the randomized nature of SGD for the bias of the training process towards low-complexity models and, thus, for implicit regularization. We take a careful look at this explanation in the context of image classification with common deep neural network architectures. We find that if we do not regularize explicitly, then SGD can be easily made to converge to poorly-generalizing, high-complexity models: all it takes is to first train on a random labeling on the data, before switching to properly training with the correct labels. In contrast, we find that in the presence of explicit regularization, pretraining with random labels has no detrimental effect on SGD. We believe that our results give evidence that explicit regularization plays a far more important role in the success of overparameterized neural networks than what has been understood until now. Specifically, by penalizing complicated models independently of their fit to the data, regularization affects training dynamics also far away from optima, making simple models that fit the data well discoverable by local methods, such as SGD.more » « less
-
more » « less
Abstract Sustained anterior negativities have been the focus of much neurolinguistics research concerned with the language-memory interface, but what neural computations do they actually reflect? During the comprehension of sentences with long-distance dependencies between elements (such as object wh-questions), prior event-related potential work has demonstrated sustained anterior negativities (SANs) across the dependency region. SANs have been traditionally interpreted as an index of working memory resources responsible for storing the first element (e.g., wh-phrase) until the second element (e.g., verb) is encountered and the two can be integrated. However, it is also known that humans pursue top-down approaches in processing long-distance dependencies—predicting units and structures before actually encountering them. This study tests the hypothesis that SANs are a more general neural index of syntactic prediction. Across three experiments, we evaluated SANs in traditional wh-dependency contrasts, but also in sentences in which subordinating adverbials (e.g., although) trigger a prediction for a second clause, compared to temporal adverbials (e.g., today) that do not. We find no SAN associated with subordinating adverbials, contra the syntactic prediction hypothesis. More surprisingly, we observe SANs across matrix questions but not embedded questions. Since both involved identical long-distance dependencies, these results are also inconsistent with the traditional syntactic working memory account of the SAN. We suggest that a more general hypothesis that sustained neural activity supports working memory can be maintained, however, if the sustained anterior negativity reflects working memory encoding at the non-linguistic discourse representation level, rather than at the sentence level.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
