For decades, research in natural language processing
(NLP) has focused on summarization. Sequence-to-sequence
models for abstractive summarization have been studied extensively,
yet generated summaries commonly suffer from fabricated
content, and are often found to be near-extractive. We argue
that, to address these issues, summarizers need to acquire the
co-references that form multiple types of relations over input
sentences, e.g., 1-to-N, N-to-1, and N-to-N relations, since the
structured knowledge for text usually appears on these relations.
By allowing the decoder to pay different attention to the input
sentences for the same entity at different generation states, the
structured graph representations generate more informative summaries.
In this paper, we propose a hierarchical graph attention
networks (HGATs) for abstractive summarization with a topicsensitive
PageRank augmented graph. Specifically, we utilize dual
decoders, a sequential sentence decoder, and a graph-structured
decoder (which are built hierarchically) to maintain the global
context and local characteristics of entities, complementing each
other. We further design a greedy heuristic to extract salient
users’ comments while avoiding redundancy to drive a model
to better capture entity interactions. Our experimental results
show that our models produce significantly higher ROUGE scores
than variants without graph-based attention on both SSECIF and
CNN/Daily Mail (CNN/DM) datasets.
more »
« less
Gated Graph Neural Networks (GG-NNs) for Abstractive Multi-Comment Summarization
Abstract—Summarization of long sequences into a concise
statement is a core problem in natural language processing,
which requires a non-trivial understanding of the weakly structured
text. Therefore, integrating crowdsourced multiple users’
comments into a concise summary is even harder because (1) it
requires transferring the weakly structured comments to structured
knowledge. Besides, (2) the users comments are informal
and noisy. In order to capture the long-distance relationships in
staggered long sentences, we propose a neural multi-comment
summarization (MCS) system that incorporates the sentence
relationships via graph heuristics that utilize relation knowledge
graphs, i.e., sentence relation graphs (SRG) and approximate discourse
graphs (ADG). Motivated by the promising results of gated
graph neural networks (GG-NNs) on highly structured data, we
develop a GG-NNs with sequence encoder that incorporates SRG
or ADG in order to capture the sentence relationships. Specifically,
we employ the GG-NNs on both relation knowledge graphs,
with the sentence embeddings as the input node features and the
graph heuristics as the edges’ weights. Through multiple layerwise
propagations, the GG-NNs generate the salience for each
sentence from high-level hidden sentence features. Consequently,
we use a greedy heuristic to extract salient users’ comments while
avoiding the noise in comments. The experimental results show
that the proposed MCS improves the summarization performance
both quantitatively and qualitatively.
more »
« less
- Award ID(s):
- 1946391
- NSF-PAR ID:
- 10321975
- Date Published:
- Journal Name:
- 2021 IEEE International Conference on Big Knowledge (ICBK)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Extractive text summarization aims at extract- ing the most representative sentences from a given document as its summary. To extract a good summary from a long text document, sen- tence embedding plays an important role. Re- cent studies have leveraged graph neural net- works to capture the inter-sentential relation- ship (e.g., the discourse graph) to learn con- textual sentence embedding. However, those approaches neither consider multiple types of inter-sentential relationships (e.g., semantic similarity & natural connection), nor model intra-sentential relationships (e.g, semantic & syntactic relationship among words). To ad- dress these problems, we propose a novel Mul- tiplex Graph Convolutional Network (Multi- GCN) to jointly model different types of rela- tionships among sentences and words. Based on Multi-GCN, we propose a Multiplex Graph Summarization (Multi-GraS) model for extrac- tive text summarization. Finally, we evaluate the proposed models on the CNN/DailyMail benchmark dataset to demonstrate the effec- tiveness of our method.more » « less
-
Scene graph generation refers to the task of automatically mapping an image into a semantic structural graph, which requires correctly labeling each extracted object and their interaction relationships. Despite the recent success in object detection using deep learning techniques, inferring complex contextual relationships and structured graph representations from visual data remains a challenging topic. In this study, we propose a novel Attentive Relational Network that consists of two key modules with an object detection backbone to approach this problem. The first module is a semantic transformation module utilized to capture semantic embedded relation features, by translating visual features and linguistic features into a common semantic space. The other module is a graph self-attention module introduced to embed a joint graph representation through assigning various importance weights to neighboring nodes. Finally, accurate scene graphs are produced by the relation inference module to recognize all entities and the corresponding relations. We evaluate our proposed method on the widely-adopted Visual Genome dataset, and the results demonstrate the effectiveness and superiority of our model.more » « less
-
Answering complex questions about textual narratives requires reasoning over both stated context and the world knowledge that underlies it. However, pretrained language models (LM), the foundation of most modern QA systems, do not robustly represent latent relationships between concepts, which is necessary for reasoning. While knowledge graphs (KG) are often used to augment LMs with structured representations of world knowledge, it remains an open question how to effectively fuse and reason over the KG representations and the language context, which provides situational constraints and nuances. In this work, we propose GreaseLM, a new model that fuses encoded representations from pretrained LMs and graph neural networks over multiple layers of modality interaction operations. Information from both modalities propagates to the other, allowing language context representations to be grounded by structured world knowledge, and allowing linguistic nuances (e.g., negation, hedging) in the context to inform the graph representations of knowledge. Our results on three benchmarks in the commonsense reasoning (i.e., CommonsenseQA, OpenbookQA) and medical question answering (i.e., MedQA-USMLE) domains demonstrate that GreaseLM can more reliably answer questions that require reasoning over both situational constraints and structured knowledge, even outperforming models 8x larger.more » « less
-
Background: Relationships between bio-entities (genes, proteins, diseases, etc.) constitute a significant part of our knowledge. Most of this information is documented as unstructured text in different forms, such as books, articles and on-line pages. Automatic extraction of such information and storing it in structured form could help researchers more easily access such information and also make it possible to incorporate it in advanced integrative analysis. In this study, we developed a novel approach to extract bio-entity relationships information using Nature Language Processing (NLP) and a graph-theoretic algorithm. Methods: Our method, called GRGT (Grammatical Relationship Graph for Triplets), not only extracts the pairs of terms that have certain relationships, but also extracts the type of relationship (the word describing the relationships). In addition, the directionality of the relationship can also be extracted. Our method is based on the assumption that a triplet exists for a pair of interactions. A triplet is defined as two terms (entities) and an interaction word describing the relationship of the two terms in a sentence. We first use a sentence parsing tool to obtain the sentence structure represented as a dependency graph where words are nodes and edges are typed dependencies. The shortest paths among the pairs of words in the triplet are then extracted, which form the basis for our information extraction method. Flexible pattern matching scheme was then used to match a triplet graph with unknown relationship to those triplet graphs with labels (True or False) in the database. Results: We applied the method on three benchmark datasets to extract the protein-protein-interactions (PPIs), and obtained better precision than the top performing methods in literature. Conclusions: We have developed a method to extract the protein-protein interactions from biomedical literature. PPIs extracted by our method have higher precision among other methods, suggesting that our method can be used to effectively extract PPIs and deposit them into databases. Beyond extracting PPIs, our method could be easily extended to extracting relationship information between other bio-entities.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICKG52313.2021.00050
