More Like this

1. AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data

   https://doi.org/10.18653/v1/2020.emnlp-main.31  

   Xu, Silei ; Semnani, Sina ; Campagna, Giovanni ; Lam, Monica ( November 2020 , Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP))

   null (Ed.)

   We propose AutoQA, a methodology and toolkit to generate semantic parsers that answer questions on databases, with no manual effort. Given a database schema and its data, AutoQA automatically generates a large set of high-quality questions for training that covers different database operations. It uses automatic paraphrasing combined with template-based parsing to find alternative expressions of an attribute in different parts of speech. It also uses a novel filtered auto-paraphraser to generate correct paraphrases of entire sentences. We apply AutoQA to the Schema2QA dataset and obtain an average logical form accuracy of 62.9% when tested on natural questions, which is only 6.4% lower than a model trained with expert natural language annotations and paraphrase data collected from crowdworkers. To demonstrate the generality of AutoQA, we also apply it to the Overnight dataset. AutoQA achieves 69.8% answer accuracy, 16.4% higher than the state-of-the-art zero-shot models and only 5.2% lower than the same model trained with human data. 

   more » « less
2. ECOD: integrating classifications of protein domains from experimental and predicted structures

   https://doi.org/10.1093/nar/gkae1029  

   Schaeffer, R. Dustin ; Medvedev, Kirill E. ; Andreeva, Antonina ; Chuguransky, Sara Rocio ; Pinto, Beatriz Lazaro ; Zhang, Jing ; Cong, Qian ; Bateman, Alex ; Grishin, Nick V. ( November 2024 , Nucleic Acids Research)

   The evolutionary classification of protein domains (ECOD) classifies protein domains using a combination of sequence and structural data (http://prodata.swmed.edu/ecod). Here we present the culmination of our previous efforts at classifying domains from predicted structures, principally from the AlphaFold Database (AFDB), by integrating these domains with our existing classification of PDB structures. This combined classification includes both domains from our previous, purely experimental, classification of domains as well as domains from our provisional classification of 48 proteomes in AFDB predicted from model organisms and organisms of concern to global health. ECOD classifies over 1.8 M domains from over 1000 000 proteins collectively deposited in the PDB and AFDB. Additionally, we have changed the F-group classification reference used for ECOD, deprecating our original ECODf library and instead relying on direct collaboration with the Pfam sequence family database to inform our classification. Pfam provides similar coverage of ECOD with family classification while being more accurate and less redundant. By eliminating duplication of effort, we can improve both classifications. Finally, we discuss the initial deployment of DrugDomain, a database of domain-ligand interactions, on ECOD and discuss future plans.

    

   more » « less
3. Human-in-the-loop Schema Induction

   https://doi.org/10.18653/v1/2023.acl-demo.1  

   Zhang, Tianyi ; Tham, Isaac ; Hou, Zhaoyi ; Ren, Jiaxuan ; Zhou, Leon ; Xu, Hainiu ; Zhang, Li ; Martin, Lara ; Dror, Rotem ; Li, Sha ; et al ( January 2023 , Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics)

   Schema induction builds a graph representation explaining how events unfold in a scenario. Existing approaches have been based on information retrieval (IR) and information extraction (IE), often with limited human curation. We demonstrate a human-in-the-loop schema induction system powered by GPT-3. We first describe the different modules of our system, including prompting to generate schematic elements, manual edit of those elements, and conversion of those into a schema graph. By qualitatively comparing our system to previous ones, we show that our system not only transfers to new domains more easily than previous approaches, but also reduces efforts of human curation thanks to our interactive interface. 

   more » « less
4. Improve Learning from Crowds via Generative Augmentation

   https://doi.org/10.1145/3447548.3467409  

   Zhendong, Chu ; Hongning, Wang ( August 2021 , Proceedings of the 27th ACM SIGKDDConference on Knowledge Discovery and Data Mining (KDD ’21))

   null (Ed.)

   Crowdsourcing provides an efficient label collection schema for supervised machine learning. However, to control annotation cost, each instance in the crowdsourced data is typically annotated by a small number of annotators. This creates a sparsity issue and limits the quality of machine learning models trained on such data. In this paper, we study how to handle sparsity in crowdsourced data using data augmentation. Specifically, we propose to directly learn a classifier by augmenting the raw sparse annotations. We implement two principles of high-quality augmentation using Generative Adversarial Networks: 1) the generated annotations should follow the distribution of authentic ones, which is measured by a discriminator; 2) the generated annotations should have high mutual information with the ground-truth labels, which is measured by an auxiliary network. Extensive experiments and comparisons against an array of state-of-the-art learning from crowds methods on three real-world datasets proved the effectiveness of our data augmentation framework. It shows the potential of our algorithm for low-budget crowdsourcing in general. 

   more » « less
5. VOCAL: Video Organization and Interactive Compositional AnaLytics

   Daum, Maureen ; Zhang, Enhao ; He, Dong ; Balazinska, Magdalena ; Haynes, Brandon ; Krishna, Ranjay ; Craig, Apryle ; Wirsing, Aaron ( January 2022 , 12th Annual Conference on Innovative Data Systems Research (CIDR ’22))

   Current video database management systems (VDBMSs) fail to support the growing number of video datasets in diverse domains because these systems assume clean data and rely on pretrained models to detect known objects or actions. Existing systems also lack good support for compositional queries that seek events con- sisting of multiple objects with complex spatial and temporal rela- tionships. In this paper, we propose VOCAL, a vision of a VDBMS that supports efficient data cleaning, exploration and organization, and compositional queries, even when no pretrained model exists to extract semantic content. These techniques utilize optimizations to minimize the manual effort required of users. 

   more » « less