More Like this

1. Designing a Uniform Meaning Representation for Natural Language Processing

   https://doi.org/10.1007/s13218-021-00722-w  

   Van Gysel, Jens E. ; Vigus, Meagan ; Chun, Jayeol ; Lai, Kenneth ; Moeller, Sarah ; Yao, Jiarui ; O’Gorman, Tim ; Cowell, Andrew ; Croft, William ; Huang, Chu-Ren ; et al ( April 2021 , KI - Künstliche Intelligenz)

   null (Ed.)

   In this paper we present Uniform Meaning Representation (UMR), a meaning representation designed to annotate the semantic content of a text. UMR is primarily based on Abstract Meaning Representation (AMR), an annotation framework initially designed for English, but also draws from other meaning representations. UMR extends AMR to other languages, particularly morphologically complex, low-resource languages. UMR also adds features to AMR that are critical to semantic interpretation and enhances AMR by proposing a companion document-level representation that captures linguistic phenomena such as coreference as well as temporal and modal dependencies that potentially go beyond sentence boundaries. 

   more » « less
2. Programming as a language for young children to express and explore mathematics in school

   https://doi.org/10.1111/bjet.13080  

   Goldenberg, E. Paul ; Carter, Cynthia J. ( May 2021 , British Journal of Educational Technology)

   Natural language helps express mathematical thinking and contexts. Conventional mathematical notation (CMN) best suits expressions and equations. Each is essential; each also has limitations, especially for learners. Our research studies how programming can be a advantageous third language that can also help restore mathematical connections that are hidden by topic‐centred curricula. Restoring opportunities for surprise and delight reclaims mathematics' creative nature. Studies of children's use of language in mathematics and their programming behaviours guide our iterative design/redesign of mathematical microworlds in which students, ages 7–11, use programming in their regular school lessonsas a language for learning mathematics. Though driven by mathematics, not coding, the microworlds develop the programming over time so that it continues to support children's developing mathematical ideas. This paper briefly describes microworlds EDC has tested with well over 400 7‐to‐8‐year‐olds in school, and others tested (or about to be tested) with over 200 8‐to‐11‐year‐olds. Our challenge was to satisfy schools' topical orientation and fit easily within regular classroom study but use and foreshadow other mathematical learning to remove the siloes. The design/redesign research and evaluation is exploratory, without formal methodology. We are also more formally studying effects on children's learning. That ongoing study is not reported here.

   What is already known

   Active learning—doing—supports learning.

   Collaborative learning—doingtogether—supports learning.

   Classroom discourse—focused, relevantdiscussion, not just listening—supports learning.

   Clear articulation of one's thinking, even just to oneself, helps develop that thinking.

   What this paper adds

   The common languages we use for classroom mathematics—natural language for conveying the meaning and context of mathematical situations and for explaining our reasoning; and the formal (written) language of conventional mathematical notation, the symbols we use in mathematical expressions and equations—are both essential but each presents hurdles that necessitate the other. Yet, even together, they are insufficient especially for young learners.

   Programming, appropriately designed and used, can be the third language that both reduces barriers and provides the missing expressive and creative capabilities children need.

   Appropriate design for use in regular mathematics classrooms requires making key mathematical content obvious, strong and the ‘driver’ of the activities, and requires reducing tech ‘overhead’ to near zero.

   Continued usefulness across the grades requires developing children's sophistication and knowledge with the language; the powerful ways that children rapidly acquire facility with (natural) language provides guidance for ways they can learn a formal language as well.

   Implications for policy and/or practice

   Mathematics teaching can take advantage of the ways children learn through experimentation and attention to the results, and of the ways children use their language brain even for mathematics.

   In particular, programming—in microworlds driven by the mathematical content, designed to minimise distraction and overhead, open to exploration and discoveryen routeto focused aims, and in which childrenself‐evaluate—can allow clear articulation of thought, experimentation with immediate feedback.

   As it aids the mathematics, it also builds computational thinking and satisfies schools' increasing concerns to broaden access to ideas of computer science.

    

   more » « less
3. Knowledge-driven Natural Language Understanding of English Text and its Applications

   Basu, Kinjal ; Varanasi, Sarat C. ; Shakerin, Farhad ; Arias, Joaquin ; Gupta, Gopal ( May 2021 , Proceedings of the AAAI Conference on Artificial Intelligence)

   null (Ed.)

   Understanding the meaning of a text is a fundamental challenge of natural language understanding (NLU) research. An ideal NLU system should process a language in a way that is not exclusive to a single task or a dataset. Keeping this in mind, we have introduced a novel knowledge driven semantic representation approach for English text. By leveraging the VerbNet lexicon, we are able to map syntax tree of the text to its commonsense meaning represented using basic knowledge primitives. The general purpose knowledge represented from our approach can be used to build any reasoning based NLU system that can also provide justification. We applied this approach to construct two NLU applications that we present here: SQuARE (Semantic-based Question Answering and Reasoning Engine) and StaCACK (Stateful Conversational Agent using Commonsense Knowledge). Both these systems work by ``truly understanding'' the natural language text they process and both provide natural language explanations for their responses while maintaining high accuracy. 

   more » « less
4. Deep Learning Methods for Sign Language Translation

   https://doi.org/10.1145/3477498  

   Ananthanarayana, Tejaswini ; Srivastava, Priyanshu ; Chintha, Akash ; Santha, Akhil ; Landy, Brian ; Panaro, Joseph ; Webster, Andre ; Kotecha, Nikunj ; Sah, Shagan ; Sarchet, Thomastine ; et al ( December 2021 , ACM Transactions on Accessible Computing)

   Many sign languages are bona fide natural languages with grammatical rules and lexicons hence can benefit from machine translation methods. Similarly, since sign language is a visual-spatial language, it can also benefit from computer vision methods for encoding it. With the advent of deep learning methods in recent years, significant advances have been made in natural language processing (specifically neural machine translation) and in computer vision methods (specifically image and video captioning). Researchers have therefore begun expanding these learning methods to sign language understanding. Sign language interpretation is especially challenging, because it involves a continuous visual-spatial modality where meaning is often derived based on context. The focus of this article, therefore, is to examine various deep learning–based methods for encoding sign language as inputs, and to analyze the efficacy of several machine translation methods, over three different sign language datasets. The goal is to determine which combinations are sufficiently robust for sign language translation without any gloss-based information. To understand the role of the different input features, we perform ablation studies over the model architectures (input features + neural translation models) for improved continuous sign language translation. These input features include body and finger joints, facial points, as well as vector representations/embeddings from convolutional neural networks. The machine translation models explored include several baseline sequence-to-sequence approaches, more complex and challenging networks using attention, reinforcement learning, and the transformer model. We implement the translation methods over multiple sign languages—German (GSL), American (ASL), and Chinese sign languages (CSL). From our analysis, the transformer model combined with input embeddings from ResNet50 or pose-based landmark features outperformed all the other sequence-to-sequence models by achieving higher BLEU2-BLEU4 scores when applied to the controlled and constrained GSL benchmark dataset. These combinations also showed significant promise on the other less controlled ASL and CSL datasets. 

   more » « less
5. Knowledge authoring for rule-based reasoning

   Gao, Tiantian ; Fodor, Paul ; Kifer, Michael ( October 2018 , The 17th International Conference on Ontologies, DataBases, and Applications of Semantics)

   Modern knowledge bases have matured to the extent of being capable of complex reasoning at scale. Unfortunately, wide deployment of this technology is still hindered by the fact that specifying the req- uisite knowledge requires skills that most domain experts do not have, and skilled knowledge engineers are in short supply. A way around this problem could be to acquire knowledge from text. However, the current knowledge acquisition technologies for information extraction are not up to the task because logic reasoning systems are extremely sensitive to er- rors in the acquired knowledge, and existing techniques lack the required accuracy by too large of a margin. Because of the enormous complexity of the problem, controlled natural languages (CNLs) were proposed in the past, but even they lack high enough accuracy. Instead of tackling the general problem of text understanding, our interest is in a related, but different, area of knowledge authoring—a technology designed to enable domain experts to manually create formalized knowledge using CNL. Our approach adopts and formalizes the FrameNet methodology for rep- resenting the meaning, enables incrementally-learnable and explainable semantic parsing, and harnesses rich knowledge graphs like BabelNet in the quest to obtain unique, disambiguated meaning of CNL sentences. Our experiments show that this approach is 95.6% accurate in standard- izing the semantic relations extracted from CNL sentences—far superior to alternative systems. 

   more » « less