An official website of the United States government
As the societal impact of Deep Neural Networks (DNNs) grows, the goals for advancing DNNs become more complex and diverse, ranging from improving a conventional model accuracy metric to infusing advanced human virtues such as fairness, accountability, transparency, and unbiasedness. Recently, techniques in Explainable Artificial Intelligence (XAI) have been attracting considerable attention and have tremendously helped Machine Learning (ML) engineers in understand AI models. However, at the same time, we started to witness the emerging need beyond XAI among AI communities; based on the insights learned from XAI, how can we better empower ML engineers in steering their DNNs so that the model’s reasonableness and performance can be improved as intended? This article provides a timely and extensive literature overview of the field Explanation-Guided Learning (EGL), a domain of techniques that steer the DNNs’ reasoning process by adding regularization, supervision, or intervention on model explanations. In doing so, we first provide a formal definition of EGL and its general learning paradigm. Second, an overview of the key factors for EGL evaluation, as well as summarization and categorization of existing evaluation procedures and metrics for EGL are provided. Finally, the current and potential future application areas and directions of EGL are discussed, and an extensive experimental study is presented aiming at providing comprehensive comparative studies among existing EGL models in various popular application domains, such as Computer Vision and Natural Language Processing domains. Additional resources related to event prediction are included in the article website:https://kugaoyang.github.io/EGL/
more »
« less
Exploring neuro-symbolic AI applications in geoscience: implications and future directions for mineral prediction
The integration of machine learning (ML) and deep learning (DL) into geoscience has experienced a pronounced uptick in recent years, a trend propelled by the intricate nature of geosystems and the abundance of data they produce. These computational methods have been harnessed across a spectrum of geoscientific challenges, from climate modeling to seismic analysis, exhibiting notable efficacy in extracting valuable insights from intricate geological datasets for applications such as mineral prediction. A thorough analysis of the literature indicates a marked escalation in AI-centric geoscience research starting in 2018, characterized by a predictive research orientation and a persistent focus on key computational terms. The thematic network and evolution analyses underscore the enduring prominence of “deep learning” and “machine learning” as pivotal themes, alongside progressive developments in “transfer learning” and “big data”. Despite these advancements, other methodologies have garnered comparatively lesser focus. While ML and DL have registered successes in the realm of mineral prediction, their amalgamation with domain-specific knowledge and symbolic reasoning could further amplify their interpretability and operational efficiency. Neuro-Symbolic AI (NSAI) emerges as a cutting-edge approach that synergizes DL’s robust capabilities with the precision of symbolic reasoning, facilitating the creation of models that are both powerful and interpretable. NSAI distinguishes itself by surmounting traditional ML constraints through the incorporation of expert insights and delivering explanatory power behind its predictive prowess, rendering it particularly advantageous for mineral prediction tasks. This literature review delves into the promising potential of NSAI, alongside ML and DL, within the geoscientific domain, spotlighting mineral prediction as a key area of focus. Despite the hurdles associated with infusing domain expertise into symbolic formats and mitigating biases inherent in symbolic reasoning, the application of NSAI in the realm of critical mineral prediction stands to catalyze a paradigm shift in the field. By bolstering prediction accuracy, enhancing decision-making processes, and fostering sustainable resource exploitation, NSAI holds the potential to significantly reshape geoscience’s future trajectory.
more »
« less
- Award ID(s):
- 2126315
- PAR ID:
- 10520111
- Publisher / Repository:
- Springer
- Date Published:
- Journal Name:
- Earth Science Informatics
- Volume:
- 17
- Issue:
- 3
- ISSN:
- 1865-0473
- Page Range / eLocation ID:
- 1819 to 1835
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We present Scallop, a language which combines the benefits of deep learning and logical reasoning. Scallop enables users to write a wide range of neurosymbolic applications and train them in a data- and compute-efficient manner. It achieves these goals through three key features: 1) a flexible symbolic representation that is based on the relational data model; 2) a declarative logic programming language that is based on Datalog and supports recursion, aggregation, and negation; and 3) a framework for automatic and efficient differentiable reasoning that is based on the theory of provenance semirings. We evaluate Scallop on a suite of eight neurosymbolic applications from the literature. Our evaluation demonstrates that Scallop is capable of expressing algorithmic reasoning in diverse and challenging AI tasks, provides a succinct interface for machine learning programmers to integrate logical domain knowledge, and yields solutions that are comparable or superior to state-of-the-art models in terms of accuracy. Furthermore, Scallop's solutions outperform these models in aspects such as runtime and data efficiency, interpretability, and generalizability.more » « less
-
Recently, a multi-agent based network automation architecture has been proposed. The architecture is named multi-agent based network automation of the network management system (MANA-NMS). The architectural framework introduced atomized network functions (ANFs). ANFs should be autonomous, atomic, and intelligent agents. Such agents should be implemented as an independent decision element, using machine/deep learning (ML/DL) as an internal cognitive and reasoning part. Using these atomic and intelligent agents as a building block, a MANA-NMS can be composed using the appropriate functions. As a continuation toward implementation of the architecture MANA-NMS, this paper presents a network traffic prediction agent (NTPA) and a network traffic classification agent (NTCA) for a network traffic management system. First, an NTPA is designed and implemented using DL algorithms, i.e., long short-term memory (LSTM), gated recurrent unit (GRU), multilayer perceptrons (MLPs), and convolutional neural network (CNN) algorithms as a reasoning and cognitive part of the agent. Similarly, an NTCA is designed using decision tree (DT), K-nearest neighbors (K-NN), support vector machine (SVM), and naive Bayes (NB) as a cognitive component in the agent design. We then measure the NTPA prediction accuracy, training latency, prediction latency, and computational resource consumption. The results indicate that the LSTM-based NTPA outperforms compared to GRU, MLP, and CNN-based NTPA in terms of prediction accuracy, and prediction latency. We also evaluate the accuracy of the classifier, training latency, classification latency, and computational resource consumption of NTCA using the ML models. The performance evaluation shows that the DT-based NTCA performs the best.more » « less
-
Improving the performance and explanations of ML algorithms is a priority for adoption by humans in the real world. In critical domains such as healthcare, such technology has significant potential to reduce the burden on humans and considerably reduce manual assessments by providing quality assistance at scale. In today’s data-driven world, artificial intelligence (AI) systems are still experiencing issues with bias, explainability, and human-like reasoning and interpretability. Causal AI is the technique that can reason and make human-like choices making it possible to go beyond narrow Machine learning-based techniques and can be integrated into human decision-making. It also offers intrinsic explainability, new domain adaptability, bias free predictions, and works with datasets of all sizes. In this tutorial of type lecture style, we detail how a richer representation of causality in AI systems using a knowledge graph (KG) based approach is needed for intervention and counterfactual reasoning (Figure 1), how do we get to model-based and domain explainability, how causal representations helps in web and health care.more » « less
-
The discovery of advanced thermal materials with exceptional phonon properties drives technological advancements, impacting innovations from electronics to superconductors. Understanding the intricate relationship between composition, structure, and phonon thermal transport properties is crucial for speeding up such discovery. Exploring innovative materials involves navigating vast design spaces and considering chemical and structural factors on multiple scales and modalities. Artificial intelligence (AI) is transforming science and engineering and poised to transform discovery and innovation. This era offers a unique opportunity to establish a new paradigm for the discovery of advanced materials by leveraging databases, simulations, and accumulated knowledge, venturing into experimental frontiers, and incorporating cutting-edge AI technologies. In this perspective, first, the general approach of density functional theory (DFT) coupled with phonon Boltzmann transport equation (BTE) for predicting comprehensive phonon properties will be reviewed. Then, to circumvent the extremely computationally demanding DFT + BTE approach, some early studies and progress of deploying AI/machine learning (ML) models to phonon thermal transport in the context of structure–phonon property relationship prediction will be presented, and their limitations will also be discussed. Finally, a summary of current challenges and an outlook of future trends will be given. Further development of incorporating AI/ML algorithms for phonon thermal transport could range from phonon database construction to universal machine learning potential training, to inverse design of materials with target phonon properties and to extend ML models beyond traditional phonons.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1007/s12145-024-01278-7
