An official website of the United States government
As the societal impact of Deep Neural Networks (DNNs) grows, the goals for advancing DNNs become more complex and diverse, ranging from improving a conventional model accuracy metric to infusing advanced human virtues such as fairness, accountability, transparency, and unbiasedness. Recently, techniques in Explainable Artificial Intelligence (XAI) have been attracting considerable attention and have tremendously helped Machine Learning (ML) engineers in understand AI models. However, at the same time, we started to witness the emerging need beyond XAI among AI communities; based on the insights learned from XAI, how can we better empower ML engineers in steering their DNNs so that the model’s reasonableness and performance can be improved as intended? This article provides a timely and extensive literature overview of the field Explanation-Guided Learning (EGL), a domain of techniques that steer the DNNs’ reasoning process by adding regularization, supervision, or intervention on model explanations. In doing so, we first provide a formal definition of EGL and its general learning paradigm. Second, an overview of the key factors for EGL evaluation, as well as summarization and categorization of existing evaluation procedures and metrics for EGL are provided. Finally, the current and potential future application areas and directions of EGL are discussed, and an extensive experimental study is presented aiming at providing comprehensive comparative studies among existing EGL models in various popular application domains, such as Computer Vision and Natural Language Processing domains. Additional resources related to event prediction are included in the article website:https://kugaoyang.github.io/EGL/
more »
« less
Event Prediction in the Big Data Era: A Systematic Survey
Events are occurrences in specific locations, time, and semantics that nontrivially impact either our society or the nature, such as earthquakes, civil unrest, system failures, pandemics, and crimes. It is highly desirable to be able to anticipate the occurrence of such events in advance to reduce the potential social upheaval and damage caused. Event prediction, which has traditionally been prohibitively challenging, is now becoming a viable option in the big data era and is thus experiencing rapid growth, also thanks to advances in high performance computers and new Artificial Intelligence techniques. There is a large amount of existing work that focuses on addressing the challenges involved, including heterogeneous multi-faceted outputs, complex (e.g., spatial, temporal, and semantic) dependencies, and streaming data feeds. Due to the strong interdisciplinary nature of event prediction problems, most existing event prediction methods were initially designed to deal with specific application domains, though the techniques and evaluation procedures utilized are usually generalizable across different domains. However, it is imperative yet difficult to cross-reference the techniques across different domains, given the absence of a comprehensive literature survey for event prediction. This article aims to provide a systematic and comprehensive survey of the technologies, applications, and evaluations of event prediction in the big data era. First, systematic categorization and summary of existing techniques are presented, which facilitate domain experts’ searches for suitable techniques and help model developers consolidate their research at the frontiers. Then, comprehensive categorization and summary of major application domains are provided to introduce wider applications to model developers to help them expand the impacts of their research. Evaluation metrics and procedures are summarized and standardized to unify the understanding of model performance among stakeholders, model developers, and domain experts in various application domains. Finally, open problems and future directions are discussed. Additional resources related to event prediction are included in the paper website: http://cs.emory.edu/∼lzhao41/projects/event_prediction_site.html.
more »
« less
- PAR ID:
- 10602788
- Publisher / Repository:
- Association for Computing Machinery (ACM)
- Date Published:
- Journal Name:
- ACM Computing Surveys
- Volume:
- 54
- Issue:
- 5
- ISSN:
- 0360-0300
- Format(s):
- Medium: X Size: p. 1-37
- Size(s):
- p. 1-37
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The COVID-19 pandemic has resulted in more than 440 million confirmed cases globally and almost 6 million reported deaths as of March 2022. Consequently, the world experienced grave repercussions to citizens’ lives, health, wellness, and the economy. In responding to such a disastrous global event, countermeasures are often implemented to slow down and limit the virus’s rapid spread. Meanwhile, disaster recovery, mitigation, and preparation measures have been taken to manage the impacts and losses of the ongoing and future pandemics. Data-driven techniques have been successfully applied to many domains and critical applications in recent years. Due to the highly interdisciplinary nature of pandemic management, researchers have proposed and developed data-driven techniques across various domains. However, a systematic and comprehensive survey of data-driven techniques for pandemic management is still missing. In this article, we review existing data analysis and visualization techniques and their applications for COVID-19 and future pandemic management with respect to four phases (namely, Response, Recovery, Mitigation, and Preparation) in disaster management. Data sources utilized in these studies and specific data acquisition and integration techniques for COVID-19 are also summarized. Furthermore, open issues and future directions for data-driven pandemic management are discussed.more » « less
-
In making validity arguments, a central consideration is whether the instrument fairly and adequately covers intended content, and this is often evaluated by experts. While common procedures exist for quantitatively assessing this, the effect of loss aversion—a cognitive bias that would predict a tendency to retain items—on these procedures has not been investigated. For more novel constructs, experts are typically drawn from adjacent domains. In such cases, a related cognitive bias, the ownership effect, would predict that experts would be more loss averse when considering items closer to their domains. This study investigated whether loss aversion and the ownership effect are a concern in standard content validity evaluation procedures. In addition to including promising items to measure a relatively novel construct, framing agency, we included distractor items linked to other areas of our evaluators’ expertise. Experts evaluated all items following procedures outlined by Lawshe (1975). We found on average, experts were able to distinguish between the intended items and distractor items. Likewise, on average, experts were somewhat more likely to reject distractor items closer to their expertise. This suggests that loss aversion and the ownership effect are not likely to bias content validation procedures.more » « less
-
Abstract Artificial intelligence (AI) can be used to improve performance across a wide range of Earth system prediction tasks. As with any application of AI, it is important for AI to be developed in an ethical and responsible manner to minimize bias and other effects. In this work, we extend our previous work demonstrating how AI can go wrong with weather and climate applications by presenting a categorization of bias for AI in the Earth sciences. This categorization can assist AI developers to identify potential biases that can affect their model throughout the AI development life cycle. We highlight examples from a variety of Earth system prediction tasks of each category of bias.more » « less
-
Popular big data frameworks, ranging from Hadoop MapReduce to Spark, rely on garbage-collected languages, such as Java and Scala. Big data applications are especially sensitive to the effectiveness of garbage collection (i.e., GC), because they usually process a large volume of data objects that lead to heavy GC overhead. Lacking in-depth understanding of GC performance has impeded performance improvement in big data applications. In this paper, we conduct the first comprehensive evaluation on three popular garbage collectors, i.e., Parallel, CMS, and G1, using four representative Spark applications. By thoroughly investigating the correlation between these big data applications’ memory usage patterns and the collectors’ GC patterns, we obtain many findings about GC inefficiencies. We further propose empirical guidelines for application developers, and insightful optimization strategies for designing big-data-friendly garbage collectors.more » « less
