More Like this

1. Narrative Sensemaking: Strategies for Narrative Maps Construction

   https://doi.org/10.1109/VIS49827.2021.9623296  

   Norambuena, Brian Felipe; Mitra, Tanushree; North, Chris (October 2021, IEEE Visualization Conference)

   Narrative sensemaking is a fundamental process to understand sequential information. Narrative maps are a visual representation framework that can aid analysts in this process. They allow analysts to understand the big picture of a narrative, uncover new relationships between events, and model connections between storylines. As a sensemaking tool, narrative maps have applications in intelligence analysis, misinformation modeling, and computational journalism. In this work, we seek to understand how analysts construct narrative maps in order to improve narrative map representation and extraction methods. We perform an experiment with a data set of news articles. Our main contribution is an analysis of how analysts construct narrative maps. The insights extracted from our study can be used to design narrative map visualizations, extraction algorithms, and visual analytics tools to support the sensemaking process. 

   more » « less
2. Semantic Explanation of Interactive Dimensionality Reduction

   https://doi.org/10.1109/VIS49827.2021.9623322  

   Bian, Yali; North, Chris; Krokos, Eric; Joseph, Sarah (October 2021, IEEE Visualization Conference (VIS))

   Interactive dimensionality reduction helps analysts explore the high dimensional data based on their personal needs and domain-specific problems. Recently, expressive nonlinear models are employed to support these tasks. However, the interpretation of these human steered nonlinear models during human-in-the-loop analysis has not been explored. To address this problem, we present a new visual explanation design called semantic explanation. Semantic explanation visualizes model behaviors in a manner that is similar to users’ direct projection manipulations. This design conforms to the spatial analytic process and enables analysts better understand the updated model in response to their interactions. We propose a pipeline to empower interactive dimensionality reduction with semantic explanation using counterfactuals. Based on the pipeline, we implement a visual text analytics system with nonlinear dimensionality reduction powered by deep learning via the BERT model. We demonstrate the efficacy of semantic explanation with two case studies of academic article exploration and intelligence analysis. 

   more » « less
3. Socially situated artificial intelligence enables learning from human interaction

   https://doi.org/10.1073/pnas.2115730119  

   Krishna, Ranjay; Lee, Donsuk; Fei-Fei, Li; Bernstein, Michael S. (September 2022, Proceedings of the National Academy of Sciences)

   Regardless of how much data artificial intelligence agents have available, agents will inevitably encounter previously unseen situations in real-world deployments. Reacting to novel situations by acquiring new information from other people—socially situated learning—is a core faculty of human development. Unfortunately, socially situated learning remains an open challenge for artificial intelligence agents because they must learn how to interact with people to seek out the information that they lack. In this article, we formalize the task of socially situated artificial intelligence—agents that seek out new information through social interactions with people—as a reinforcement learning problem where the agent learns to identify meaningful and informative questions via rewards observed through social interaction. We manifest our framework as an interactive agent that learns how to ask natural language questions about photos as it broadens its visual intelligence on a large photo-sharing social network. Unlike active-learning methods, which implicitly assume that humans are oracles willing to answer any question, our agent adapts its behavior based on observed norms of which questions people are or are not interested to answer. Through an 8-mo deployment where our agent interacted with 236,000 social media users, our agent improved its performance at recognizing new visual information by 112%. A controlled field experiment confirmed that our agent outperformed an active-learning baseline by 25.6%. This work advances opportunities for continuously improving artificial intelligence (AI) agents that better respect norms in open social environments. 

   more » « less
4. Event-Driven Deep Learning for Edge Intelligence (EDL-EI) †

   https://doi.org/10.3390/s21186023  

   Shah, Sayed Khushal; Tariq, Zeenat; Lee, Jeehwan; Lee, Yugyung (September 2021, Sensors)

   null (Ed.)

   Edge intelligence (EI) has received a lot of interest because it can reduce latency, increase efficiency, and preserve privacy. More significantly, as the Internet of Things (IoT) has proliferated, billions of portable and embedded devices have been interconnected, producing zillions of gigabytes on edge networks. Thus, there is an immediate need to push AI (artificial intelligence) breakthroughs within edge networks to achieve the full promise of edge data analytics. EI solutions have supported digital technology workloads and applications from the infrastructure level to edge networks; however, there are still many challenges with the heterogeneity of computational capabilities and the spread of information sources. We propose a novel event-driven deep-learning framework, called EDL-EI (event-driven deep learning for edge intelligence), via the design of a novel event model by defining events using correlation analysis with multiple sensors in real-world settings and incorporating multi-sensor fusion techniques, a transformation method for sensor streams into images, and lightweight 2-dimensional convolutional neural network (CNN) models. To demonstrate the feasibility of the EDL-EI framework, we presented an IoT-based prototype system that we developed with multiple sensors and edge devices. To verify the proposed framework, we have a case study of air-quality scenarios based on the benchmark data provided by the USA Environmental Protection Agency for the most polluted cities in South Korea and China. We have obtained outstanding predictive accuracy (97.65% and 97.19%) from two deep-learning models on the cities’ air-quality patterns. Furthermore, the air-quality changes from 2019 to 2020 have been analyzed to check the effects of the COVID-19 pandemic lockdown. 

   more » « less
5. Supporting Theory Building in Design-Based Research through Large Scale Data-Based Models

   https://doi.org/10.5281/zenodo.12729820  

   Arastoopour_Irgens, Golnaz; Adisa, Ibrahim Oluwajoba; Sistla, Deepika; Famaye, Tolulope; Bailey, Cinamon; Behboudi, Atefeh; Adefisayo, Adenike Omalara (January 2024, International Educational Data Mining Society)

   Benjamin, Paaßen; Carrie, Demmans Epp (Ed.)

   Although the fields of educational data mining and learning analytics have grown in terms of the analytic sophistication and breadth of applications, the impact on theory-building has been limited. To move these fields forward, studies should not only be driven by learning theory but also the analytics should be used to inform theory. In this paper, we present an approach for integrating educational data mining models with design-based research approaches to promote theory-building that is informed by data-based models. This approach aligns theory, design of the learning environment, data collection, and analytic methods through iterations that focus on the refinement and improvement of all these components. We provide an example from our own work which is driven by a critical constructionist learning framework, the design and development of a digital learning environment for elementary-school aged children to learn about artificial intelligence within sociopolitical contexts, and the use of epistemic network analysis as a tool for modeling learning. We conclude with how this approach can be reciprocally beneficial in that educational data miners can use their models to inform theory and learning scientists can augment their theory-building practices through big data models. 

   more » « less