More Like this

1. A Framework for Interactive Exploratory Learning Analytics

   https://doi.org/10.1007/978-3-319-91152-6_25  

   Mahzoon, Mohammad J; Maher, Mary Lou; Eltayeby, Omar; Dou, Wenwen; Grace, Kazjon (January 2018, Lecture notes in computer science)

   Many analytic tools have been developed to discover knowledge from student data. However, the knowledge discovery process requires advanced analytical modelling skills, making it the province of data scientists. This impedes the ability of educational leaders, professors, and advisors to engage with the knowledge discovery process directly. As a result, it is challenging for analysis to take advantage of domain expertise, making its outcome often neither interesting nor useful. Usually the outcome produced from such analytic tools is static, preventing domain experts from exploring different hypotheses by changing data models or predictive models inside the tool. We have developed a framework for interactive and exploratory learning analytics which begins to address these challenges. We engaged in data exploration and hypotheses generation with our university domain experts by conducting two focus groups. We used the findings of these focus groups to validate our framework, arguing that it enables domain experts to explore the data, analysis and interpretation of student data to discover useful and interesting knowledge. 

   more » « less
2. Towards Interactive Curation & Automatic Tuning of ML Pipelines

   https://doi.org/10.1145/3209889.3209891  

   Binnig, Carsten; Buratti, Benedetto; Chung, Yeounoh; Cousins, Cyrus; Kraska, Tim; Shang, Zeyuan; Upfal, Eli; Zeleznik, Robert; Zgraggen, Emanuel (January 2018, Proceedings of the Second Workshop on Data Management for End-to-End Machine Learning)

   Democratizing Data Science requires a fundamental rethinking of the way data analytics and model discovery is done. Available tools for analyzing massive data sets and curating machine learning models are limited in a number of fundamental ways. First, existing tools require well-trained data scientists to select the appropriate techniques to build models and to evaluate their outcomes. Second, existing tools require heavy data preparation steps and are often too slow to give interactive feedback to domain experts in the model building process, severely limiting the possible interactions. Third, current tools do not provide adequate analysis of statistical risk factors in the model development. In this work, we present the first iteration of QuIC-M (pronounced quick-m), an interactive human-in-the-loop data exploration and model building suite. The goal is to enable domain experts to build the machine learning pipelines an order of magnitude faster than machine learning experts while having model qualities comparable to expert solutions. 

   more » « less
3. Curio: A Dataflow-Based Framework for Collaborative Urban Visual Analytics

   https://doi.org/10.1109/TVCG.2024.3456353  

   Moreira, Gustavo; Hosseini, Maryam; Veiga, Carolina; Alexandre, Lucas; Colaninno, Nicola; de_Oliveira, Daniel; Ferreira, Nivan; Lage, Marcos; Miranda, Fabio (January 2025, IEEE Transactions on Visualization and Computer Graphics)

   Over the past decade, several urban visual analytics systems and tools have been proposed to tackle a host of challenges faced by cities, in areas as diverse as transportation, weather, and real estate. Many of these tools have been designed through collaborations with urban experts, aiming to distill intricate urban analysis workflows into interactive visualizations and interfaces. However, the design, implementation, and practical use of these tools still rely on siloed approaches, resulting in bespoke systems that are difficult to reproduce and extend. At the design level, these tools undervalue rich data workflows from urban experts, typically treating them only as data providers and evaluators. At the implementation level, they lack interoperability with other technical frameworks. At the practical use level, they tend to be narrowly focused on specific fields, inadvertently creating barriers to cross-domain collaboration. To address these gaps, we present Curio, a framework for collaborative urban visual analytics. Curio uses a dataflow model with multiple abstraction levels (code, grammar, GUI elements) to facilitate collaboration across the design and implementation of visual analytics components. The framework allows experts to intertwine data preprocessing, management, and visualization stages while tracking the provenance of code and visualizations. In collaboration with urban experts, we evaluate Curio through a diverse set of usage scenarios targeting urban accessibility, urban microclimate, and sunlight access. These scenarios use different types of data and domain methodologies to illustrate Curio’s flexibility in tackling pressing societal challenges. Curio is available at urbantk.org/curio. 

   more » « less
4. Flexible and scalable annotation tool to develop scene understanding datasets

   https://doi.org/10.1145/3546930.3547499  

   Elahi, Md Fazle; Tian, Renran; Luo, Xiao (June 2022, Workshop on Human-In-the-Loop Data Analytics (HILDA ’22))

   Recent progress in data-driven vision and language-based tasks demands developing training datasets enriched with multiple modalities representing human intelligence. The link between text and image data is one of the crucial modalities for developing AI models. The development process of such datasets in the video domain requires much effort from researchers and annotators (experts and non-experts). Researchers re-design annotation tools to extract knowledge from annotators to answer new research questions. The whole process repeats for each new question which is timeconsuming. However, since the last decade, there has been little change in how the researchers and annotators interact with the annotation process. We revisit the annotation workflow and propose a concept of an adaptable and scalable annotation tool. The concept emphasizes its users’ interactivity to make annotation process design seamless and efficient. Researchers can conveniently add newer modalities to or augment the extant datasets using the tool. The annotators can efficiently link free-form text to image objects. For conducting human-subject experiments on any scale, the tool supports the data collection for attaining group ground truth. We have conducted a case study using a prototype tool between two groups with the participation of 74 non-expert people. We find that the interactive linking of free-form text to image objects feels intuitive and evokes a thought process resulting in a high-quality annotation. The new design shows ≈ 35% improvement in the data annotation quality. On UX evaluation, we receive above-average positive feedback from 25 people regarding convenience, UI assistance, usability, and satisfaction. 

   more » « less
5. A Formative Study of Interactive Bias Metrics in Visual Analytics Using Anchoring Bias

   Wall, E; Blaha, L; Paul, C; Endert, A (January 2019, Human-Computer Interaction – INTERACT)

   Interaction is the cornerstone of how people perform tasks and gain insight in visual analytics. However, people’s inherent cognitive biases impact their behavior and decision making during their interactive visual analytic process. Understanding how bias impacts the visual analytic process, how it can be measured, and how its negative effects can be mitigated is a complex problem space. Nonetheless, recent work has begun to approach this problem by proposing theoretical computational metrics that are applied to user interaction sequences to measure bias in real-time. In this paper, we implement and apply these computational metrics in the context of anchoring bias. We present the results of a formative study examining how the metrics can capture anchoring bias in real-time during a visual analytic task. We present lessons learned in the form of considerations for applying the metrics in a visual analytic tool. Our findings suggest that these computational metrics are a promising approach for characterizing bias in users’ interactive behaviors. 

   more » « less