skip to main content

Title: iBall: Augmenting Basketball Videos with Gaze-moderated Embedded Visualizations
We present iBall, a basketball video-watching system that leverages gaze-moderated embedded visualizations to facilitate game understanding and engagement of casual fans. Video broadcasting and online video platforms make watching basketball games increasingly accessible. Yet, for new or casual fans, watching basketball videos is often confusing due to their limited basketball knowledge and the lack of accessible, on-demand information to resolve their confusion. To assist casual fans in watching basketball videos, we compared the game-watching behaviors of casual and die-hard fans in a formative study and developed iBall based on the findings. iBall embeds visualizations into basketball videos using a computer vision pipeline, and automatically adapts the visualizations based on the game context and users’ gaze, helping casual fans appreciate basketball games without being overwhelmed. We confirmed the usefulness, usability, and engagement of iBall in a study with 16 casual fans, and further collected feedback from another 8 die-hard fans.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ; ;
Date Published:
Journal Name:
CHI Conference on Human Factors in Computing Systems
Page Range / eLocation ID:
1 to 18
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    Digital media are pervasive in the lives of young people and provide opportunities for them to learn about STEM. Multiple theories argue that the STEM media environment may shape how youth see a STEM career in their future. Yet, little is known about how pre-college digital media consumption may be related to students’ STEM career interest at the beginning of college. The wide variety of STEM media also raises the question of potentially different effects and pathways by media type. In this study, we collected a nationally representative sample of more than 15,000 students in their first year in U.S. colleges and universities. We asked about their career interests at the beginning of college and also asked them to retrospectively report their STEM media consumption during high school.


    We found that watching STEM-related TV and online videos, as well as playing STEM-related video games during high school, were positively associated with students’ STEM career interests at the beginning of college. However, we also found that STEM media consumption did not impact directly on STEM career interest, but acted through two intermediaries: STEM identity (I and others see me as a STEM person) and three personal career outcome expectations: a high interest in self-development (enhancement and use of talents), and low interests in material status (money, fame, power) and in interpersonal relationships (helping, and working with, other people).


    This study finds that STEM media have a significant effect in fostering STEM career interest, with most of the effect coming from STEM TV, STEM video viewing, and STEM video games. The effect is mediated mainly through students’ identity and, to a lesser extent, through personal values, such as self-development, material, and interpersonal relationship values. This study suggests that media communication should be mindful of how different platforms may deliver nuanced and varied messages of what STEM careers may afford and who can succeed in STEM.

    more » « less
  2. Language-guided smart systems can help to design next-generation human-machine interactive applications. The dense text description is one of the research areas where systems learn the semantic knowledge and visual features of each video frame and map them to describe the video's most relevant subjects and events. In this paper, we consider untrimmed sports videos as our case study. Generating dense descriptions in the sports domain to supplement journalistic works without relying on commentators and experts requires more investigation. Motivated by this, we propose an end-to-end automated text-generator, SpecTextor, that learns the semantic features from untrimmed videos of sports games and generates associated descriptive texts. The proposed approach considers the video as a sequence of frames and sequentially generates words. After splitting videos into frames, we use a pre-trained VGG-16 model for feature extraction and encoding the video frames. With these encoded frames, we posit a Long Short-Term Memory (LSTM) based attention-decoder pipeline that leverages soft-attention mechanism to map the semantic features with relevant textual descriptions to generate the explanation of the game. Because developing a comprehensive description of the game warrants training on a set of dense time-stamped captions, we leverage two available public datasets: ActivityNet Captions and Microsoft Video Description. In addition, we utilized two different decoding algorithms: beam search and greedy search and computed two evaluation metrics: BLEU and METEOR scores. 
    more » « less
  3. Badminton is a fast-paced sport that requires a strategic combination of spatial, temporal, and technical tactics. To gain a competitive edge at high-level competitions, badminton professionals frequently analyze match videos to gain insights and develop game strategies. However, the current process for analyzing matches is time-consuming and relies heavily on manual note-taking, due to the lack of automatic data collection and appropriate visualization tools. As a result, there is a gap in effectively analyzing matches and communicating insights among badminton coaches and players. This work proposes an end-to-end immersive match analysis pipeline designed in close collaboration with badminton professionals, including Olympic and national coaches and players. We present VIRD, a VR Bird (i.e., shuttle) immersive analysis tool, that supports interactive badminton game analysis in an immersive environment based on 3D reconstructed game views of the match video. We propose a top-down analytic workflow that allows users to seamlessly move from a high-level match overview to a detailed game view of individual rallies and shots, using situated 3D visualizations and video. We collect 3D spatial and dynamic shot data and player poses with computer vision models and visualize them in VR. Through immersive visualizations, coaches can interactively analyze situated spatial data (player positions, poses, and shot trajectories) with flexible viewpoints while navigating between shots and rallies effectively with embodied interaction. We evaluated the usefulness of VIRD with Olympic and national-level coaches and players in real matches. Results show that immersive analytics supports effective badminton match analysis with reduced context-switching costs and enhances spatial understanding with a high sense of presence. 
    more » « less
  4. Video programs are important, accessible educational resources for young children, especially those from an under-resourced backgrounds. These programs’ potential can be amplified if children are allowed to socially interact with media characters during their video watching. This paper presents the design and empirical investigation of interactive science-focused videos in which the main character, powered by a conversational agent, engaged in contingent conversation with children by asking children questions and providing responsive feedback. We found that children actively interacted with the media character in the conversational videos and their parents spontaneously provided support in the process. We also found that the children who watched the conversational video performed better in the immediate, episode-specific science assessment compared to their peers who watched the broadcast, non-interactive version of the same episode. Several design implications are discussed for using conversational technologies to better support child active learning and parent involvement in video watching. 
    more » « less
  5. Embodied Interaction (EI) offers unique opportunities to uncover novel ways to achieve experiential learning whilst keeping students stimulated and engaged. Spatial abilities have been repeatedly demonstrated as a success predictor for educations and professions in Science, Technology, Engineering and Mathematics. However, many researchers argue that training and assessment of this pertinent reasoning skill is vastly underrepresented in the school curriculum. This paper presents TetRotation, a PhD centred on how affordances coming from Multimodal Analytics can be coupled with EI to nurture Mental Rotation (MR) skills. The overarching objectives of the project are two fold. First, the TetRotation Interaction Design study will highlight best practices identified through the assessment of efficiency, level of engagement and learning gains achieved when using gesture based EI to solve MR tasks. Next, in the TetRotation Game study, these design practices will guide the implementation of an interactive serious game purposed to support the development of MR skills. This research relies on mixed method techniques, including data collections from users' actions, like motion sensing, EEG, gaze tracking, video-recordings, click streams, interviews and surveys. 
    more » « less