skip to main content

Title: Ply: A Visual Web Inspector for Learning from Professional Webpages
While many online resources teach basic web development, few are designed to help novices learn the CSS concepts and design patterns experts use to implement complex visual fea- tures. Professional webpages embed these design patterns and could serve as rich learning materials, but their stylesheets are complex and difficult for novices to understand. This paper presents Ply, a CSS inspection tool that helps novices use their visual intuition to make sense of professional webpages. We introduce a new visual relevance testing technique to identify properties that have visual effects on the page, which Ply uses to hide visually irrelevant code and surface unintuitive relation- ships between properties. In user studies, Ply helped novice developers replicate complex web features 50% faster than those using Chrome Developer Tools, and allowed novices to recognize and explain unfamiliar concepts. These results show that visual inspection tools can support learning from complex professional webpages, even for novice developers.
; ; ;
Award ID(s):
Publication Date:
Journal Name:
Symposium on User Interface Software and Technology
Sponsoring Org:
National Science Foundation
More Like this
  1. Agrawal, Garima (Ed.)
    Cybersecurity education is exceptionally challenging as it involves learning the complex attacks; tools and developing critical problem-solving skills to defend the systems. For a student or novice researcher in the cybersecurity domain, there is a need to design an adaptive learning strategy that can break complex tasks and concepts into simple representations. An AI-enabled automated cybersecurity education system can improve cognitive engagement and active learning. Knowledge graphs (KG) provide a visual representation in a graph that can reason and interpret from the underlying data, making them suitable for use in education and interactive learning. However, there are no publicly available datasets for the cybersecurity education domain to build such systems. The data is present as unstructured educational course material, Wiki pages, capture the flag (CTF) writeups, etc. Creating knowledge graphs from unstructured text is challenging without an ontology or annotated dataset. However, data annotation for cybersecurity needs domain experts. To address these gaps, we made three contributions in this paper. First, we propose an ontology for the cybersecurity education domain for students and novice learners. Second, we develop AISecKG, a triple dataset with cybersecurity-related entities and relations as defined by the ontology. This dataset can be used to construct knowledgemore »graphs to teach cybersecurity and promote cognitive learning. It can also be used to build downstream applications like recommendation systems or self-learning question-answering systems for students. The dataset would also help identify malicious named entities and their probable impact. Third, using this dataset, we show a downstream application to extract custom-named entities from texts and educational material on cybersecurity.« less
  2. Abstract How does STEM knowledge learned in school change students’ brains? Using fMRI, we presented photographs of real-world structures to engineering students with classroom-based knowledge and hands-on lab experience, examining how their brain activity differentiated them from their “novice” peers not pursuing engineering degrees. A data-driven MVPA and machine-learning approach revealed that neural response patterns of engineering students were convergent with each other and distinct from novices’ when considering physical forces acting on the structures. Furthermore, informational network analysis demonstrated that the distinct neural response patterns of engineering students reflected relevant concept knowledge: learned categories of mechanical structures. Information about mechanical categories was predominantly represented in bilateral anterior ventral occipitotemporal regions. Importantly, mechanical categories were not explicitly referenced in the experiment, nor does visual similarity between stimuli account for mechanical category distinctions. The results demonstrate how learning abstract STEM concepts in the classroom influences neural representations of objects in the world.
  3. Science simulations are widely used in classrooms to support inquiry-based learning of complex science concepts. These tools typically rely on interactive visual displays to convey relationships. Auditory displays, including verbal description and sonification (non-speech audio), combined with alternative input capabilities, may provide an enhanced experience for learners, particularly learners with visual impairment. We completed semi-structured interviews and usability testing with eight adult learners with visual impairment for two audio-enhanced simulations. We analyzed trends and edge cases in participants' interaction patterns, interpretations, and preferences. Findings include common interaction patterns across simulation use, increased efficiency with second use, and the complementary role that description and sonification play in supporting learning opportunities. We discuss how these control and display layers work to encourage exploration and engagement with science simulations. We conclude with general and specific design takeaways to support the implementation of auditory displays for accessible simulations.
  4. Navigating webpages with screen readers is a challenge even with recent improvements in screen reader technologies and the increased adoption of web standards for accessibility, namely ARIA. ARIA landmarks, an important aspect of ARIA, lets screen reader users access different sections of the webpage quickly, by enabling them to skip over blocks of irrelevant or redundant content. However, these landmarks are sporadically and inconsistently used by web developers, and in many cases, even absent in numerous web pages. Therefore,we propose SaIL, a scalable approach that automatically detects the important sections of a web page, and then injects ARIA landmarks into the corresponding HTML markup to facilitate quick access to these sections. The central concept underlying SaIL is visual saliency, which is determined using a state-of-the-art deep learning model that was trained on gaze-tracking data collected from sighted users in the context of web browsing. We present the findings of a pilot study that demonstrated the potential of SaIL in reducing both the time and effort spent in navigating webpages with screen readers.
  5. Puerto Rico is exposed to multiple hazards including hurricanes, earthquakes, and floods. The Resilient Infrastructure and Sustainability Education Undergraduate Program (RISE-UP) at the University of Puerto Rico aims to introduce students to interdisciplinary problem-solving related to real challenges, especially those associated with the occurrence of natural disasters. The objective of this work is to share our experience with experiential learning related to structural engineering. The lessons learned from this experience, from the student ́s perspective, could encourage faculty members to develop similar undertakings in their programs and students to participate when opportunities arise. During the 2019 fall semester, we enrolled in a course which covered the relationship between design and natural disasters, with an emphasis on rapid response to recover during the aftermath. The course combined lectures and in-class exercises on basic structural analysis, classifications of structures and the use of the FEMA Rapid Visual Screening (P-154) form. This was complemented with field visits of structures affected by Hurricane Maria where we developed several case studies. From December of 2019 to February 2020, Puerto Rico suffered an earthquake swarm reaching magnitudes as high as 6.4, which caused structural damages throughout the South West of the island. Following these events, wemore »were able to use the training acquired during our course in a real-life, post-disaster situation. At the University, we participated in visual inspection brigades, where we aided professional engineers and faculty members in data collection and categorizing building damages. Our involvement helped streamline efforts as we provided additional support in report writing and organization of the data collected using GIS and other tools. The results of the visual inspections indicated that in many cases pre- existing conditions were aggravated by the earthquakes. Furthermore, we also witnessed firsthand the complexities of assessing infrastructure damage during and following high seismic activity. This experience enhanced our awareness of the significance of our profession in ensuring the safety of others both immediately after an earthquake and in the face of future disasters.« less