skip to main content


Title: HackEd: A Pedagogical Analysis of Online Vulnerability Discovery Exercises
Hacking exercises are a common tool for security education, but there is limited investigation of how they teach security concepts and whether they follow pedagogical best practices. This paper enumerates the pedagogical practices of 31 popular online hacking exercises. Specifically, we derive a set of pedagogical dimensions from the general learning sciences and educational literature, tailored to hacking exercises, and review whether and how each exercise implements each pedagogical dimension. In addition, we interview the organizers of 15 exercises to understand challenges and tradeoffs that may occur when choosing whether and how to implement each dimension.We found hacking exercises generally were tailored to students’ prior security experience and support learning by limiting extraneous load and establishing helpful online communities. Conversely, few exercises explicitly provide overarching conceptual structure or direct support for metacognition to help students transfer learned knowledge to new contexts. Immediate and tailored feedback and secure development practice were also uncommon. Additionally, we observed a tradeoff between providing realistic challenges and burdening students with extraneous cognitive load, with benefits and drawbacks at any point on this axis. Based on our results, we make suggestions for exercise improvement and future work to support organizers.  more » « less
Award ID(s):
1801545
PAR ID:
10357738
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
IEEE Symposium on Security and Privacy
Page Range / eLocation ID:
1268 to 1285
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. This paper presents an innovative approach to improve engineering students’ problem-solving skills by implementing think-aloud exercises. Sometimes engineering students claim they do not know where to start with the problem-solving process, or they are not sure how to proceed to the next steps when they get stuck. A systematic training that focuses on the problem-solving process and the justification of each step could help. Think-aloud techniques help make the invisible mental processes visible to learners. Engineering think-aloud technique engages students and helps them make their way through a solving process step-by-step, reasoning along with them. In this study, a multiple faceted systematic approach that integrates think-aloud exercises through video assignments and oral exams were developed and implemented in two pilot engineering classes. We present our think-aloud exercises and oral exams structures in each of the courses and their impacts on students' learning outcomes, and students’ perceptions towards the pedagogical approach. Both quantitative and qualitative results show that the think-aloud exercise assignments and oral exams enhance students’ problem-solving skills and promote learning. 
    more » « less
  2. Chinn, Clark ; Tan, Edna ; Chan, Carol K. ; Kali, Yael (Ed.)
    This paper examines an online professional learning intervention to develop teachers’ pedagogical design capacity to develop five-dimensional (5D) learning and assessment opportunities, which involve integrated use of science and engineering practices, disciplinary core ideas, and crosscutting concepts in science to make sense of phenomena and problems that are interesting to students and support students as knowers, doers, and users of science. We present findings from our design study, which suggest both the promise of such an approach and some of the challenges and tensions experienced by teachers as they chose and used phenomena to support 5D learning opportunities for students. 
    more » « less
  3. null (Ed.)
    Faculty learning communities (FLCs), established prepandemic to disseminate and discuss evidence-based teaching practices as part of an NSF-funded project, Investigating Student Success Using Evidence-Based Strategies-eXpanded (ISSUES-X), proved effective at facilitating learning during a pandemic. As our university made the decision to operate primarily online for the 2020–2021 academic year in response to COVID-19, the nine-facilitator ISSUES-X team supported faculty and their online teaching by offering weekly synchronous open-house video conferences and providing a one-week summer institute. The institute focused on applying principles of How People Learn (NRC, 1999) to the online environment, and intentionally modeled these pedagogical practices. Practices included: online approaches and tools to developing rapport with students; asynchronous reading and assignments sandwiched around synchronous active learning exercises over video conferencing; office hours and other support outside of regular meeting times; and routines for metacognitive reflection. Our pre-existing FLC structure allowed us to support many faculty during a time of crisis, illustrating the value of FLCs as a normative practice in academia. 
    more » « less
  4. Purpose In response to the evolving COVID-19 pandemic, many universities have transitioned to online instruction. With learning promising to be online, at least in part, for the near future, instructors may be thinking of providing online collaborative learning opportunities to their students who are increasingly isolated from their peers because of social distancing guidelines. This paper aims to provide design recommendations for online collaborative project-based learning exercises based on this research in a software engineering course at the university level. Design/methodology/approach Through joint work between learning scientists, course instructors and software engineering practitioners, instructional design best practices of alignment between the context of the learners, the learning objectives, the task and the assessment are actualized in the design of collaborative programming projects for supporting learning. The design, first segments a short real-time collaborative exercise into tasks, each with a problem-solving phase where students participate in collaborative programming, and a reflection phase for reflecting on what they learned in the task. Within these phases, a role-assignment paradigm scaffolds collaboration by assigning groups of four students to four complementary roles that rotate after each task. Findings By aligning each task with granular learning objectives, significant pre- to post-test learning from the exercise as well as each task is observed. Originality/value The roles used in the paradigm discourage divide-and-conquer tendencies often associated with collaborative projects. By requiring students to discuss conflicting ideas to arrive at a consensus implementation, their ideas are made explicit, thus providing opportunities for clarifying misconceptions through discussion and learning from the collaboration. 
    more » « less
  5. Abstract Background

    Elementary educators are increasingly asked to teach engineering design, motivating study of how they learn to teach this discipline. In particular, there is a need to examine how teachers reason about pedagogical situations and dilemmas in engineering—how they draw on their disciplinary understandings, attention to students' thinking, and pedagogical practices to support students' learning.

    Purpose/Hypothesis

    The purpose of our qualitative study was to examine elementary teachers' pedagogical reasoning in an online graduate program. We asked: What stances do teachers take toward learning and teaching engineering design? How do these stances shift over the course of the program?

    Design/Method

    We identified two teachers, Alma and Margaret, who exhibited productive shifts in their pedagogical reasoning during the program. Drawing on interviews and videos of their teaching, we developed case studies characterizing their stances toward teaching and learning engineering.

    Results

    Alma shifted in her reasoning about teaching the design process, from treating it as linear, discrete steps to recognizing the dynamic, overlapping nature of design practices. Similarly, Margaret shifted in how she reasoned about failure and iteration, recognizing the need to help students analyze unexpected design performances to learn from and iterate on their designs. For both teachers, these shifts were dynamic and nonlinear, reflecting both context‐sensitivity and growing stability in their reasoning.

    Conclusions

    Engineering teacher educators should provide opportunities for teachers to reason about the specific pedagogical dilemmas in engineering and consider how teachers integrate disciplinary understandings with attention to students' reasoning and actions and pedagogical practices.

     
    more » « less