skip to main content


Title: A Systematic Mapping Study on Gamification Applications for Undergraduate Cybersecurity Education
Gamification in education presents a number of benefits that can theoretically facilitate higher engagement and motivation among students when learning complex, technical concepts. As an innovative, high-potential educational tool, many educators and researchers are attempting to implement more effective gamification into undergraduate coursework. Cyber Security Operations (CSO) education is no exception. CSO education traditionally requires comprehension of complex concepts requiring a high level of technical and abstract thinking. By properly applying gamification to complex CSO concepts, engagement in students should see an increase. While an increase is expected, no comprehensive study of CSO gamification applications (GA) has yet been undertaken to fully synthesize the use and outcomes of existing implementations. To better understand and explore gamification in CSO education, a deeper analysis of current gamification applications is needed. This research outlines and conducts a methodical, comprehensive literature review using the Systematic Mapping Study process to identify implemented and evaluated GAs in undergraduate CSO education. This research serves as both a comprehensive repository and synthesis of existing GAs in cybersecurity, and as a starting point for further CSO GA research. With such a review, future studies can be undertaken to better understand CSO GAs. A total of 74 papers were discovered which evaluated GAs undergraduate CSO education, through literature published between 2007 and June 2022. Some publications discussed multiple GAs, resulting in a total of 80 undergraduate CSO GAs listing at https://bit.ly/3S260GS. The study outlines each GA identified and provides a short overview of each GA. It also provides a summary of engagement-level characteristics currently exhibited in existing CSO education GAs and discusses common themes and findings discovered in the course of the study.  more » « less
Award ID(s):
1750038
NSF-PAR ID:
10413468
Author(s) / Creator(s):
Date Published:
Journal Name:
Journal on Cybersecurity Education, Research and Practice
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Gamification presents potential benefits in courses that traditionally require the comprehension of complex concepts and a high level of technical and abstract thinking. Courses in Cyber Security Operations (CSO) undergraduate education meet these criterion. This research evaluates organizational constructs that have been applied to gamification applications (GAs) in CSO education. It utilizes framing theory and frame-reflective discourse analysis to outline frames based on engagement levels and analyzes the current distribution of GAs. The following organizational constructs for GAs in data structures and algorithms education apply to CSO education: Enhanced Examination (EE), Visualization of Abstract Ideas (VAI), Dynamic Gamification (DG), Social and Collaborative Engagement (SGE), and Collaborative Gamification Development (CGD). Three additional frames are identified: Missions and Quests (MQ), Simulations (Sim) and Aspirational Learning (AL). MQ GAs have process-driven quests, stories, and/or descriptive scenarios to augment engagement. Sim GAs use environmental immersion to demonstrate real world problem solving while allowing freedom of movement. AL GAs use goal-based designs like Capture The Flag (CTF) missions to enhance engagement. Twenty-seven existing CSO GAs fit within the MQ frame as CSO education lends itself well to these types of experiences. Seventeen CSO GAs fall within the AL GA frame, many of these manifesting as CTF missions. Seventeen CSO GAs fit in the EE Frame due to their optimization in the analysis of learning progress. Nine Sim GAs were successfully deployed in CSO education, followed by 4 VAI, 3 SGE, and 3 DG GAs. 
    more » « less
  2. Computers are used in almost all the fields in our daily life –they are used in various occupations and do the tasks with greater precision and as a result, made the life more comfortable. As such, more than 500,000 computing jobs remain unfulfilled in the US (Reported by app association), and many nations need more computer scientist. Therefore, this urge the need for engineering education community and researchers to focus more on underrepresentation of women in CS due to the fact that women currently comprise only 15.7% of computing degrees awarded ; Computer Science has one of the most considerable gender disparities in science, technology, and engineering and the number of female students choosing computer science as their major remains underrepresented regardless of recent improvements; and the reason behind this statistic is the challenges that lessen students’ motivation in CS majors; Programming courses have always had a negative image among students and usually need more practice. In order to increase the number of female students in CS and ensure the health of the community, there is a need to better understand and discover a mechanism that can improve women’s participation in computer science which leads to attracting more female students in computer science. Researchers have explored various engagements strategies in the fields of computer science. One of the strategies that have seen an increase and garnered attention in the last two decades is the use of video game elements or gamification in different fields such as education. Gamification -which usually refers to using video game mechanics in activities not related to video games - aims to increase participants’ engagement and enjoyment. This notion has been increasing popularity over time especially among especially education researchers because game elements -which provide challenges to the players and motivate them to set goals- can be used in learning environments appropriately to enhance the motivation of learners. While there is a strong body of literature around the implications of gamification on student learning, there are inconsistent results in the literature with regards to the interests or attitudes of women. This review aims to provide a critical evaluation of the use of gamification in the application in the existing literature in 1) education 2) computer science and 3) women in computer science to provide a basis for more targeted learning engagement strategies to motivate and retain more women in computing fields and build on the literature on gamification and gender. 
    more » « less
  3. Major challenges in engineering education include retention of undergraduate engineering students (UESs) and continued engagement after the first year when concepts increase in difficulty. Additionally, employers, as well as ABET, look for students to demonstrate non-technical skills, including the ability to work successfully in groups, the ability to communicate both within and outside their discipline, and the ability to find information that will help them solve problems and contribute to lifelong learning. Teacher education is also facing challenges given the recent incorporation of engineering practices and core ideas into the Next Generation Science Standards (NGSS) and state level standards of learning. To help teachers meet these standards in their classrooms, education courses for preservice teachers (PSTs) must provide resources and opportunities to increase science and engineering knowledge, and the associated pedagogies. To address these challenges, Ed+gineering, an NSF-funded multidisciplinary collaborative service learning project, was implemented into two sets of paired-classes in engineering and education: a 100 level mechanical engineering class (n = 42) and a foundations class in education (n = 17), and a fluid mechanics class in mechanical engineering technology (n = 23) and a science methods class (n = 15). The paired classes collaborated in multidisciplinary teams of 5-8 undergraduate students to plan and teach engineering lessons to local elementary school students. Teams completed a series of previously tested, scaffolded activities to guide their collaboration. Designing and delivering lessons engaged university students in collaborative processes that promoted social learning, including researching and planning, peer mentoring, teaching and receiving feedback, and reflecting and revising their engineering lesson. The research questions examined in this pilot, mixed-methods research study include: (1) How did PSTs’ Ed+gineering experiences influence their engineering and science knowledge?; (2) How did PSTs’ and UESs’ Ed+gineering experiences influence their pedagogical understanding?; and (3) What were PSTs’ and UESs’ overall perceptions of their Ed+gineering experiences? Both quantitative (e.g., Engineering Design Process assessment, Science Content Knowledge assessment) and qualitative (student reflections) data were used to assess knowledge gains and project perceptions following the semester-long intervention. Findings suggest that the PSTs were more aware and comfortable with the engineering field following lesson development and delivery, and often better able to explain particular science/engineering concepts. Both PSTs and UESs, but especially the latter, came to realize the importance of planning and preparing lessons to be taught to an audience. UESs reported greater appreciation for the work of educators. PSTs and UESs expressed how they learned to work in groups with multidisciplinary members—this is a valuable lesson for their respective professional careers. Yearly, the Ed+gineering research team will also request and review student retention reports in their respective programs to assess project impact. 
    more » « less
  4. Previous studies have convincingly shown that traditional, content-centered, and didactic teaching methods are not effective for developing a deep understanding and knowledge transfer. Nor does it adequately address the development of critical problem-solving skills. Active and collaborative instruction, coupled with effective means to encourage student engagement, invariably leads to better student learning outcomes irrespective of academic discipline. Despite these findings, the existing construction engineering programs, for the most part, consist of a series of fragmented courses that mainly focus on procedural skills rather than on the fundamental and conceptual knowledge that helps students become innovative problem-solvers. In addition, these courses are heavily dependent on traditional lecture-based teaching methods focused on well-structured and closed-ended problems that prepare students to plug variables into equations to get the answer. Existing programs rarely offer a systematic approach to allow students to develop a deep understanding of the engineering core concepts and discover systematic solutions for fundamental problems. Without properly understanding these core concepts, contextualized in domain-specific settings, students are not able to develop a holistic view that will help them to recognize the big picture and think outside the box to come up with creative solutions for arising problems. The long history of empirical learning in the field of construction engineering shows the significant potential of cognitive development through direct experience and reflection on what works in particular situations. Of course, the complex nature of the construction industry in the twenty-first century cannot afford an education through trial and error in the real environment. However, recent advances in computer science can help educators develop virtual environments and gamification platforms that allow students to explore various scenarios and learn from their experiences. This study aims to address this need by assessing the effectiveness of guided active exploration in a digital game environment on students’ ability to discover systematic solutions for fundamental problems in construction engineering. To address this objective, through a research project funded by the NSF Division of Engineering Education and Centers (EEC), we designed and developed a scenario-based interactive digital game, called Zebel, to guide students solve fundamental problems in construction scheduling. The proposed gamified pedagogical approach was designed based on the Constructivism learning theory and a framework that consists of six essential elements: (1) modeling; (2) reflection; (3) strategy formation; (4) scaffolded exploration; (5) debriefing; and (6) articulation. We also designed a series of pre- and post-assessment instruments for empirical data collection to assess the effectiveness of the proposed approach. The proposed gamified method was implemented in a graduate-level construction planning and scheduling course. The outcomes indicated that students with no prior knowledge of construction scheduling methods were able to discover systematic solutions for fundamental scheduling problems through their experience with the proposed gamified learning method. 
    more » « less
  5. This research paper introduces a unique system called ZORQ that is a combination of a game development frame- work and a gamification framework (GDGF). The ZORQ GDGF acts as a catalyst to help motivate students by increasing student engagement and success within undergraduate Computer Science (CS) education, regardless of student experience and background. The dynamic gamification elements utilized within the GDGF make it an attractive learning method for students. After col- laborative game space customization, ZORQ gameplay sees each student tasked with designing a ship movement philosophy and then implementing their own code to autonomously control the ship in an interstellar game space filled with supplies, obstacles, and enemy ships. The particulars of engagements between ships can vary greatly by semester, along with the resources/objects present in the game, depending on the collaborative customization and the independent ship strategies implemented. A preliminary Z O R Q trial was conducted over five years in an undergraduate Data Structures and Algorithms (DSA) course. The ZORQ trial is designed to fulfill the following objectives: 1) implement DSA concepts discussed within the course, 2) identify appropriate problem-solving approaches, 3) apply one or more solutions, 4) build depth with a coding language, 5) bridge the gap between limited concept assignments and large, multi-developer software systems by allowing students to build code within a larger architecture, 6) introduce students to version control, 7) illustrate the use of prior mathematics coursework in practical applications, and 8) introduce unit testing in software systems.In exit surveys, students expressed overwhelming satisfaction with this approach. More than 84% of the students surveyed found the system useful in their educational experience and saw benefit to inspecting a completed software project. 82% of the students found that Z O R Q increased software development com- prehension. 80% enjoyed using their own personal creativity in designing a ship controller, 76% found ZORQ helped them learn how to implement and use DSAs. 71% found the system engaging and found the system interaction to be clear and understandable. Observations of student performance in later courses suggest better student maturity and comprehension in preparation for proposing and implementing their own independent projects. 
    more » « less