Through discussing the inequitable access of BIPOC+ cadets, researchers have found the gap in computer science (CS) enrollment between BIPOC+ cadets and non-BIPOC+ cadets is an access and opportunity gap with "only 35% of schools in which 75%-100% of their student population is from a racial or ethnic group underrepresented in CS offer CS." [1]. When investigating the access to CS courses for cadets within California, researchers found that diverse representation in student identity was lacking. Specifically, the findings indicate that policies are not addressing the need for practices to ensure equitable access, enrollment, and experience, as well as staffing considerations that represent a diversity of individuals [1].
Coinciding with and occasionally compounding the inequities faced by BIPOC+ cadets are the issues facing women in CS. While female cadets accounted for 55% of the AP tests taken in 2018, they only accounted for 28% of AP CS tests taken that year (Bruno). In one study, researchers investigated factors that impact girls' (10-16 years old) career aspirations in computer science [2]. Hur et.al found that, based on the participant's limited experiences and exposure to CS, there were very few aspirations for future careers in CS, and short term experiences like camps weren't enough to shift career aspirations. However, the researchers did find that girls need to build confidence in and become aware of CS-related coursework to pursue more CS-related experiences [3].
In an effort to meet the growing demands of diverse individuals with cybersecurity experience, CSforALL and the Air Force Junior Reserves Officer Training Corps (AF JROTC) collaborated to develop an intervention that brings cybersecurity as well as computer science (CS) to high schools with JROTC programs across the country. This collaboration resulted in the creation of the JROTC-CS Demonstration Project, which launched in February 2020, one month prior to the COVID-19 pandemic shutting down and altering the delivery of education to over 50 millions cadets across the United States.
To date, the JROTC-CS Demonstration Project has engaged with teams of educators at 30 high schools with JROTC programs. The demonstration project was designed to test implementation models for long-term scale-up of evidence-based CS and cybersecurity education programs for cadets (cadets) in the JROTC. At scale, this project has the potential to engage over 500,000 high school cadets in computer science and cybersecurity education pathways, as well as build computing education capacity at over 3,400 JROTC high schools that serve over 4 million cadets.
Our research question for this study was: In what ways does the JROTC-CS experience impact the cognitive (e.g., knowledge and skills) and non-cognitive factors (e.g., social and emotional behaviors) of JROTC cadets?
Established as an aspect of the National Defense Act of 1916, the JROTC program intends to "..provide a quality citizenship, character, and leadership development program, while fostering enduring partnerships and relationships with high schools, educational institutions, and communities that help meet our citizen development mission". An update to the act in 1964 required all military branches to have their own JROTC programs. The Air Force JROTC program (AFJROTC) focuses on Aerospace Science in conjunction with the framework of leadership development and wellness all JROTC programs include. Their curriculum targets areas of study integral to preparing cadets to most effectively serve their community. Currently, computer science skills as well as knowledge of cybersecurity are of particular interest to AFJROTC due to the high demand and low supply of the skill set within both the public and private sectors.
The AFJROTC program is situated in over 3,400 high schools and has over 500,000 JROTC cadets in the U.S. and overseas. Among the cadets, 55% are underrepresented ethnic populations and 40% female. Over 50% of the cadets are located at Title 1 Schools, indicating that they are in communities that support families with low socioeconomic status. Based on inequalities resulting from many historical and structural contexts, novice or historically underrepresented learners in CS may have more barriers when learning or interacting with sophisticated interfaces in web-based educational settings [4]. Thus, given the racial and financial distribution of cadets, it is not surprising that 68% of them do not have access to Advanced Placement (AP) CS courses in their schools.
To meet the goals of addressing the current workforce needs for cybersecurity and computer science specialists, CSforALL and the AFJROTC partnered to create the JROTC-CS Demonstration Project [5]. This Project focuses on a long-term scale-up of evidence-based computer science (CS) and cybersecurity education programs within high schools that have JROTC programs. The Project offers a multi-year pathway to JROTC Cadets in order to earn a badge (an award of recognition) from their JROTC programs (Figure 1), and supports the mission of CSforALL, which is to make high-quality CS education an integral part of the educational experience for all cadets and teachers.
To build capacity for CS and cybersecurity education among the 30 schools invited as part of the Demonstration Project cohort, CSforALL implemented a modified version of their SCRIPT workshop [6]. This workshop provided a strategic way to encourage and develop evidence-based CS course (e.g., AP CS Principles) offerings. Each school had a team of educators (e.g., administrators, teachers, JROTC instructors, and/or guidance counselors) attend the SCRIPT workshop. They received guidance on implementing sustainable practices for AP CSP and cybersecurity learning experiences for their cadets, including recruiting cadets. The school teams then took their plans back to their schools and implemented them. This included starting to offer a foundational AP CSP course in the fall semester, having guidance counselors recruit cadets into the new CS course, and leveraging the JROTC instructor and cadet mentorship to encourage cadets to take the new foundational CS course and participate in CyberPatriot. Over the course of one year, some of the cadets were engaged in CS courses that otherwise they would not have had the opportunity to learn in the prior year, due to the school newly offering the course.
To answer our research question, we used formal qualitative techniques to collect and analyze non-numeric data 1 . created two interview protocols and conducted two sets of interviews. For the first research question, In what ways does the JROTC-CS experience impact the cognitive (e.g., knowledge and skills) and non-cognitive factors (e.g., social and emotional behaviors) of JROTC cadets?, we chose a qualitative study that included a semi-structured interview protocol and used deductive coding with an a priori set of codes established from the interview protocols.
Our research design was approved by an Institutional Review Board.
Cadets were engaged in the intervention for one year at their school, which was part of the JROTC-CS cohort for the first year. They either were engaged in Computer Science or Cybersecurity or both.
We used proportional stratified sampling to help us identify 20 cadets to interview to understand general impacts of the intervention. We used the following criteria for cadets:
• Been engaged in AP CS Principles course and/or Cybersecurity education during the 2020-2021 and/or 2021-22 school year.
• Half from Title I schools • Half will be girls or nonbinary cadets • Half will be underrepresented cadets (Black, Latinx, Indigenous)
• Geographically dispersed across six JROTC-CS schools
We were able to recruit 17 cadets through their JROTC-CS instructors at their schools (Table 1).
Once we identified, we contacted the cadets with an invitation to participate. We sent each cadet a consent form to completed if they were 18 years of age or older. We sent them a consent form for their guardian and/or parent to sign and an assent form for the cadet to sign if they were under the age of 18. Once the appropriate forms were signed, we scheduled the interviews. Each participant received a $100 gift card.
We used a semi-structured interview protocol to conduct interviews using a secure, private Zoom channel, recorded each interview and used a secure transcription service for transcribing. We replaced the names of cadets from interviews prior to sending for transcription and secured the
Table 1: Participant Personal Demographic Data -General Impacts Pseudonym Gender Race/Ethnicity Age Year Years in JROTC CS/Cyber experience Amy F Hispanic 16 Junior 0 AP CSP, Programming Andy M White 16 Soph 2 CyberPatriot (2 yrs) AP CSP, Programming, Cybersecurity Ahmik M Native American 15 Soph 2 Robotics, Cybersecurity, Programming, CyberPatriot (2 yrs) Benny M Mexican/White 16 Soph 2 CyberPatriot (5 yrs), AP CSP Camille F White 18 Junior 4 Web development; Business office specialist Declan M White 16 Soph 2 CyberPatriot (2 yrs), Cybersecurity, Computer Science Donny M Hispanic 18 Senior 4 CyberPatriot (4 yrs), AP CSP, Cybersecurity Gavin M White 18 Senior 4 CyberPatriot (2 yrs), Two CS Courses Grayson M African American 16 Junior 2 CyberPatriot (2 yrs) Jacob M White 16 Soph 2 CyberPatriot (2 yrs), CS I & II, Programming Joshua M White 16 Soph 2 CyberPatriot (1 yr), Two CS courses Jude M White 16 Soph 3 CyberPatriot (2 yrs) Mason M Latino 16 Soph 2 CyberPatriot (2 yrs), IT Fundamentals Michele F Biracial 16 First 1 CS (1.5 yrs), CyberPatriot (1 yr) Rochelle F Asian 15 First 2 CyberPatriot (4 yrs) Seraphine F Hispanic 17 Junior 3 CyberPatriot, Basic CS I & II Tia F Black, White 16 Soph 2 CyberPatriot, Hour of code transcripts on local, password-protected computers. Transcriptions were then uploaded on password protected, two-factor authenticated cloud software (Dedoose) for conducting qualitative analysis [10].
For exploring the impacts of the intervention on cadets, we created an interview protocol that targeted their CS and Cybersecurity knowledge and skills and non-cognitive factors (e.g. social
Table 2: Interview Protocol Themes Theme Related Questions Access Awareness of CS and Cybersecurity offerings at their school as well as if they were encouraged to participate in CS and Cybersecurity offerings Participation Participation in CS and Cybersecurity offerings Learning Learning experience and how they were impacted by the intervention as well as the pandemic Engagement Learning engagement and how they were impacted by the intervention as well as the pandemic Other noncognitive factors Interest in CS/Cybersecurity, belongingness, perceived abilities in CS and Cybersecurity, relevance of technology, confidence using technology, and future plans for participating in CS/Cybersecurity offerings, all in the context of how they were impacted by the pandemic AP exam Participation in AP Exams, particularly AP CSP and AP CS A
and emotional behaviors) based on their experiences in CyberPatriot and CS courses. We developed the protocols based on a Theory of Impacts we developed for student experiences [11].
The 17 interviews lasted between 24 and 50 minutes, with a mean average of 30.5 minutes.
We analyzed the data using deductive coding by first developing a set of codes a priori for each of the themes. We used the Framework Method for analyzing our data [12]. Two researchers coded the first half of one interview together, then conducted their analysis asynchronously using Dedoose. We ensured for interrater reliability by checking and discussing codes that were coded differently and came to an agreement of how they should be coded. We then coded all other interviews independently and applied a cross-check on each others' code to ensure consistency across the codes. As we synthesized the data in the narrative, we also paid attention to how the excerpts from the codes aligned.
One of the researchers has been formally trained in quantitative, qualitative, and mixed methods education research methods and has extensive experience working as a CSEd researcher at the primary, secondary, and post-secondary levels. This co-author has worked closely in studying inequities within interventions and how these inequities can be addressed. The perspective that this co-author brings to this study is one of respect for qualitative methodologies for understanding the personal experiences of cadets, while interpreting the data in ways in which action can be taken. The other co-author brings an outsider, liberal arts perspective to the research. They are interested in equity initiatives.
A total of 17 cadets participated in a post-intervention interview that explored their experiences in CS and Cybersecurity courses and extra-curricular activities. This interview did not delve into the effects of Covid-19 on cadets' learning experience; however, responses may have been influenced by the changes in instruction caused by school responses to the pandemic or by more general impacts of the pandemic.
The cadets interviewed interacted with several different aspects of CS through their course work and many also participated and/or competed in CyberPatriot. CS course offerings varied based on the school, although Python and web development were mentioned by most cadets. Joshua, Amy, and Rochelle mentioned their CS courses included some game design instruction. Additionally, Benny was enthused about his cyber business course in which, "We were given a budget and we were planning a trip and so we had to calculate what we needed, all the places that we were stopping, and how much money that it was going to take." cadets often remarked on the ability of their CS courses to assist in their exploration of different topics and potential future opportunities. Andy in particular shared, "What I like most about them is just the thrill of discovery. Being able to learn something new and then put it into motion and see your work run and do stuff. That part is really satisfying."
Depending on instructor and school resources, cadets were also able to interact with professionals in the CS field. Joshua recalls his class had occasionally "gotten on a Zoom call with ethical hackers and webpage designers." Ahmik's school provided even more as he shared, "we've even had an adopt a school program that brought out three huge companies to come talk to us all as a group." However, Amy's courses did not have the same degree for career learning and she said, "...I wish we had more of an insight of what it would look like when we are actually doing our job in the future."
Cadets were asked if they were satisfied with their decision to take CS and Cybersecurity courses and extracurricular courses. Aside from one person, cadets reported positive impacts on their experiences with a few common reasons. Several of the cadets (Mason, Jude, and Camille) found their CS and Cybersecurity activities fun and valued their experiences for it. Others (Donny, Joshua, Ahmik, Jacob, and Benny) specifically related their enjoyment to the community they were able to build and interact with as well as the value they placed on their teammates and the work they did together. Three cadets (Declan, Gavin, and Andy) were appreciative of the training they gained, since they believe it will assist them with their future jobs, or with making major decisions regarding their career goals. Finally, four cadets (Tia, Amy, Seraphine, Rochelle, and Grayson) were all grateful for the chance to learn more about technology and the people who work with it.
Michele was the only student who was not particularly happy with their experience. When asked if she was glad to have taken CS courses she responded that she "didn't really have much else to enroll in" and she did not particularly enjoy the subject matter.
We inquired as to the role JROTC and course instructors played in supporting cadets' CS and CyberPatriot education.
Cadets reported feeling supported by their JROTC instructors in several different capacities. Ahmik shared that his instructor, "He was there watching us at every practice that we could make it till four o'clock so we wouldn't have to leave early. . . He even went out of his way to find us, I think, three different instructors or mentors in the same year so we could keep on furthering the team..." Gavin's instructor "brought in people that were to help us because he couldn't help us on his own." Mason shared "whenever the competition is or if there's not going to be, she makes sure we have a room that we can actually like prepare as a team, in practices, makes sure we have guest speakers to tell us what's happening." Jude commented, "He's very supportive of us. He's monitoring us while we're there. He supplies us with food. He's really funny. If we're ever stressed out, he'll tell us a good joke and he'll remind us of different things.... He does all the things he should be as an instructor, as a coach, and he's a great guy." Other cadets noted receiving constructive criticism and helpful reminders from their instructors.
As with all educational opportunities, cadets' experiences with CS courses directly depended on their instructors' ability and interactivity. Aside from the usual classroom support, some cadets reported their teachers going above and beyond to provide them with additional assistance. After school, Rochelle's instructor stayed behind to assist with exam preparation. Ahmik portrayed his instructor as particularly influential, saying he "has given me endless resources, taking me to events like the Stemies. . . . many networking events that have been able to help me get resources so I can further myself in my job career. He's given me countless links and websites to different places where I can find out what jobs will look like in cybersecurity and what opportunities there are out there. I mean, I had no idea that there was computers certifications, and more than one at that. Here he is giving me all the material that I'll need to know so I can study Security+, [and attending the Cyber Academy] I didn't even know existed."
Seraphine's main support was not the instructor of her CS course, but the teacher in charge of her remote computer lab. She explained, "I think she gave me a lot of support....I felt like she was very supportive and she was probably the main reason that I kept pushing to finish the class was because I knew she was working hard for me. I needed to work hard for her." Other cadets remarked on the straightforwardness of the feedback their instructors gave as well as their willingness to support cadets finding answers and information on their own or with necessary guidance.
In response to researchers inquiring as to her ability to apply what she learned in her CS courses to her real life, Amy recalled, "[M]y brother...came to me and asked me to make him a website..., so I was like, 'Okay, yes, I could do that.'" Gavin and Andy replied by mentioning building their own computers with Andy additionally noting, "The fact that I'm able to understand the connectors, what they do, be able to set up systems, install that, and the fact that I am one of the go-to people, former of my friends had the tech support problem or something like that. I apply it all the time."
A student's ability to apply skills to real-life situations is a worthy outcome in itself, however, the JROTC-CS Project seeks to do more than technically literate citizens, it intends to build a diverse new generation of computer scientists. To do so, courses need to build confidence in overall CS knowledge and ability. To gauge the program's current efficacy in doing just that, researchers asked cadets to rate their CS ability from 1-10. Most cadets rated their CS ability at 6-7 due to still having much to learn. However, there were a few outliers. Seraphine placed herself at 4-4.5 because it had been a while since she had taken CS courses and was concerned that "technology progresses really fast. If you don't keep with it, it's not going to still be there." Jacob also placed himself at a four and said "I still have much, I need to focus a lot more on AP Computer Science." On the further end, Jude placed himself at an 8.
We also asked cadets how their CS abilities and knowledge compared to that of their peers. Among the responses, there was a stratification of cadets placing themselves above, at, and below their peers with a preference for above. The majority of cadets ranked themselves above or on par with their peers with only Jacob placing himself below.
Self-Assessed Knowledge CyberPatriot similarly had an impact on student's applicable knowledge outside of the program. cadets noted that the program had them thinking more critically about several aspects of device, network, and site security. Rochelle took into consideration what she learned about hacking and understands the value of Virtual Private Networks. Ahmik finds himself consistently, "checking what port my apps are running on or if I'm tracking MAC addresses to see if my computer's functioning correctly." Similarly, Jude installed a firewall for his mother.
When asked to rate their cybersecurity skills on a 1-10 scale, answers varied more than with CS.
Joshua rates himself at a 6/7 due to a lack of ability to a "lack of being able to go to the practices, and competitions and stuff for Cybersecurity." Ahmik, while considering himself to be the most skilled member of his CyberPatriot team, rates himself an 8.5 because his financial situation causes him to "rely on grants and scholarships and the resources of my teachers." Gavin and Grayson, rating themselves at a five and four respectively, find they had a mixed bag of ability in regards to CyberSecurity, doing well in some areas and less so in others. Andy placed himself at an 8, citing his experience and enthusiasm in the topic as a hobby. Jude also placed himself at an 8. Mason rated himself at a 6/7 stating, "I wouldn't say I'm like the sharpest pencil, but I wouldn't say I'm a novice." Jacob placed himself at a seven for similar reasons to Mason.
Rochelle placed herself at a 7 on account of how much her knowledge has increased. Seraphine rated herself the lowest at 2 because she, "just wouldn't know how to use it and when."
Compared to responses on CS courses, more cadets considered their Cybersecurity skills to be below their peers. Cadets like Jude and Mason place themselves on the same level as their team due to feeling like everyone has a specialty. Rochelle and Joshua ranked themselves below their peers with Rochelle citing a lack of experience compared to others.
cadets were asked if they felt like they belonged in their CS and Cybersecurity courses as well as if they felt like aspects of the demographic (i.e. race, gender, socioeconomic status) had an effect on their experiences in those courses. As white males comprise the majority of cadets and instructors involved in CS and Cybersecurity, cadets identifying as such consistently felt they belonged and that their demographic did not have a bearing on their experiences with belonging in CS and Cybersecurity.
Minority and female cadets had a broader spread of experiences with belonging. Seraphine in particular had a difficult time in her courses on account of her gender.
In both the CyberPatriot and in my courses, I was the only girl. That was a little different because it was just guys around me and there was no other girls there. It felt like I had to do more and especially in my courses, sometimes I would submit work that was nearly identical to everyone else's because it would be a formula and I would receive fewer points. I wouldn't really know what I had done wrong, but sometimes it seemed like it was a challenge just because I was a girl and there was no other girls there to help. Guys worked with guys and that was just me....It may not have felt like I belonged in my course, just because I was secluded and I had to work on my own obviously, but I do feel like with how hard I worked and how much I pushed to achieve, I was the top of that class. Even if I didn't belong, I made it where I belong.
Yet, the male cadets that attended her same school did not note any such knowledge of inequity, with one even stating he believed the CS experience to be, "pretty equal across the board for everyone." Amy, while not noticing any direct exclusion, noted that "there's not a lot of women in the field of computer science and cybersecurity so it does make it a little bit more challenging because that just means that I have to be able to know the things, know everything. I have to be able to prove myself pretty much." Rochelle and Camille felt more positive about their place in their CS courses. Rochelle also found herself to be the only girl in her CS classes; however, she stated that "The teacher was new to computer science, but she was a female, so I felt comfortable. She was new too and I was new too, so it made me feel comfortable. . . " Camille began her course feeling out of place because her skills and understanding weren't at the same level as other cadets, but she noted "I had a teacher who was more than willing to sit with me and make Male cadets from underrepresented demographics had similarly mixed experiences. Mason did not feel his demographics had an impact on his belonging while Benny stated, "I think the fact that I am Mexican makes it a little bit hard because sometimes when I tell people outside of school that I'm a part of CyberPatriot, they make fun of me for that. They're like, 'Oh, we're not supposed to do that. You're Mexican, blah, blah, blah.' I think that's definitely hard on me." Donny attributed his initial difficulty fitting in to his moving half way through the school year when dynamics would have already been established. In reference to his race, Grayson said "I don't know. Honestly, I think it had a minor effect. . . For me, it's important, but it's not the end-all-be-all or anything. It's just another factor in my journey within cybersecurity. I guess you could say that it has had its effect." And Ahmik said, "I think my financial bracket had hindered me at one point because I just wasn't doing very well financially. My family, I have a single father and we live off of social security, but at the same time, I ended up turning that into a driving factor."
As mentioned above, CyberPatriot is a team-centric program that requires close collaboration between the involved cadets. This environment of collaboration affected the cadet's sense of belonging. Seraphine, whose issues with CS classes we mentioned, said in comparison to those classes, "I felt more welcome in CyberPatriot. . . " Donny felt that "in CyberPatriot, we don't really care about religion. We don't think about any of that, so identity is not really a thing. We're all equal in a way. We're just all trying to work in the same process, figuring out the same thing." Grayson mirrors that sentiment saying, " I think in the CyberPatriot it matters less than in my other classes. I feel like, in CyberPatriot, they don't care. As long as you know what you're doing with the computers, that's all that matters to them."
Aside from Amy and Seraphine, all cadets surveyed have intentions to continue their CS and or cybersecurity education. Seraphine would only like to continue with CS in a hobby capacity with "a club or just a side activity for people that aren't super, super dedicated and they don't want their entire lives to be Cybersecurity or computer science, but they want to be able to understand it," while Amy stated, "To be honest, I am not really interested just because I don't really like making games."
We asked the cadets about their goals and whether or not their CS and Cybersecurity experiences had an impact on what they wanted to do in the future. Rochelle appreciated her CS experience as eye-opening and that reminded her "That's not what I just have to do. I could do other stuff as well. I don't have to just do one thing." Tia "wanted to do something in cybersecurity, but then when I got through these courses and things like that, I was like, 'Okay, you can't just be that generic.' I found out that there's field engineers, there's technicians, and all these different things. I got a more in-depth explanation of what people do inside of that generic career field." Camille had a similar experience with her instructor constantly bringing in people and resources to help the cadets explore their future career options with CS and cybersecurity. Grayson noted "It has got me interested in the field because originally I didn't really have a plan for going into college. I wasn't really sure what I wanted to do in college. I never really thought about-Well, I thought about a major, but it was aerospace engineering, but I then switch to computer science because that just seemed more appealing to me. It was something that I was good at, so I thought, 'Hey, why not go into the cybersecurity field?'" While many cadets found this exploration of CS to be influential and worthwhile, cadets like Gavin and Camille contemplated their choices and found their interests to lie elsewhere.
We then inquired about their college and occupational goals, resulting in a variety of responses. Gavin intends to study psychology and Camille is looking into linguistics. While Grayson has vague notions of entering the cybersecurity field, Donny had already been accepted into college as a network security major. Tia gave two options, "If I can get into the Air Force Academy, I want to go do Cybersecurity, but if I don't, and I get into the Naval Academy, then I want to go into nuclear engineering." Jacob wants to pursue flight with either the Air National Guard or the Air Force. Michelle wants to study biomedical engineering from a Cybersecurity perspective as "I'm interested in the cyber aspect, because again, one of the main problems, at least what I think with biomedical engineering, is how things run, how they're coded, and such. I mean like just think about if you are in a hospital and you just have this loud beeping alarm, because it thinks there's something wrong when in reality there isn't, it just thinks, 'Okay, this number is too high or too low or whatever and therefore, I must trigger an alarm,' when in reality it needs a bit more intelligence."
Benny expressed his desire to become an aerospace engineer. Ahmik laid out several plans, "First of which is going into the Air Force Academy because I wanted to be able to have a steady functioning job where I knew I'd be able to get a job coming out of the air force and maybe work long enough to earn a pension. Then if that wouldn't work, I was also thinking of going for the J-100 ROTC scholarship, so that I'd be able to get a full ride at a school with an ROTC core, for the same reasons as getting into the Air Force Academy or just a regular college that I could get into the field of cyber with." Joshua wants to go into CS and game design with the goal of becoming a game designer and, similarly, Andy intends to become a computer programmer (like his father).
Interviewing 17 cadets across a variety of schools' socio-economic status, geographic locale, and demographic location provides a robust viewpoint from each student. However, this qualitative study has its limits and we caution to use the data as intended-to gain insight into the impacts of the intervention directly on a set of cadets and to potentially form hypothesis that can be tested further. While our study ended prior to being able to investigate the hypothesis through a quantitative study, the findings generally seem to indicate that overall the intervention had a positive impact on the cadets.
Overall, the quality and quantity of course offerings varied mostly based on school resources. This is to be expected since it is a well-established fact that in the United States, the quality of education and the variety of courses offered depends on the resources available to the school [13].
Further, all but one student enjoyed CS and cybersecurity and the reasons for this varied. This is a positive outcome of the intervention and may be due to several components of the intervention, such as teacher PD and the involvement of the JROTC instructors in supporting the cadets.
In this section, we consider the commonalities within each of the categories.
Some cadets, while often engaging in similar learning experiences, did not have access to the same courses and resources as others did. This is unsurprising, given the level of inequity that exists in K-12 districts, schools and classrooms [13,14]-inequities that include computer science classrooms [15]. This lack of access to courses and resources led to less fulfilling experiences as particularly noted by two of the cadets.
There is a known link between a lack of resources in schools and lower academic achievement among cadets [16]. Without proper equipment, including Internet access, cadets at schools are not receiving the same learning opportunities as cadets in other schools. It is known that teachers can help to fill the gaps in their underresourced schools [17]. This also occurred within this study. The resources available as well as access to information on outside resources were heavily dependent on individual instructors and their ability and degree of dedication.
As part of their learning experiences, some of the cadets were able to speak to and learn directly from CS professionals about careers, with one mentioning the Adopt A School program instituted by the JROTC-CS program. This provided cadets with first-hand knowledge about jobs in the field, which has been shown to be an effective practice in raising career awareness [18,19].
As CyberPatriot is a competition with standardized curricula, the content experiences were more uniform than with CS courses. Central to CyberPatriot is the Youth Cyber Defense Competition.
Due to the focus on competition, cadets build their well of knowledge as a team, rather than individuals looking for a grade. This was highlighted in cadets' recollections of their CyberPatriot experiences. Further, CyberPatriot has been shown to have impacts on students' career awareness and learning [20], and cadets we interviewed also shared their enjoyment of and learning from participating in CyberPatriot.
Integral to a student's success and growth is the support they receive from educators and other mentor figures. Overall, the cadets felt supported by their JROTC instructor and finding ways that would enable their success, including coaching them, mentoring them, and providing support in ways that can build camaraderie. Leveraging this connection between JROTC instructors and cadets was one of the key benefits from the JROTC-CS project.
The goal of any educational opportunity ought to be to instill knowledge and skills that cadets can take and extend into their everyday lives. The ability of cadets to take what they have learned and apply that knowledge to their lives can be a strong signifier of successful learning and is referenced through Bloom's taxonomy (as it shows application, understanding, and remembrance) [21]. When it comes to CS, most cadets had a similar level of confidence in their ability and knowledge on the topic. Common reasoning, regardless of the exact self-score, surrounded the feeling that there was much more to learn and that their rating was more than sufficient for that specific point in their education.
We cautiously correlate self-assessed knowledge questions to self-efficacy, though we are aware of hte limitations of a two-item self-rating question. Self-efficacy is highly-correlated with academic achievement [22] as well as choosing to study CS [23]. As such, our questions revealed a variety of levels. However, the majority of students who rated themselves were in the higher range (6 to 7) on the 10 point scale, with only one rating themselves below 5. Similarly, they rated themselves on average greater than their peers, which also providers a marker of self-efficacy in CS.
Cybersecurity self-efficacy ranged more than with CS, and this may have had to do with the team-based nature that allows cadets to more directly compare themselves to their peers. Student's like Ahmik who take on leadership roles within their team are bound to feel more confident in their ability when their position insinuates a greater understanding than others while cadets like Seraphine could understandably feel less confident when they feel like "not one of the central members" of their team.
Two items of interest surfaced from the interviews. The first was the fact that financial insecurity played a part in Ahmik's lowering his score on this item. For youth transitioning to adult hood, previous research has found that both unemployment and parental financial intervention and support (e.g., not letting students gain financial balance in their lives independently) have negative consequences for youth's self-efficacy [24]. The second items was the ratings for cybersecurity were much lower than CS. This could be due to the fact that more students in our study took CS rather than cybersecurity. It could also be due to the fact that their CS course may not have pushed them as much as the CyberPatriot cybersecurity competitions.
Answers in regard to cadets' sense of belonging understandably varied along race and gender lines, in line with a plethora of research on this topic within computer science and cybersecurity [25][26][27]. With respect to CS, minority and girls had a broader spread of experiences with belonging when compared to their white, boy peers. While the white boys consistently felt a sense of belonging, the other students were quick to share their varied and negative experiences.
However, it must be noted that responses to belonging in CS courses versus CyberPatriot were very different. Several minority and female cadets responded that they felt like they belonged more in their CyberPatriot classes and extracurriculars than that of their CS ones. Thus, it seems a team platform may have a positive effect in this regard.
The ultimate goal of the JROTC and CSforALL collaboration is to influence cadets into developing an interest in CS and Cybersecurity and possibly entering those fields in the future. Given the interest in continuing CS/cybersecurity education that the majority of cadets expressed, value in the intervention is evident. However, we also note that cadet interest may have led them to take a CS course or join Cybersecurity, so it is difficult to state anything more than correlation.
Additionally, most of the cadets interviewed designated desired careers within the CS/Cybersecurity field. Some cadets like Andy already had an interest in the topics and thus were already primed to enter the field regardless of the intervention, but Grayson and others developed their goals based on what they have been able to learn through the intervention.
Despite this intervention happening during the pandemic when learning modalities changed significantly and both teachers and cadets faced challenges that impacted the delivery and receiving of content, the results show that the intervention had an impact on cadets. Some of the cadets were given new opportunities to learn CS and cybersecurity, and all but one of the 17 interviewees responded positively to their learning experiences. The results further highlighted the age-old problem of differences in education received by cadets in underresourced schools-a problem that is known to impact historically marginalized groups more. While this is a problem woven into the fabric of the funding for education in the U.S., this intervention defied this in some small ways by leading to additional cadets receiving education through its engagement with teams at schools that engaged in efforts to bring CS and cybersecurity to their students.
We adhere to the qualitative methodology, which brings voice to all participants. As such, and in line with formal qualitative methods, we do not quantify our qualitative data. Instead, we group similar data that we found supports categories to help provide insight and in-depth knowledge of the subject. It is a known misconception that qualitative data should be quantified. One novel response could provide critical insight that ten similar responses may not yield, bringing new understanding into the subject field of study[7][8][9]