skip to main content


Title: Improving Research and Experience Reports of Pre-College Computing Activities: A Gap Analysis
This paper provides a detailed examination of pre-college computing activities as reported in three Association of Computing Machinery (ACM) venues (2012-2016). Ninety-two articles describing informal learning activities were reviewed for 24 program elements (i.e., activity components, and student/instructor demographics). These 24 program elements were defined and shaped by a virtual focus group study and the articles themselves. Results indicate that the majority of authors adequately report age/grade levels of participants, number of participants, the type of activity, when the activity was offered, the tools/languages used in the activity, and whether the activity was required or elective. However, there is a deficiency in reporting many other important and foundational program elements, including contact hours of activity participants, clear learning objectives, the prior experience of participants (students and instructors), and many more. In conjunction with previous work, this paper provides recommendations to reduce these deficiencies. The Recommendations for Reporting Pre-College Computing Activities (Version 1.0) are presented to help researchers improve the quality of papers, set a standard of necessary data needed to replicate studies, and provide a basis for comparing activities and activity outcomes across multiple studies and experiences.  more » « less
Award ID(s):
1757402
NSF-PAR ID:
10058207
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
SIGCSE '18 Proceedings of the 49th ACM Technical Symposium on Computer Science Education
Page Range / eLocation ID:
964 to 969
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. This paper provides a detailed examination of pre-college computing activities as reported in three Association of Computing Machinery (ACM) venues (2012-2016). Ninety-two articles describing informal learning activities were reviewed for 24 program elements (i.e., activity components, and student/instructor demographics). These 24 program elements were defined and shaped by a virtual focus group study and the articles themselves. Results indicate that the majority of authors adequately report age/grade levels of participants, number of participants, the type of activity, when the activity was offered, the tools/languages used in the activity, and whether the activity was required or elective. However, there is a deficiency in reporting many other important and foundational program elements, including contact hours of activity participants, clear learning objectives, the prior experience of participants (students and instructors), and many more. In conjunction with previous work, this paper provides recommendations to reduce these deficiencies. The Recommendations for Reporting Pre-College Computing Activities (Version 1.0) are presented to help researchers improve the quality of papers, set a standard of necessary data needed to replicate studies, and provide a basis for comparing activities and activity outcomes across multiple studies and experiences. 
    more » « less
  2. There has been considerable investment in pre-college educational interventions for all areas of STEM (including computer science). The goal of many of these initiatives is to engage and interest students early in their educational career. In this study, a systematic literature review was undertaken to determine the demographic and program data collected and reported for the field of computing education and for other STEM disciplines for activities that were not designed as part of the formal in-class curriculum (e.g., outreach activities). A comparison-contrast analysis of the resulting 342 articles found similarities and key differences in the reporting of this data as well as overarching characteristics of missing or incomplete reporting across disciplines. Authors from both fields reported equally well in the four categories studied: information about evaluation, participant gender, participant race and/or ethnicity, and activity demographics. However, the computing education articles were more likely to have clearly stated research questions and comparative analysis based on demographic characteristics. They were less likely to include the number of participants in the study, participant age/grade level, socioeconomic status, disability information, location of intervention, and instructor demographics. Through this analysis, it was determined that reporting can be improved across all disciplines to improve the quantity of data needed to replicate studies and to provide complete data sets that provide for the comparison of collected data. 
    more » « less
  3. The Improving Student Experiences to Increase Student Engagement (ISE-2) grant was awarded to Texas A&M University by the National Science Foundation, through EEC-Engineering Diversity Activities (Grant No. 1648016) with the goal of increasing student engagement and retention in the College of Engineering. The major component of the intervention was a faculty development program aimed to increase active learning, improve classroom climates, and decrease implicit bias and deficit thinking. Faculty teaching first- and second-year Engineering courses participated in the ISE-2 faculty development program, with the first cohort (n = 10) in Summer 2017 and the second cohort (n = 5) in Summer 2018. This paper describes the content of each of these components of the faculty development program and provides access to a Google drive (still in development at the time of the abstract) with resources for others to use. The faculty development program consisted of three workshops, a series of coffee hour conversations, and two deliverables from the participants (a teaching plan at the conclusion of the summer training and a final reflection a year following the training). Anchoring the program was a framework for teaching in a diverse classroom (Adams & Love, 2009). Workshop 1 (early May) consisted of an overview of the ISE-2 program. During the first workshop, faculty were introduced to social cognitive biases and the behaviors that result from these biases. During this workshop, the ISE-2 team shared findings from a climate study related to the classroom experiences of students at the College of Engineering. Workshop 2 (mid-May) focused on how undergraduate students learn, provided evidence for the effectiveness of active learning strategies, and exposed faculty participants to active learning strategies. Workshop 3 (early August) integrated the material from the first two workshops as faculty participants prepared to apply the material to their own teaching. Prior to each workshop, the faculty participants were provided with pre-workshop readings to familiarize them with some of the content matter. Coffee hour conversations—informal discussions between the participating faculty and the ISE-2 team centered around a teaching topic selected by participants—were conducted on a near-weekly basis between the second and third workshops. Handouts and worksheets were provided at each coffee hour and served to guide the coffee hour discussions. After the last workshop but before the Fall semester, faculty participants created a teaching plan to incorporate what they learned in the ISE-2 program into their own teaching. At the end of the academic year, the faculty participants are tasked with completing a final reflection on how ISE-2 has affected their teaching in the previous academic year. In this paper, we will report the content of each of the three workshops and explain how these workshops are related to the overarching goals of the ISE-2 program. Then, we will discuss how each of the coffee hour conversation topics complement the material covered in the workshops. Lastly, we will explore the role of the teaching plans and final reflections in changing instructional practices for faculty. 
    more » « less
  4. The Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) program, managed by the U.S. National Science Foundation (NSF), provides grants to institutions of higher education to disburse scholarships for low-income, high-achieving domestic students enrolled in a STEM major. Despite the crucial role that two-year colleges (2YCs) epitomize in providing open-access affordable education to a diverse student population, the majority of NSF S-STEM scholarships are awarded to four-year institutions, which tend to have specialized personnel working on the preparation and submission of proposals. In this paper, we report a summary of the activities and evaluation of a "Capacity Building Workshops for Competitive S-STEM Proposals from Two-Year Colleges in the Western U.S.", funded by the NSF S-STEM program, aiming to facilitate submissions to the NSF S-STEM program from two-year colleges (2YCs). The workshop was offered in 2019 (in person) and in 2020 and 2021 (virtual), initially to support 2YCs in the Western region of the US and was expanded nationwide in 2020. During participation in the two-day workshop, several aspects of proposal submission were reviewed, in particular, the two NSF Merit Review Criteria of Intellectual Merit and Broader Impacts. Pre- and post- workshop support was also available via virtual office hours and webinars that addressed specific elements required to be included in S-STEM proposals. The evaluation of the workshop has been performed via post-workshop survey administered through Qualtrics™. A journal paper reporting on the evaluation of all three offerings of the workshop has been submitted and currently in review. In this paper, we intend to reflect on the successful features of this workshop series and the lessons learned throughout the three offerings. Over three years, 2019, 2020 and 2021, the program supported 103 participants on 51 teams from 2YCs. The program assisted at least 31 2YCs submit their S-STEM proposals to NSF, and 12 of these 2YCs received S-STEM grants. An additional 2YC proposal was first recommended for an award, but the proposal was subsequently declined for reasons unconnected to the content of proposal itself. The 3-year funding rate is 39%; if the above-mentioned proposal that received an award recommendation but was then declined is taken into account, the award rate is 42%. 
    more » « less
  5. This paper summarizes the overall approach and assessment of a National Science Foundation Research Experience for Undergraduates Site focused on sustainable civil and environmental infrastructure in rural areas. This site has hosted over 60 students over 5 years, including 1 year of virtual participation due to travel restrictions associated with the COVID-19 pandemic. Detailed discussion and results are provided with respect to the recruitment approach, including particular attention to first-generation college students, and the potential negative impacts of the COVID-19 pandemic on first-generation applicants. This site also incorporates targeted instruction on technical writing, which occurs over several weeks throughout the first half of the summer and culminates with a final conference paper deliverable. This approach has yielded over 20 peer-reviewed journal articles, conference papers, or national conference presentations, which have been co-authored by the undergraduate student participants. External evaluation of this site has included both formative and summative assessments, including pre-program, mid-program, and post-program surveys and focus groups, which has enabled a successful continuous improvement cycle, in which cohort-building activities, technical writing deliverables, and mentor training have been gradually incorporated or enhanced. Results of this assessment have also been used to quantify the site’s success with respect to student exposure and interest in research and graduate education. In addition to most participants persisting in STEM fields, many have gone on to pursue graduate school in civil and environmental engineering and win national fellowships. 
    more » « less