skip to main content

Attention:

The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, May 23 until 2:00 AM ET on Friday, May 24 due to maintenance. We apologize for the inconvenience.


Title: A Systematic Review Exploring the Differences in Reported Data for Pre-College Educational Activities for Computer Science, Engineering, and Other STEM Disciplines
There has been considerable investment in pre-college educational interventions for all areas of STEM (including computer science). The goal of many of these initiatives is to engage and interest students early in their educational career. In this study, a systematic literature review was undertaken to determine the demographic and program data collected and reported for the field of computing education and for other STEM disciplines for activities that were not designed as part of the formal in-class curriculum (e.g., outreach activities). A comparison-contrast analysis of the resulting 342 articles found similarities and key differences in the reporting of this data as well as overarching characteristics of missing or incomplete reporting across disciplines. Authors from both fields reported equally well in the four categories studied: information about evaluation, participant gender, participant race and/or ethnicity, and activity demographics. However, the computing education articles were more likely to have clearly stated research questions and comparative analysis based on demographic characteristics. They were less likely to include the number of participants in the study, participant age/grade level, socioeconomic status, disability information, location of intervention, and instructor demographics. Through this analysis, it was determined that reporting can be improved across all disciplines to improve the quantity of data needed to replicate studies and to provide complete data sets that provide for the comparison of collected data.  more » « less
Award ID(s):
1933671 1625335
NSF-PAR ID:
10099397
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Education Sciences
Volume:
9
Issue:
2
ISSN:
2227-7102
Page Range / eLocation ID:
69
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background Billions of dollars are spent annually on grant-funded STEM (science, technology, engineering, and mathematics) education programs. These programs help students stay on track toward STEM careers when standard educational practices do not adequately prepare them for these careers. It is important to know that reliable and accurate student participation and completion data are being collected about these programs. This multiple case study investigates how student data are collected and reported for a national STEM education program in the United States, the National Science Foundation (NSF) Advanced Technological Education (ATE) program. Our overall aim is to provide insights to funding agencies, STEM education faculty, and others who are interested in addressing issues related to the collection and reporting of student participation and completion data within their own contexts. Emphasis is placed on the barriers encountered in collecting participation and completion data, particularly with regard to unduplicated participation counts and marketable credential data. The ATE program was selected for this study because there is already a mechanism (known as the ATE Survey) in place for annually collecting systematic data across all projects within the program. Results A multiple case study, including interviews of primary investigators, allowed for in-depth analysis of the ATE Survey’s point-in-time data on project-level participation in various activities, and for identification of the following barriers to tracking student-level data: lack of time and help to gather these data, lack of a consistent system for tracking students across different institutions, and a perceived lack of guidance from the funding agency about what data to track. We also saw that different data are needed from different projects to determine a project’s true impact. Defining “success” the same way across all projects is inadequate. Conclusions Although, due to the limited sample size, these findings cannot be generalized to the larger ATE population, they provide specific insights into the various barriers that projects encounter in collecting participation and completion data. 
    more » « less
  2. First-generation (FG) and/or low-income (LI) engineering student populations are of particular interest in engineering education. However, these populations are not defined in a consistent manner across the literature or amongst stakeholders. The intersectional identities of these groups have also not been fully explored in most quantitative-based engineering education research. This research paper aims to answer the following three research questions: (RQ1) How do students’ demographic characteristics and college experiences differ depending on levels of parent educational attainment (which forms the basis of first-generation definitions) and family income? (RQ2) How do ‘first-generation’ and ‘low-income’ definitions impact results comparing to their continuing-generation and higher-income peers? (RQ3) How does considering first-generation and low-income identities through an intersectional lens deepen insight into the experiences of first-generation and low-income groups? Data were drawn from a nationally representative survey of engineering juniors and seniors (n = 6197 from 27 U.S. institutions). Statistical analyses were conducted to evaluate respondent differences in demographics (underrepresented racial/ethnic minority (URM), women, URM women), college experiences (internships/co-ops, having a job, conducting research, and study abroad), and engineering task self-efficacy (ETSE), based on various definitions of ‘first generation’ and ‘low income’ depending on levels of parental educational attainment and self-reported family income. Our results indicate that categorizing a first-generation student as someone whose parents have less than an associate’s degree versus less than a bachelor’s degree may lead to different understandings of their experiences (RQ1). For example, the proportion of URM students is higher among those whose parents have less than an associate’s degree than among their “associate’s degree or more” peers (26% vs 11.9%). However, differences in college experiences are most pronounced among students whose parents have less than a bachelor’s degree compared with their “bachelor’s degree or more” peers: having a job to help pay for college (55.4% vs 47.3%), research with faculty (22.7% vs 35.0%), and study abroad (9.0% vs 17.3%). With respect to differences by income levels, respondents are statistically different across income groups, with fewer URM students as family income level increases. As family income level increases, there are more women in aggregate, but fewer URM women. College experiences are different for the middle income or higher group (internship 48.4% low and lower-middle income vs 59.0% middle income or higher; study abroad 11.2% vs 16.4%; job 58.6% vs 46.8%). Despite these differences in demographic characteristics and college experiences depending on parental educational attainment and family income, our dataset indicates that the definition does not change the statistical significance when comparing between first-generation students and students who were continuing-generation by any definition (RQ2). First-generation and low-income statuses are often used as proxies for one another, and in this dataset, are highly correlated. However, there are unique patterns at the intersection of these two identities. For the purpose of our RQ3 analysis, we define ‘first-generation’ as students whose parents earned less than a bachelor’s degree and ‘low-income’ as low or lower-middle income. In this sample, 68 percent of students were neither FG nor LI while 11 percent were both (FG&LI). On no measure of demographics or college experience is the FG&LI group statistically similar to the advantaged group. Low-income students had the highest participation in working to pay for college, regardless of parental education, while first-generation students had the lower internship participation than low-income students. Furthermore, being FG&LI is associated with lower ETSE compared with all other groups. These results suggest that care is required when applying the labels “first-generation” and/or “low-income” when considering these groups in developing institutional support programs, in engineering education research, and in educational policy. Moreover, by considering first-generation and low-income students with an intersectional lens, we gain deeper insight into engineering student populations that may reveal potential opportunities and barriers to educational resources and experiences that are an important part of preparation for an engineering career. 
    more » « less
  3. There is little research or understanding of curricular differences between two- and four-year programs, career development of engineering technology (ET) students, and professional preparation for ET early career professionals [1]. Yet, ET credentials (including certificates, two-, and four-year degrees) represent over half of all engineering credentials awarded in the U.S [2]. ET professionals are important hands-on members of engineering teams who have specialized knowledge of components and engineering systems. This research study focuses on how career orientations affect engineering formation of ET students educated at two-year colleges. The theoretical framework guiding this study is Social Cognitive Career Theory (SCCT). SCCT is a theory which situates attitudes, interests, and experiences and links self-efficacy beliefs, outcome expectations, and personal goals to educational and career decisions and outcomes [3]. Student knowledge of attitudes toward and motivation to pursue STEM and engineering education can impact academic performance and indicate future career interest and participation in the STEM workforce [4]. This knowledge may be measured through career orientations or career anchors. A career anchor is a combination of self-concept characteristics which includes talents, skills, abilities, motives, needs, attitudes, and values. Career anchors can develop over time and aid in shaping personal and career identity [6]. The purpose of this quantitative research study is to identify dimensions of career orientations and anchors at various educational stages to map to ET career pathways. The research question this study aims to answer is: For students educated in two-year college ET programs, how do the different dimensions of career orientations, at various phases of professional preparation, impact experiences and development of professional profiles and pathways? The participants (n=308) in this study represent three different groups: (1) students in engineering technology related programs from a medium rural-serving technical college (n=136), (2) students in engineering technology related programs from a large urban-serving technical college (n=52), and (3) engineering students at a medium Research 1 university who have transferred from a two-year college (n=120). All participants completed Schein’s Career Anchor Inventory [5]. This instrument contains 40 six-point Likert-scale items with eight subscales which correlate to the eight different career anchors. Additional demographic questions were also included. The data analysis includes graphical displays for data visualization and exploration, descriptive statistics for summarizing trends in the sample data, and then inferential statistics for determining statistical significance. This analysis examines career anchor results across groups by institution, major, demographics, types of educational experiences, types of work experiences, and career influences. This cross-group analysis aids in the development of profiles of values, talents, abilities, and motives to support customized career development tailored specifically for ET students. These findings contribute research to a gap in ET and two-year college engineering education research. Practical implications include use of findings to create career pathways mapped to career anchors, integration of career development tools into two-year college curricula and programs, greater support for career counselors, and creation of alternate and more diverse pathways into engineering. Words: 489 References [1] National Academy of Engineering. (2016). Engineering technology education in the United States. Washington, DC: The National Academies Press. [2] The Integrated Postsecondary Education Data System, (IPEDS). (2014). Data on engineering technology degrees. [3] Lent, R.W., & Brown, S.B. (1996). Social cognitive approach to career development: An overivew. Career Development Quarterly, 44, 310-321. [4] Unfried, A., Faber, M., Stanhope, D.S., Wiebe, E. (2015). The development and validation of a measure of student attitudes toward science, technology, engineeirng, and math (S-STEM). Journal of Psychoeducational Assessment, 33(7), 622-639. [5] Schein, E. (1996). Career anchors revisited: Implications for career development in the 21st century. Academy of Management Executive, 10(4), 80-88. [6] Schein, E.H., & Van Maanen, J. (2013). Career Anchors, 4th ed. San Francisco: Wiley. 
    more » « less
  4. This paper provides a detailed examination of pre-college computing activities as reported in three Association of Computing Machinery (ACM) venues (2012-2016). Ninety-two articles describing informal learning activities were reviewed for 24 program elements (i.e., activity components, and student/instructor demographics). These 24 program elements were defined and shaped by a virtual focus group study and the articles themselves. Results indicate that the majority of authors adequately report age/grade levels of participants, number of participants, the type of activity, when the activity was offered, the tools/languages used in the activity, and whether the activity was required or elective. However, there is a deficiency in reporting many other important and foundational program elements, including contact hours of activity participants, clear learning objectives, the prior experience of participants (students and instructors), and many more. In conjunction with previous work, this paper provides recommendations to reduce these deficiencies. The Recommendations for Reporting Pre-College Computing Activities (Version 1.0) are presented to help researchers improve the quality of papers, set a standard of necessary data needed to replicate studies, and provide a basis for comparing activities and activity outcomes across multiple studies and experiences. 
    more » « less
  5. This paper provides a detailed examination of pre-college computing activities as reported in three Association of Computing Machinery (ACM) venues (2012-2016). Ninety-two articles describing informal learning activities were reviewed for 24 program elements (i.e., activity components, and student/instructor demographics). These 24 program elements were defined and shaped by a virtual focus group study and the articles themselves. Results indicate that the majority of authors adequately report age/grade levels of participants, number of participants, the type of activity, when the activity was offered, the tools/languages used in the activity, and whether the activity was required or elective. However, there is a deficiency in reporting many other important and foundational program elements, including contact hours of activity participants, clear learning objectives, the prior experience of participants (students and instructors), and many more. In conjunction with previous work, this paper provides recommendations to reduce these deficiencies. The Recommendations for Reporting Pre-College Computing Activities (Version 1.0) are presented to help researchers improve the quality of papers, set a standard of necessary data needed to replicate studies, and provide a basis for comparing activities and activity outcomes across multiple studies and experiences. 
    more » « less