There is a critical need for more students with engineering and computer science majors to enter
into, persist in, and graduate from four-year postsecondary institutions. Increasing the diversity
of the workforce by inclusive practices in engineering and science is also a profound identified
need. According to national statistics, the largest groups of underrepresented minority students in
engineering and science attend U.S. public higher education institutions. Most often, a large
proportion of these students come to colleges and universities with unique challenges and needs,
and are more likely to be first in their family to attend college. In response to these needs,
engineering education researchers and practitioners have developed, implemented and assessed
interventions to provide support and help students succeed in college, particularly in their first
year. These interventions typically target relatively small cohorts of students and can be managed
by a small number of faculty and staff. In this paper, we report on “work in progress” research in
a large-scale, first-year engineering and computer science intervention program at a public,
comprehensive university using multivariate comparative statistical approaches.
Large-scale intervention programs are especially relevant to minority serving institutions that
prepare growing numbers of students who are first in their family to attend college and who are
also under-resourced, financially. These students most often encounter academic difficulties and
come to higher education with challenging experiences and backgrounds. Our studied first-year
intervention program, first piloted in 2015, is now in its 5th year of implementation. Its
intervention components include: (a) first-year block schedules, (b) project-based introductory
engineering and computer science courses, (c) an introduction to mechanics course, which
provides students with the foundation needed to succeed in a traditional physics sequence, and
(d) peer-led supplemental instruction workshops for calculus, physics and chemistry courses.
This intervention study responds to three research questions: (1) What role does the first-year
intervention’s components play in students’ persistence in engineering and computer science
majors across undergraduate program years? (2) What role do particular pedagogical and cocurricular
support structures play in students’ successes? And (3) What role do various student
socio-demographic and experiential factors play in the effectiveness of first-year interventions?
To address these research questions and therefore determine the formative impact of the firstyear
engineering and computer science program on which we are conducting research, we have
collected diverse student data including grade point averages, concept inventory scores, and data
from a multi-dimensional questionnaire that measures students’ use of support practices across
their four to five years in their degree program, and diverse background information necessary to
determine the impact of such factors on students’ persistence to degree. Background data
includes students’ experiences prior to enrolling in college, their socio-demographic
characteristics, and their college social capital throughout their higher education experience. For
this research, we compared students who were enrolled in the first-year intervention program to
those who were not enrolled in the first-year intervention. We have engaged in cross-sectional
2
data collection from students’ freshman through senior years and employed multivariate
statistical analytical techniques on the collected student data.
Results of these analyses were interesting and diverse. Generally, in terms of backgrounds, our
research indicates that students’ parental education is positively related to their success in
engineering and computer science across program years. Likewise, longitudinally (across
program years), students’ college social capital predicted their academic success and persistence
to degree. With regard to the study’s comparative research of the first-year intervention, our
results indicate that students who were enrolled in the first-year intervention program as
freshmen continued to use more support practices to assist them in academic success across their
degree matriculation compared to students who were not in the first-year program. This suggests
that the students continued to recognize the value of such supports as a consequence of having
supports required as first-year students. In terms of students’ understanding of scientific or
engineering-focused concepts, we found significant impact resulting from student support
practices that were academically focused. We also found that enrolling in the first-year
intervention was a significant predictor of the time that students spent preparing for classes and
ultimately their grade point average, especially in STEM subjects across students’ years in
college. In summary, we found that the studied first-year intervention program has longitudinal,
positive impacts on students’ success as they navigate through their undergraduate experiences
toward engineering and computer science degrees.
more »
« less
Criteria for collapsing rating scale responses: A case study of the CLASS
Assessments of students’ attitudes and beliefs often rely on questions with rating scales that ask students the extent to which they agree or disagree with a statement. Unlike traditional physics problems with a single correct answer, rating scale questions often have a spectrum of 5 or more responses, none of which are correct. Researchers have found that responses on rating scale items can generally be treated as continuous and that unless there is good evidence to do otherwise, response categories should not be collapsed [1–3]. We discuss two potential reasons for collapsing response categories (lack of use and redundancy) and how to empirically test for them. To illustrate these methods, we use them on the Colorado Learning Attitudes about Science Survey. We found that students used all the response categories on the CLASS but that three of them were potentially redundant. This led us to conclude that the CLASS should be scored on a 5-point or 3-point scale, rather than the 2-point scale recommended by the instrument developers [4]. More broadly, we recommend the judicious use of data manipulations when scoring assessments and retaining all response categories unless there is a strong rational for collapsing them.
more »
« less
- NSF-PAR ID:
- 10192528
- Date Published:
- Journal Name:
- 2019 Physics Education Research Conference Proceedings
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Gender, Soft Skills, and Patient Experience in Online Physician Reviews: A Large-Scale Text AnalysisBackground Online physician reviews are an important source of information for prospective patients. In addition, they represent an untapped resource for studying the effects of gender on the doctor-patient relationship. Understanding gender differences in online reviews is important because it may impact the value of those reviews to patients. Documenting gender differences in patient experience may also help to improve the doctor-patient relationship. This is the first large-scale study of physician reviews to extensively investigate gender bias in online reviews or offer recommendations for improvements to online review systems to correct for gender bias and aid patients in selecting a physician. Objective This study examines 154,305 reviews from across the United States for all medical specialties. Our analysis includes a qualitative and quantitative examination of review content and physician rating with regard to doctor and reviewer gender. Methods A total of 154,305 reviews were sampled from Google Place reviews. Reviewer and doctor gender were inferred from names. Reviews were coded for overall patient experience (negative or positive) by collapsing a 5-star scale and coded for general categories (process, positive/negative soft skills), which were further subdivided into themes. Computational text processing methods were employed to apply this codebook to the entire data set, rendering it tractable to quantitative methods. Specifically, we estimated binary regression models to examine relationships between physician rating, patient experience themes, physician gender, and reviewer gender). Results Female reviewers wrote 60% more reviews than men. Male reviewers were more likely to give negative reviews (odds ratio [OR] 1.15, 95% CI 1.10-1.19; P<.001). Reviews of female physicians were considerably more negative than those of male physicians (OR 1.99, 95% CI 1.94-2.14; P<.001). Soft skills were more likely to be mentioned in the reviews written by female reviewers and about female physicians. Negative reviews of female doctors were more likely to mention candor (OR 1.61, 95% CI 1.42-1.82; P<.001) and amicability (OR 1.63, 95% CI 1.47-1.90; P<.001). Disrespect was associated with both female physicians (OR 1.42, 95% CI 1.35-1.51; P<.001) and female reviewers (OR 1.27, 95% CI 1.19-1.35; P<.001). Female patients were less likely to report disrespect from female doctors than expected from the base ORs (OR 1.19, 95% CI 1.04-1.32; P=.008), but this effect overrode only the effect for female reviewers. Conclusions This work reinforces findings in the extensive literature on gender differences and gender bias in patient-physician interaction. Its novel contribution lies in highlighting gender differences in online reviews. These reviews inform patients’ choice of doctor and thus affect both patients and physicians. The evidence of gender bias documented here suggests review sites may be improved by providing information about gender differences, controlling for gender when presenting composite ratings for physicians, and helping users write less biased reviews.more » « less
-
The landscapes of many elementary, middle, and high school math classrooms have undergone major transformations over the last half-century, moving from drill-and-skill work to more conceptual reasoning and hands-on manipulative work. However, if you look at a college level calculus class you are likely to find the main difference is the professor now has a whiteboard marker in hand rather than a piece of chalk. It is possible that some student work may be done on the computer, but much of it contains the same type of repetitive skill building problems. This should seem strange given the advancements in technology that allow more freedom than ever to build connections between different representations of a concept. Several class activities have been developed using a combination of approaches, depending on the topic. Topics covered in the activities include Riemann Sums, Accumulation, Center of Mass, Volumes of Revolution (Discs, Washers, and Shells), and Volumes of Similar Cross-section. All activities use student note outlines that are either done in a whole group interactive-lecture approach, or in a group work inquiry-based approach. Some of the activities use interactive graphs designed on desmos.com and others use physical models that have been designed in OpenSCAD and 3D-printed for students to use in class. Tactile objects were developed because they should provide an advantage to students by enabling them to physically interact with the concepts being taught, deepening their involvement with the material, and providing more stimuli for the brain to encode the learning experience. Web-based activities were developed because the topics involved needed substantial changes in graphical representations (i.e. limits with Riemann Sums). Assessment techniques for each topic include online homework, exams, and online concept questions with an explanation response area. These concept questions are intended to measure students’ ability to use multiple representations in order to answer the question, and are not generally computational in nature. Students are also given surveys to rate the overall activities as well as finer grained survey questions to try and elicit student thoughts on certain aspects of the models, websites, and activity sheets. We will report on student responses to the activity surveys, looking for common themes in students’ thoughts toward specific attributes of the activities. We will also compare relevant exam question responses and online concept question results, including common themes present or absent in student reasoning.more » « less
-
There is little research or understanding of curricular differences between two- and four-year programs, career development of engineering technology (ET) students, and professional preparation for ET early career professionals [1]. Yet, ET credentials (including certificates, two-, and four-year degrees) represent over half of all engineering credentials awarded in the U.S [2]. ET professionals are important hands-on members of engineering teams who have specialized knowledge of components and engineering systems. This research study focuses on how career orientations affect engineering formation of ET students educated at two-year colleges. The theoretical framework guiding this study is Social Cognitive Career Theory (SCCT). SCCT is a theory which situates attitudes, interests, and experiences and links self-efficacy beliefs, outcome expectations, and personal goals to educational and career decisions and outcomes [3]. Student knowledge of attitudes toward and motivation to pursue STEM and engineering education can impact academic performance and indicate future career interest and participation in the STEM workforce [4]. This knowledge may be measured through career orientations or career anchors. A career anchor is a combination of self-concept characteristics which includes talents, skills, abilities, motives, needs, attitudes, and values. Career anchors can develop over time and aid in shaping personal and career identity [6]. The purpose of this quantitative research study is to identify dimensions of career orientations and anchors at various educational stages to map to ET career pathways. The research question this study aims to answer is: For students educated in two-year college ET programs, how do the different dimensions of career orientations, at various phases of professional preparation, impact experiences and development of professional profiles and pathways? The participants (n=308) in this study represent three different groups: (1) students in engineering technology related programs from a medium rural-serving technical college (n=136), (2) students in engineering technology related programs from a large urban-serving technical college (n=52), and (3) engineering students at a medium Research 1 university who have transferred from a two-year college (n=120). All participants completed Schein’s Career Anchor Inventory [5]. This instrument contains 40 six-point Likert-scale items with eight subscales which correlate to the eight different career anchors. Additional demographic questions were also included. The data analysis includes graphical displays for data visualization and exploration, descriptive statistics for summarizing trends in the sample data, and then inferential statistics for determining statistical significance. This analysis examines career anchor results across groups by institution, major, demographics, types of educational experiences, types of work experiences, and career influences. This cross-group analysis aids in the development of profiles of values, talents, abilities, and motives to support customized career development tailored specifically for ET students. These findings contribute research to a gap in ET and two-year college engineering education research. Practical implications include use of findings to create career pathways mapped to career anchors, integration of career development tools into two-year college curricula and programs, greater support for career counselors, and creation of alternate and more diverse pathways into engineering. Words: 489 References [1] National Academy of Engineering. (2016). Engineering technology education in the United States. Washington, DC: The National Academies Press. [2] The Integrated Postsecondary Education Data System, (IPEDS). (2014). Data on engineering technology degrees. [3] Lent, R.W., & Brown, S.B. (1996). Social cognitive approach to career development: An overivew. Career Development Quarterly, 44, 310-321. [4] Unfried, A., Faber, M., Stanhope, D.S., Wiebe, E. (2015). The development and validation of a measure of student attitudes toward science, technology, engineeirng, and math (S-STEM). Journal of Psychoeducational Assessment, 33(7), 622-639. [5] Schein, E. (1996). Career anchors revisited: Implications for career development in the 21st century. Academy of Management Executive, 10(4), 80-88. [6] Schein, E.H., & Van Maanen, J. (2013). Career Anchors, 4th ed. San Francisco: Wiley.more » « less
-
The United Nation’s Sustainable Development Goals state climate change could irreversibly affect future generations and is one of the most urgent issues facing society. To date, most education research on climate change examines middle and high school students’ knowledge without considering the link between understanding and interest to address such issues in their career. In research on students’ attitudes about sustainability, we found that half of first-year college engineering students, in our nationally representative sample of all college students at 4-year institutions (n = 937), do not believe climate change is caused by humans. This lack of belief in human-caused climate change is a significant problem in engineering education because our results also indicate engineering students who do not believe in human caused climate change are less likely to want to address climate change in their careers. This dismal finding highlights a need for improving student understanding and attitudes toward climate change in order to produce engineers prepared and interested in solving complex global problems in sustainability. To advance understanding about students’ understanding of climate change and their agency to address the issue, we developed the CLIMATE survey to measure senior undergraduate engineering students’ Climate change literacy, engineering identity, career motivations, and agency through engineering. The survey was designed for students in their final senior design, or capstone course, just prior to entering the workforce. We developed the survey using prior national surveys and newly written questions categorized into six sections: (1) career goals and motivation, (2) college experiences, (3) agency, (4) climate literacy, (5) people and the planet, and (6) demographic information. We conducted focus groups with students to establish face and content validity of the survey. We collected pilot data with 200 engineering students in upper-level engineering courses to provide validity evidence for the use of these survey items to measure students and track changes across the undergraduate curriculum for our future work. In this paper, we narrate the development of the survey supported by literature and outline the next step for further validation and distribution on a national scale. Our intent is to receive feedback and input about the questions being asked and the CLIMATE instrument. Our objective is to share the nationally representative non-identifiable responses (the estimated goal is 4,000 responses) openly with education researchers interested in students understanding about climate change, their engineering identity, career motivations, and agency through engineering. Ultimately, we want this research to become a catalyst for teaching about topics related to climate change in engineering and its implications for sustainability.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1119/perc.2019.pr.Van_Dusen
