Title: Assessment of student learning – field application. Earth Educators Rendezvous (4th annual),
Our research, Landscapes of Deep Time in the Red Earth of France (NSF International Research Experience for Students project), aims to mentor
U.S. undergraduate science students from underserved populations (e.g. students of Native American heritage and/or first-generation college
students) in geological research. During the first field season (June 2018) formative and summative assessments (outlined below) will be issued to
assist in our evaluation of student learning. The material advancement of a student's sedimentological skillsets and self-efficacy development in
research applications are a direct measure of our program's success.
(1) Immediately before and after the program, students will self-rank their competency of specific skillsets (e.g. data collection, lithologic
description, use of field equipment) in an anonymous summative assessment.
(2) Formative assessments throughout the field season (e.g. describing stratigraphic section independently, oral and written communication of
results) will assess improved comprehension of the scientific process.
(3) An anonymous attitudinal survey will be issued at the conclusion of the field season to shed light on the program's quality as a whole,
influence on student desire to pursue a higher-level degree/career in STEM, and effectiveness of the program on aiding the development of
participant confidence and self-efficacy in research design and application.
We discuss herein the results of first-year assessments with a focus on strategies for improvement. We expect each individual's outcomes to differ
depending on his/her own characteristics and background. Furthermore, some of the most valued intentions of this experience are inherently
difficult to measure (e.g., improved understanding of the scientific process, a stimulated passion to pursue a STEM career). We hope to address
shortcomings in design; e.g. Where did we lose visibility on certain aspects of the learning experience? How can we revise the format and content
of our assessment to better evaluate student participants and improve our program in subsequent years? more »« less
Colaninno, Carol E.(
, Journal of archaeology and education)
null
(Ed.)
Participation in an archaeological field school is the entry point to a professional career in the discipline. Despite the importance of field schools, few scholars have investigated achieved student-learning outcomes or lasting impacts on students from participation in archaeological field research. We report on the educational design, learning objectives, and results of three years of formative and summative assessments for an interdisciplinary, archaeology and ecology research program for undergraduate students. Our learning objectives include promoting scientific literacy and communication, critical thinking and STEM skills, and capacities in archaeological and ecological interdisciplinarity. Using developed rubrics that account for both critical thinking and STEM understanding, self-administered competency surveys, and program-developed items, we found significant gains in nearly all learning objectives. Students demonstrated growth in program specific content, perceived abilities in their scientific and discipline specific skills, critical thinking skills, and scientific communication skills. These educational outcomes and assessment tools have implications for how we design and evaluate field learning in archaeology and may be applied to field school instruction.
Advancements in information technology and computational intelligence have transformed the manufacturing landscape, allowing firms to produce highly complex and customized product in a relatively short amount of time. However, our research shows that the lack of a skilled workforce remains a challenge in the manufacturing industry. To that end, providing research experience to undergraduates has been widely reported as a very effective approach to attract students to industry or graduate education in engineering and other STEM-based majors. This paper presents assessment results of two cohorts of Cybermanufacturing REU at a major university. Students were recruited from across the United States majoring in multiple engineering fields, such as industrial engineering, mechanical engineering, chemical engineering, mechatronics, manufacturing, and computer science. Several of the participants were rising sophomores or juniors who did not have any industry internship or prior research experience. In total 20 students
(ten per year) participated in the program and worked on individual project topics under the guidance of faculty and graduate student mentors. Unlike a typical REU program, the
Cybermanufacturing REU involved a few unique activities, such as a 48-hour intense design and prototype build experience (also known as Aggies Invent), industry seminars, and industry visits. Overall, the REU students demonstrated significant gains in all of the twelve research-related competencies that were assessed as a part of formative and summative evaluation process. While almost all of them wanted to pursue a career in advanced manufacturing, including Cybermanufacturing, the majority of the participants preferred industry over graduate school. The paper provides an in-depth discussion on the findings of the REU program evaluation and its impact on undergraduate students with respect to their future plans and career choice. The analysis is also done by gender, ethnicity, academic level (sophomore, junior, senior), and type of home institution (e.g., large research universities, rural and small schools) to explore if there was any
significant difference in mean research competency scores based on these attributes.
This paper summarizes the overall approach and assessment of a National Science Foundation Research Experience for Undergraduates Site focused on sustainable civil and environmental infrastructure in rural areas. This site has hosted over 60 students over 5 years, including 1 year of virtual participation due to travel restrictions associated with the COVID-19 pandemic. Detailed discussion and results are provided with respect to the recruitment approach, including particular attention to first-generation college students, and the potential negative impacts of the COVID-19 pandemic on first-generation applicants. This site also incorporates targeted instruction on technical writing, which occurs over several weeks throughout the first half of the summer and culminates with a final conference paper deliverable. This approach has yielded over 20 peer-reviewed journal articles, conference papers, or national conference presentations, which have been co-authored by the undergraduate student participants. External evaluation of this site has included both formative and summative assessments, including pre-program, mid-program, and post-program surveys and focus groups, which has enabled a successful continuous improvement cycle, in which cohort-building activities, technical writing deliverables, and mentor training have been gradually incorporated or enhanced. Results of this assessment have also been used to quantify the site’s success with respect to student exposure and interest in research and graduate education. In addition to most participants persisting in STEM fields, many have gone on to pursue graduate school in civil and environmental engineering and win national fellowships.
The importance of diversifying the national STEM workforce is well-established in the literature (Marrongelle, 2018). This need extends to graduate education in the STEM fields, leading N.C. A&T to invest considerably in graduate education and wraparound support initiatives that help graduate students build science identity and competencies for careers both within and beyond academia. The NSF-funded Bridges to the Doctorate project will integrate culturally reflective mentoring and professional development specifically designed for Black, Latinx, and Native American Ph.D. students. This holistic, graduate student development model includes academic and professional skill-building for STEM careers alongside targeted support for pursuing fellowship opportunities. This paper discusses the planned mentoring approach for the aforementioned program and previous approaches to mentoring graduate students used at N.C. A&T. The BD Fellows program will support formal and informal mentoring relationships, as mentoring contributes towards retention in STEM graduate programs (Ragins, 2007). BD Fellows will participate in monthly one-hour seminars on how to identify, establish, and maintain informal mentoring relationships (Schwartz et al., 2018; Parnes et al., 2020), while STEM faculty will attend seminars on leveraging their social networks as vital sources of mentorship for the BD Fellows. Using a multi-pronged collaborative approach, this model integrates the evidence-based domains of self-efficacy (Laurencelle & Scanlan, 2018; Lent et al., 1994; Lent et al., 2008), science/research identity (Lent et al., 2015; Zimmerman, 2000), and social cognitive career theory (Lent et al., 2005; Lent and Brown, 2006) to recruit, enroll, and graduate LSAMP Fellows with STEM doctoral degrees. Guided by the theories, the following questions will be addressed: (1) To what extent is culturally reflective mentoring identified as a critical driver of B2D Fellows’ success? (2) To what extent are the program’s training components fostering increases in B2D Fellow’s self-efficacy, competency, and science identity? (3) What is the strength of the correlation between participation in the program training components, mentoring activities, and persistence in graduate school? (4) To what extent does the perceived importance of self-efficacy, competency, and science identity differ by race/ethnicity and gender? These data will be analyzed using both formative and summative assessments of program outcomes. Quantitative data will include pre-, post-, and exit surveys. Qualitative data will assess the impact of mentoring and program support. This study will be guided by established protocols that have been approved by the N.C. A&T IRB. It is anticipated that our BD Fellows program will significantly impact the retention and graduation rates of underrepresented minority STEM graduate students in our doctoral programs, thus producing a diverse workforce of STEM professionals. Materials from the program recruiting cycle, mentoring workshops, and the structured fellowship application process will be disseminated freely to other LSAMP and minority-serving institutions across the country. Strategies and outcomes of this project will be published in peer-reviewed journals and shared in conference proceedings.
The Academy of Engineering Success (AcES) program, established in 2012 and supported by NSF S-STEM award number 1644119 throughout 2016-2021, employs literature-based, best practices to support and retain underprepared and underrepresented students in engineering through graduation with the ultimate goal of diversifying the engineering workforce. A total of 71 students, including 21 students supported by S-STEM scholarships, participated in the AcES program between 2016-2019 at a large R1 institution in the mid-Atlantic region.
All AcES students participate in a common program during their first year, comprised of: a one-week summer bridge experience, a common fall professional development course and spring “Engineering in History” course, and a common academic advisor. These students also have opportunities for: (1) faculty-student, student-student, and industry mentor-student interaction, (2) academic support and student success education, and (3) major and career exploration – all designed to help students develop feelings of institutional inclusion, engineering self-efficacy and identity, and academic and professional success skills. They also participate in the GRIT, Longitudinal Assessment of Engineering Self-Efficacy (LAESE), and the Motivated Strategies for Learning Questionnaire (MSLQ) surveys plus individual and focus group interviews at the start, midpoint, and end of each fall semester and at the end of the spring semester. The surveys provide a measure of students’ GRIT, their beliefs related to the intrinsic value of engineering and learning, their feelings of inclusion and test anxiety, and their self-efficacy related to engineering, math, and coping skills. The interviews provide information related to the student experience, feelings of inclusion, and program impact.
Institutional data, combined with the survey and interview responses, are used to examine four research questions designed to examine the relationship of the elements of the AcES program to participants’ academic success and retention in engineering. Early analyses of the student retention data and survey responses from the 2017 and 2018 cohorts indicated students who ultimately left engineering before the start of their second year initially scored higher in areas of engineering self-efficacy and test anxiety, than those who stayed in engineering, while those who retained to the second year began their engineering education with lower self-efficacy scores, but higher scores related to the belief in the intrinsic value of engineering, learning strategy use, and coping self-efficacy. These results suggest that students who start with unrealistically high expectations of their performance leave engineering at higher rates than students who start with lower personal performance expectations, but have stronger value of the field and strategies for meeting challenges. These data appear to support the Kruger-Dunning effect in which students with limited knowledge of a specific field overestimate their abilities to perform in that area or underestimate the level of effort success may require.
This paper will add an analysis of the academic success and retention data from 2019 cohort to this research, discuss the impact of COVID-19 to this program and research, as well as illuminate the quantitative results with the qualitative data from individual and focus group interviews regarding the aspects of the AcES program that impact student success, their expectations and methods for overcoming academic challenges, and their feelings of motivation and inclusion.
@article{osti_10099330,
place = {Country unknown/Code not available},
title = {Assessment of student learning – field application. Earth Educators Rendezvous (4th annual),},
url = {https://par.nsf.gov/biblio/10099330},
abstractNote = {Our research, Landscapes of Deep Time in the Red Earth of France (NSF International Research Experience for Students project), aims to mentor U.S. undergraduate science students from underserved populations (e.g. students of Native American heritage and/or first-generation college students) in geological research. During the first field season (June 2018) formative and summative assessments (outlined below) will be issued to assist in our evaluation of student learning. The material advancement of a student's sedimentological skillsets and self-efficacy development in research applications are a direct measure of our program's success. (1) Immediately before and after the program, students will self-rank their competency of specific skillsets (e.g. data collection, lithologic description, use of field equipment) in an anonymous summative assessment. (2) Formative assessments throughout the field season (e.g. describing stratigraphic section independently, oral and written communication of results) will assess improved comprehension of the scientific process. (3) An anonymous attitudinal survey will be issued at the conclusion of the field season to shed light on the program's quality as a whole, influence on student desire to pursue a higher-level degree/career in STEM, and effectiveness of the program on aiding the development of participant confidence and self-efficacy in research design and application. We discuss herein the results of first-year assessments with a focus on strategies for improvement. We expect each individual's outcomes to differ depending on his/her own characteristics and background. Furthermore, some of the most valued intentions of this experience are inherently difficult to measure (e.g., improved understanding of the scientific process, a stimulated passion to pursue a STEM career). We hope to address shortcomings in design; e.g. Where did we lose visibility on certain aspects of the learning experience? How can we revise the format and content of our assessment to better evaluate student participants and improve our program in subsequent years?},
journal = {Earth Educator's Rendevous Abstracts},
author = {Lily Pfeifer, Lynn Soreghan},
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
