More Like this

1. Development of a College Student Validation Survey: A Design-Based Research Approach

   May, TA; Bright, DN; Fan, YK; Fornaro, C; Koskey, KL; Heverin, T (May 2023, Mainsl haul)

   Rendón’s (1994) seminal research on validation theory (VT) provided a model for understanding how validating experiences can positively influence “culturally diverse” (p. 33) students in higher education. Validation is “an enabling, confirming and supportive process initiated by in- and out-of-class agents that fosters academic and interpersonal development” (Rendón, 1994, p. 44) and is critical for the transition, persistence, and success of college students (Rendón, 1994, 2002). Through this theoretical model, scholars have extensively explored how institutions can provide validating experiences by developing supportive learning environments for general undergraduate populations and specific groups such as Black, Latinx, low-income, first-generation, and two-year college students (e.g., Allen, 2016; Bauer, 2014). Many prior studies have relied on qualitative methods. While Rendón and Muñoz (2011) have called for further study of validation’s impact on student outcomes through quantitative methods, few quantitative instruments of VT exist. The primary tool used for assessing VT consists of two scales from the larger Diverse Learning Environments (DLE; Hurtado et al., 2011) survey that have demonstrated their effectiveness for measuring academic validation in class and general interpersonal validation among college students at large (Hurtado et al., 2015). DLE scales were not, however, designed to match Rendón’s full four-component conception of VT (i.e., academic in-class, academic out-of-class, interpersonal in-class, interpersonal out-of-class). Thus, a new measure of VT is necessary to capture quantitative information aligned with Rendón’s model. The purpose of this study was to expand the field of quantitative VT research by presenting validity evidence from a new survey entitled the Validation Theory Survey (VTS) that was designed to align with Rendón’s VT model and to be used with undergraduate students. One overarching research question guided this study: To what extent did validity evidence (i.e., content, response process, consequential, and internal structure) support the use of the VTS to evaluate undergraduates’ perceptions of their academic and interpersonal validating experiences inside and outside higher education classrooms? 

   more » « less
2. A Measure of Affect toward Key Elements of Engineering Professional Practice.

   Patrick, A D; Choe, N H; Martins, L L; Borrego, M J; Kendall, M R; Seepersad, C C (June 2017, ASEE annual conference & exposition proceedings)

   Identity, or how people choose to define themselves, is emerging as an attractive explanation for who persists in engineering. Many studies of engineering identity build off of prior work in math and science identity, emphasizing the academic aspects of engineering. However, affect towards professional practice is also central to engineering identity development. This paper describes the methods used to create a new survey measure of individuals’ affect toward elements of engineering practice. We followed the item generation, refinement, and instrument validation steps required for psychometric validation of a new survey measure. We generated items deductively using the literature on engineering professional skills and practice and inductively based on interviews with practicing engineers, engineering graduate students, and engineering undergraduate students. We blended the inductively and deductively derived item lists to create a list of initial items for the measure. We circulated this list of items to a set of engineering and professional identity experts to establish face validity and made modifications based on their feedback. The final list included 34 items. These 34 items were administered in a questionnaire survey in the fall of 2016 to 1465 engineering undergraduates in three majors at two institutions. We conducted an exploratory factor analysis (EFA) and established internal consistency using Cronbach’s alpha on a subset of the analytical sample data (n=384). The resulting factors fit our a priori assumption of the factors theorized to characterize affect towards engineering professional practice. Using the remaining data (n=904), we conducted a confirmatory factor analysis on the reduced set of items resulting from EFA. The results indicate an emergent factor structure for affect towards elements of engineering practice. 

   more » « less
3. Student Persistence Factors for Engineering and Computing Undergraduates

   Gholezadeh, Sona; Petrulis, Bob; Gatzke Edward P (June 2023, ASEE Annual Conference)

   The research and evaluation team of an S-STEM project at a large, research-intensive Southeastern public university conducted a cross-sectional survey as a first step to compare factors which may influence undergraduate student persistence in engineering and computing. All engineering and computing students were invited to participate in the survey, and 282 (10.4%) provided responses. The respondents included 15 high financial need students who were participating in the S-STEM program, of which 7 were first-year students and 8 were sophomores. The remaining 267 respondents were undergraduates ranging from first-year to seniors. Survey questions were adapted from previously developed instruments on self-efficacy, sense-of-belonging, identity, community involvement, and overall college experience. Additional questions related to stress levels, academic life, use and effectiveness of academic supports, and the impacts of COVID-19 on their college experiences. The team compared responses by level of academic progression, declared major, gender, and race/ethnicity. Student responses showed a variety of similarities and differences between subgroups. Overall, the students said that they often attended lectures (in-person or online) and came to class prepared. At the same time, students rated these activities as the least effective academic supports. On the other hand, the students rated working assigned or extra homework problems and studying for exams as their most effective activities. Consistently among the subgroups, the students said their community involvement and identity as developing engineers were relatively low while self-efficacy and team self-efficacy were seen as stronger personal skills. The students said they were highly stressed about their grades and academic success in general, and about finances and future careers. They reported feeling less stress about aspects such as living away from home and negotiating the university social scene. Students reported spending the most time preparing for class in their first year compared to students in later years. Female students (104 responses) reported higher levels of community involvement, engineering identity, and engagement in college life compared to male students (142 responses) while there was little gender-related difference in self-efficacy and sense of belonging. Levels of self-efficacy and team self-efficacy did not show large differences based on year in college. Interestingly, first-year students expressed the highest levels of engineering identity while senior students the lowest. Senior students reported the lowest community involvement, sense of belonging, and engineering identity compared to other students. Overall, students from different races self-reported the same levels of self-efficacy. Black/African American students reported the highest levels of community involvement, college life, and identity. There were no substantial differences in self-efficacy among the different engineering and computing majors. This study is a first step in analysis of the students’ input. In addition to surveying the students, the team also conducted interviews of the participating S-STEM students, and analysis of these interviews will provide greater depth to interpretation of the survey results. Overall, the research and evaluation team’s intention is to provide insight to the project’s leadership in how best to support the success of first-year engineering and computing students. https://peer.asee.org/student-persistence-factors-for-engineering-and-computing-undergraduates 

   more » « less
4. Evaluating an Abbreviated Version of the Circumplex Team Scan Inventory of Within-Team Interpersonal Norms

   https://doi.org/10.1027/1015-5759/a000752  

   Locke, Kenneth D.; Martin, Chris C. (December 2022, European Journal of Psychological Assessment)

   Abstract. The Circumplex Team Scan (CTS) assesses the degree to which a team’s interaction/communication norms reflect each segment (16th) of the interpersonal circle/circumplex. We developed and evaluated an abbreviated 16-item CTS-16 that uses one CTS item to measure each segment. Undergraduates ( n = 446) completing engineering course projects in 139 teams completed the CTS-16. CTS-16 items showed a good fit to confirmatory structural models (e.g., that expect greater positive covariation between items theoretically closer to the circumplex). Individuals’ ratings sufficiently reflected team-level norms to justify averaging team members’ ratings. However, individual items’ marginal reliabilities suggest using the CTS-16 to assess general circumplex-wide patterns rather than specific segments. CTS-16 ratings correlated with respondents’ and their teammates’ ratings of team climate (inclusion, justice, psychological safety). Teams with more extraverted (introverted) members were perceived as having more confident/engaged (timid/hesitant) cultures. Members predisposed to social alienation perceived their team’s culture as relatively disrespectful/unengaged, but their teammates did not corroborate those perceptions. The results overall support the validity and utility of the CTS-16 and of an interpersonal circumplex model of team culture more generally. 

   more » « less
5. Constructing a Measure of Affect Towards Professional Practice: What matters for Engineers?

   Patrick, A. D.; Borrego, M. J.; Martins, L. L.; Choe, N. H.; Seepersad, C. C.; Kendall, M. R. (January 2017, Research in Engineering Education Symposium)

   Identity, or how people choose to define themselves, is emerging as an explanation for who pursues and persists in engineering. Recent developments in the study of engineering identity, including studies of math and science identity, tend to emphasize the academic aspects of engineering without considering aspects of professional practice central to the development of an engineering identity. This paper outlines the methods used to create a new survey measure: affect toward elements of engineering practice. We followed the item generation, refinement, and instrument validation steps required for psychometric validation of a new survey measure. Through this process a final list of 34 items was administered in a survey in the fall of 2016 to engineering undergraduates. We conducted an exploratory factor analysis and established internal consistency using Cronbach’s alpha on a subset of the data sample (n=384). The resulting factors reflect key elements of affect towards engineering professional practice. 

   more » « less