skip to main content


Title: A Preliminary Exploration of the Role of Surveys In Student Reflection and Behavior.
Surveys often are used in educational research to gather information about respondents without considering the effect of survey questions on survey-takers themselves. Does the very act of taking a survey influence perspectives, mindsets, and even behaviors? Does a survey itself effectuate attitudinal change? Such effects of surveys, and implications for survey data interpretation, warrant close attention. There is a long tradition of research on surveys as behavioral interventions within political science and social psychology, but limited attention has been given to the topic in engineering education, and higher education more broadly. Recently the engineering education community has started to examine the potential effects of assessment techniques (including surveys) as catalysts for reflection. In March 2014, the Consortium to Promote Reflection in Engineering Education (CPREE), representing a two-year collaboration amongst 12 campuses, was established to promote “a broader understanding and use of reflective techniques in engineering education.”1 CPREE’s formation suggests a growing recognition of reflection as an important and underemphasized aspect of an engineer’s education. CPREE defines reflection as “exploring the meaning of experiences and the consequences of the meanings for future action” and emphasizes the importance of taking action as a result of ascribing meaning to experiences.1 Surveys may be one of several tools that may create opportunities for reflection; others include “exam wrappers” and “homework wrappers” that encourage students to explore how they feel about an assignment or task as part of making meaning of it2,3 (and stimulating the kind of reflection that can lead to action). The current study bridges these two frameworks of behavioral interventions and reflection to consider the “extra-ordinate” dimensions of survey-taking and explores how survey participation may (1) support students’ reflection on past experiences, meaningmaking of these experiences, and insights that “inform [their] path going forward,”1 and (2) be associated with students’ subsequent behaviors. We first review a broader literature on the interventional effects on surveys in political studies and social psychology, after which we present the results obtained from including an optional reflection question at the end of an engineering education survey. We conclude that educators would benefit from considering the range of potential impacts that responding to questions may have on students’ thoughts and actions, rather than treating surveys as neutral data collection devices when designing their research.  more » « less
Award ID(s):
1636442
NSF-PAR ID:
10043002
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Proceedings of the American Society for Engineering Education Annual Conference, June 25-28. Columbus, OH.
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. High levels of stress and anxiety are common amongst college students, particularly engineering students. Students report lack of sleep, grades, competition, change in lifestyle, and other significant stressors throughout their undergraduate education (1, 2). Stress and anxiety have been shown to negatively impact student experience (3-6), academic performance (6-8), and retention (9). Previous studies have focused on identifying factors that cause individual students stress while completing undergraduate engineering degree programs (1). However, it not well-understood how a culture of stress is perceived and is propagated in engineering programs or how this culture impacts student levels of identification with engineering. Further, the impact of student stress has not been directly considered in engineering regarding recruitment, retention, and success. Therefore, our guiding research question is: Does the engineering culture create stress for students that hinder their engineering identity development? To answer our research question, we designed a sequential mixed methods study with equal priority of quantitative survey data and qualitative individual interviews. Our study participants are undergraduate engineering students across all levels and majors at a large, public university. Our sample goal is 2000 engineering student respondents. We combined three published surveys to build our quantitative data collection instrument, including the Depression Anxiety Stress Scales (DASS), Identification with engineering subscale, and Engineering Department Inclusion Level subscale. The objective of the quantitative instrument is to illuminate individual perceptions of the existence of an engineering stress culture (ESC) and create an efficient tool to measure the impact ESC on engineering identity development. Specifically, we seek to understand the relationships among the following constructs; 1) identification with engineering, 2) stress and anxiety, and 3) feelings of inclusion within their department. The focus of this paper presents the results of the pilot of the proposed instrument with 20 participants and a detailed data collection and analysis process. In an effort to validate our instrument, we conducted a pilot study to refine our data collection process and the results will guide the data collection for the larger study. In addition to identifying relationships among construct, the survey data will be further analyzed to specify which demographics are mediating or moderating factors of these relationships. For example, does a student’s 1st generation status influence their perception of stress or engineering identity development? Our analysis may identify discipline-specific stressors and characterize culture components that promote student anxiety and stress. Our objective is to validate our survey instrument and use it to inform the protocol for the follow-up interviews to gain a deeper understanding of the responses to the survey instrument. Understanding what students view as stressful and how students identify stress as an element of program culture will support the development of interventions to mitigate student stress. References 1. Schneider L (2007) Perceived stress among engineering students. A Paper Presented at St. Lawrence Section Conference. Toronto, Canada. Retrieved from: www. asee. morrisville. edu. 2. Ross SE, Niebling BC, & Heckert TM (1999) Sources of stress among college students. Social psychology 61(5):841-846. 3. Goldman CS & Wong EH (1997) Stress and the college student. Education 117(4):604-611. 4. Hudd SS, et al. (2000) Stress at college: Effects on health habits, health status and self-esteem. College Student Journal 34(2):217-228. 5. Macgeorge EL, Samter W, & Gillihan SJ (2005) Academic Stress, Supportive Communication, and Health A version of this paper was presented at the 2005 International Communication Association convention in New York City. Communication Education 54(4):365-372. 6. Burt KB & Paysnick AA (2014) Identity, stress, and behavioral and emotional problems in undergraduates: Evidence for interaction effects. Journal of college student development 55(4):368-384. 7. Felsten G & Wilcox K (1992) Influences of stress and situation-specific mastery beliefs and satisfaction with social support on well-being and academic performance. Psychological Reports 70(1):291-303. 8. Pritchard ME & Wilson GS (2003) Using emotional and social factors to predict student success. Journal of college student development 44(1):18-28. 9. Zhang Z & RiCharde RS (1998) Prediction and Analysis of Freshman Retention. AIR 1998 Annual Forum Paper. 
    more » « less
  2. Introduction and Theoretical Frameworks Our study draws upon several theoretical foundations to investigate and explain the educational experiences of Black students majoring in ME, CpE, and EE: intersectionality, critical race theory, and community cultural wealth theory. Intersectionality explains how gender operates together with race, not independently, to produce multiple, overlapping forms of discrimination and social inequality (Crenshaw, 1989; Collins, 2013). Critical race theory recognizes the unique experiences of marginalized groups and strives to identify the micro- and macro-institutional sources of discrimination and prejudice (Delgado & Stefancic, 2001). Community cultural wealth integrates an asset-based perspective to our analysis of engineering education to assist in the identification of factors that contribute to the success of engineering students (Yosso, 2005). These three theoretical frameworks are buttressed by our use of Racial Identity Theory, which expands understanding about the significance and meaning associated with students’ sense of group membership. Sellers and colleagues (1997) introduced the Multidimensional Model of Racial Identity (MMRI), in which they indicated that racial identity refers to the “significance and meaning that African Americans place on race in defining themselves” (p. 19). The development of this model was based on the reality that individuals vary greatly in the extent to which they attach meaning to being a member of the Black racial group. Sellers et al. (1997) posited that there are four components of racial identity: 1. Racial salience: “the extent to which one’s race is a relevant part of one’s self-concept at a particular moment or in a particular situation” (p. 24). 2. Racial centrality: “the extent to which a person normatively defines himself or herself with regard to race” (p. 25). 3. Racial regard: “a person’s affective or evaluative judgment of his or her race in terms of positive-negative valence” (p. 26). This element consists of public regard and private regard. 4. Racial ideology: “composed of the individual’s beliefs, opinions and attitudes with respect to the way he or she feels that the members of the race should act” (p. 27). The resulting 56-item inventory, the Multidimensional Inventory of Black Identity (MIBI), provides a robust measure of Black identity that can be used across multiple contexts. Research Questions Our 3-year, mixed-method study of Black students in computer (CpE), electrical (EE) and mechanical engineering (ME) aims to identify institutional policies and practices that contribute to the retention and attrition of Black students in electrical, computer, and mechanical engineering. Our four study institutions include historically Black institutions as well as predominantly white institutions, all of which are in the top 15 nationally in the number of Black engineering graduates. We are using a transformative mixed-methods design to answer the following overarching research questions: 1. Why do Black men and women choose and persist in, or leave, EE, CpE, and ME? 2. What are the academic trajectories of Black men and women in EE, CpE, and ME? 3. In what way do these pathways vary by gender or institution? 4. What institutional policies and practices promote greater retention of Black engineering students? Methods This study of Black students in CpE, EE, and ME reports initial results from in-depth interviews at one HBCU and one PWI. We asked students about a variety of topics, including their sense of belonging on campus and in the major, experiences with discrimination, the impact of race on their experiences, and experiences with microaggressions. For this paper, we draw on two methodological approaches that allowed us to move beyond a traditional, linear approach to in-depth interviews, allowing for more diverse experiences and narratives to emerge. First, we used an identity circle to gain a better understanding of the relative importance to the participants of racial identity, as compared to other identities. The identity circle is a series of three concentric circles, surrounding an “inner core” representing one’s “core self.” Participants were asked to place various identities from a provided list that included demographic, family-related, and school-related identities on the identity circle to reflect the relative importance of the different identities to participants’ current engineering education experiences. Second, participants were asked to complete an 8-item survey which measured the “centrality” of racial identity as defined by Sellers et al. (1997). Following Enders’ (2018) reflection on the MMRI and Nigrescence Theory, we chose to use the measure of racial centrality as it is generally more stable across situations and best “describes the place race holds in the hierarchy of identities an individual possesses and answers the question ‘How important is race to me in my life?’” (p. 518). Participants completed the MIBI items at the end of the interview to allow us to learn more about the participants’ identification with their racial group, to avoid biasing their responses to the Identity Circle, and to avoid potentially creating a stereotype threat at the beginning of the interview. This paper focuses on the results of the MIBI survey and the identity circles to investigate whether these measures were correlated. Recognizing that Blackness (race) is not monolithic, we were interested in knowing the extent to which the participants considered their Black identity as central to their engineering education experiences. Combined with discussion about the identity circles, this approach allowed us to learn more about how other elements of identity may shape the participants’ educational experiences and outcomes and revealed possible differences in how participants may enact various points of their identity. Findings For this paper, we focus on the results for five HBCU students and 27 PWI students who completed the MIBI and identity circle. The overall MIBI average for HBCU students was 43 (out of a possible 56) and the overall MIBI scores ranged from 36-51; the overall MIBI average for the PWI students was 40; the overall MIBI scores for the PWI students ranged from 24-51. Twenty-one students placed race in the inner circle, indicating that race was central to their identity. Five placed race on the second, middle circle; three placed race on the third, outer circle. Three students did not place race on their identity circle. For our cross-case qualitative analysis, we will choose cases across the two institutions that represent low, medium and high MIBI scores and different ranges of centrality of race to identity, as expressed in the identity circles. Our final analysis will include descriptive quotes from these in-depth interviews to further elucidate the significance of race to the participants’ identities and engineering education experiences. The results will provide context for our larger study of a total of 60 Black students in engineering at our four study institutions. Theoretically, our study represents a new application of Racial Identity Theory and will provide a unique opportunity to apply the theories of intersectionality, critical race theory, and community cultural wealth theory. Methodologically, our findings provide insights into the utility of combining our two qualitative research tools, the MIBI centrality scale and the identity circle, to better understand the influence of race on the education experiences of Black students in engineering. 
    more » « less
  3. This work investigates how innovations propagate through two professional networks (guilds): the Kern Entrepreneurial Engineering Network (KEEN) and the Consortium to Promote Reflection in Engineering Education (CPREE). Previous research has demonstrated that the adoption of pedagogical innovations is supported by the socialization of the innovation among potential adopters. In this work, we use social network analysis to explore the impact of professional connections on innovation adoption. Our research questions are: (1) How does overall social structure differ between guilds? (2) How do measures of social network structures relate to innovation adoption? A survey was distributed to members of KEEN and CPREE to capture the interactions respondents had while adopting the guild’s innovation. Social networks were generated for each guild and each respondent. These networks were analyzed to identify relationships between social network measures and the frequency of use of the innovation. Responses to open-ended questions were analyzed using thematic coding. The guilds’ overall structures impacted the formation and structure of distinct clusters/cliques, but these differing structures did not appear to affect sustained adoption. Individuals’ ego networks demonstrated a weak negative correlation between the frequency of adoption and the individual’s ego network density. Our results imply that having a diverse network exposes instructors to more ideas or allows them to see one idea from many perspectives.

     
    more » « less
  4. An increasingly global environment expects graduating Engineering students to perform, live and work across cultures. Most intercultural competence research and associated global engineering education is focused on developing the global engineering skill set through long-term travel experiences such as study abroad programs. These programs can be expensive from both a time and money standpoint, limiting the participation to more privileged members of a community, and are not scalable to support broader participation. This work-in-progress addresses this research gap by focusing on the development of the students’ global learner mindset without requiring extensive travel. The project will investigate four different global engagement interventions, including the use of engineering case studies, the intentional formation of multi-national student teams, a Collaborative Online International Learning (COIL) research project, and a community engaged project within a short course. These interventions can be used to develop a holistic global learner mindset and global engineering education approach to foster global competence in undergraduate engineering students. The four global engagement interventions will be grounded in the global engineering competency (GEC) theoretical framework and assessed for their ability to foster a global learner mindset in engineering students. A mixed-methods approach will be used to assess students’ global learner mindset and skill set. This research will use the Global Engagement Survey (GES), the Global Engineering Competency Scale (GECS) and specific questions developed by the researchers to evaluate improvements in the participating students’ global engineering skill set and answer specific research questions including: 1) To what extent can global competence be developed in engineering students through the use of the proposed global engagement interventions; and 2) what are the relative strengths of each of the proposed global engagement interventions in developing global engineering competence? Combined, these research measures will provide both an accurate picture of how each global engagement intervention impacts the formation of a global learner mindset in engineering education, and also its associated ability to develop and/or improve global engineering skills. The outcomes of this study will generate valuable knowledge to understand how each global engagement intervention impacts the formation of global engineering competence. In this work-in-progress study, the authors discuss the four global engagement interventions with specific learning objectives that have been mapped to the overall student outcomes for the project. These objectives have also been mapped to the GES and GECS instruments. Finally the faculty members have developed qualitative tools to augment the GES and GECS to identify the global engineering skill sets each intervention is generating. This paper lays the foundation before implementing the interventions and performing their associated assessments over the several subsequent semesters. 
    more » « less
  5. An increasingly global environment expects graduating Engineering students to perform, live and work across cultures. Most intercultural competence research and associated global engineering education is focused on developing the global engineering skill set through long-term travel experiences such as study abroad programs. These programs can be expensive from both a time and money standpoint, limiting the participation to more privileged members of a community, and are not scalable to support broader participation. This work-in-progress addresses this research gap by focusing on the development of the students’ global learner mindset without requiring extensive travel. The project will investigate four different global engagement interventions, including the use of engineering case studies, the intentional formation of multi-national student teams, a Collaborative Online International Learning (COIL) research project, and a community engaged project within a short course. These interventions can be used to develop a holistic global learner mindset and global engineering education approach to foster global competence in undergraduate engineering students. The four global engagement interventions will be grounded in the global engineering competency (GEC) theoretical framework and assessed for their ability to foster a global learner mindset in engineering students. A mixed-methods approach will be used to assess students’ global learner mindset and skill set. This research will use the Global Engagement Survey (GES), the Global Engineering Competency Scale (GECS) and specific questions developed by the researchers to evaluate improvements in the participating students’ global engineering skill set and answer specific research questions including: 1) To what extent can global competence be developed in engineering students through the use of the proposed global engagement interventions; and 2) what are the relative strengths of each of the proposed global engagement interventions in developing global engineering competence? Combined, these research measures will provide both an accurate picture of how each global engagement intervention impacts the formation of a global learner mindset in engineering education, and also its associated ability to develop and/or improve global engineering skills. The outcomes of this study will generate valuable knowledge to understand how each global engagement intervention impacts the formation of global engineering competence. In this work-in-progress study, the authors discuss the four global engagement interventions with specific learning objectives that have been mapped to the overall student outcomes for the project. These objectives have also been mapped to the GES and GECS instruments. Finally the faculty members have developed qualitative tools to augment the GES and GECS to identify the global engineering skill sets each intervention is generating. This paper lays the foundation before implementing the interventions and performing their associated assessments over the several subsequent semesters. 
    more » « less