More Like this

1. Exploring the Validity of the Engineering Design Self-Efficacy Scale for Secondary School Students (Research To Practice)

   Lee, E.; Carberry, A. R.; Dalal, M.; Miller, M. J. (July 2021, 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference)

   null (Ed.)

   The purpose of this study is to re-examine the validity evidence of the engineering design self-efficacy (EDSE) scale scores by Carberry et al. (2010) within the context of secondary education. Self-efficacy refers to individuals’ belief in their capabilities to perform a domain-specific task. In engineering education, significant efforts have been made to understand the role of self-efficacy for students considering its positive impact on student outcomes such as performance and persistence. These studies have investigated and developed measures for different domains of engineering self-efficacy (e.g., general academic, domain-general, and task-specific self-efficacy). The EDSE scale is a frequently cited measure that examines task-specific self-efficacy within the domain of engineering design. The original scale contains nine items that are intended to represent the engineering design process. Initial score validity evidence was collected using a sample consisting of 202 respondents with varying degrees of engineering experience including undergraduate/graduate students and faculty members. This scale has been primarily used by researchers and practitioners with engineering undergraduate students to assess changes in their engineering design self-efficacy as a result of active learning interventions, such as project-based learning. Our work has begun to experiment using the scale in a secondary education context in conjunction with an increased introduction to engineering in K-12 education. Yet, there still is a need to examine score validity and reliability of this scale in non-undergraduate populations such as secondary school student populations. This study fills this important gap by testing construct validity of the original nine items of the EDSE scale, supporting proper use of the scale for researchers and practitioners. This study was conducted as part of a larger, e4usa project investigating the development and implementation of a yearlong project-based engineering design course for secondary school students. Evidence of construct validity and reliability was collected using a multi-step process. First, a survey that includes the EDSE scale was administered to the project participating students at nine associated secondary schools across the US at the beginning of Spring 2020. Analysis of collected data is in progress and includes Exploratory Factor Analysis (EFA) on the 137 responses. The evidence of score reliability will be obtained by computing the internal consistency of each resulting factor. The resulting factor structure and items will be analyzed by comparing it with the original EDSE scale. The full paper will provide details about the psychometric evaluation of the EDSE scale. The findings from this paper will provide insights on the future usage of the EDSE scale in the context of secondary engineering education. 

   more » « less
2. The Development of a Measure of Engineering Identity

   https://doi.org/10.18260/p.26122  

   Godwin, Allison (January 2016, ASEE Annual Conference & Exposition)

   Measures of subject-related role identities in physics and math have been developed from research on the underlying constructs of identity in science education. The items for these measures capture three constructs of identity: students’ interest in the subject, students’ feeling of recognition by others, and students’ beliefs about their performance/competence in the subject area. In prior studies with late secondary and early post-secondary students, participants did not distinguish between performance beliefs (e.g., believing that they can do well in a particular subject) and competence beliefs (e.g., believing that they can understand a particular subject); therefore, performance/competence beliefs are measured as a single construct. These validated measures have been successful in predicting STEM career choices including physics, math, and engineering. Based on these measures of identity, literature on engineering identity, and my prior work on understanding engineering choice and belongingness through students’ science and math identities at the transition from high school to college, I developed a set of new engineering identity measures that capture and overall identification as an engineer, future engineering career identification, and students’ engineering-related interest, recognition, and performance/competence beliefs. I conducted a pilot survey of 371 first-year engineering students at three institutions within the U.S. during the spring semester of 2015. An exploratory factor analysis (EFA) was performed to examine the underlying structure of the piloted questions about students’ engineering identity. The measures loaded on three separate constructs that were consistent with the hypothesized constructs of interest, performance/competence and recognition. The developed items were used in a subsequent study deployed in the fall semester of 2015 that measured more than 2500 first-year engineering students’ attitudes and beliefs at four institutions within the U.S. The data on engineering identity measures from this second survey were analyzed using confirmatory factor analysis (CFA). The results indicated that the developed measures do extract a significant portion of the average variance in the latent constructs and the internal consistency of the measures (Cronbach’s α) falls within the acceptable and better range. The development of these items provides ways for engineering education researchers to more deeply explore the underlying self-beliefs in students’ engineering identity formation through quantitative measures with strong evidence for validity. 

   more » « less
3. Innovation Self-Efficacy: A Very Brief Measure for Engineering Students.

   Schar, M.; Gilmartin, S.; Harris, A.; Rieken, B.; Sheppard, S. (January 2017, Proceedings for the American Society for Engineering Education Annual Conference, June 25-28. Columbus, OH.)

   When survey time is limited, education researchers may be faced with the choice of using an extremely brief measure of innovativeness or using no measure at all. To meet the need for a very brief measure, a 5-item innovation self-efficacy (ISE.5) scale was developed using the 19- item Dyer et al. Innovative Behavior Scale (IBS) as a starting point, adapted for undergraduate engineering students, and then condensed using confirmatory factor analysis. The ISE.5 measures innovation self-efficacy as a unitary construct drawn from Dyer et al.’s five innovative behavior components (Questioning, Observing, Experimenting, Networking Ideas and Associational Thinking) and has good internal and external validity as well as good test-retest reliability. The ISE.5 (as a measure of innovation self-efficacy) is shown to be an important mediator between innovation interests and a desire to pursue innovative work as a career postgraduation. This mediator relationship is consistent among important sub-populations of engineering students such as females, underrepresented minorities and first generation college students. While not a substitute for a full multi-factor innovation assessment tool, the ISE.5 can serve as an important indicator of innovation self-efficacy among an undergraduate engineering student population. 

   more » « less
4. Assessing Drawing Self-efficacy: A Validation Study Using Exploratory Factor Analysis (EFA) for the Drawing Self-efficacy Instrument (DSEI)

   https://doi.org/10.18260/1-2--36704  

   Jaison, Donna; Merzdorf, Hillary; Williford, Blake; White, Lance; Watson, Karan; Douglas, Kerrie; Hammond, Tracy (July 2021, 2021 ASEE Virtual Annual Conference)

   Drawing, as a skill, is closely tied to many creative fields and it is a unique practice for every individual. Drawing has been shown to improve cognitive and communicative abilities, such as visual communication, problem-solving skills, students’ academic achievement, awareness of and attention to surrounding details, and sharpened analytical skills. Drawing also stimulates both sides of the brain and improves peripheral skills of writing, 3-D spatial recognition, critical thinking, and brainstorming. People are often exposed to drawing as children, drawing their families, their houses, animals, and, most notably, their imaginative ideas. These skills develop over time naturally to some extent, however, while the base concept of drawing is a basic skill, the mastery of this skill requires extensive practice and it can often be significantly impacted by the self-efficacy of an individual. Sketchtivity is an AI tool developed by Texas A&M University to facilitate the growth of drawing skills and track their performance. Sketching skill development depends in part on students’ self-efficacy associated with their drawing abilities. Gauging the drawing self-efficacy of individuals is critical in understanding the impact that this drawing practice has had with this new novel instrument, especially in contrast to traditional practicing methods. It may also be very useful for other researchers, educators, and technologists. This study reports the development and initial validation of a new 13-item measure that assesses perceived drawing self efficacy. The13 items to measure drawing self efficacy were developed based on Bandura’s guide for constructing Self-Efficacy Scales. The participants in the study consisted of 222 high school students from engineering, art, and pre-calculus classes. Internal consistency of the 13 observed items were found to be very high (Cronbach alpha: 0.943), indicating a high reliability of the scale. Exploratory Factor Analysis was performed to further investigate the variance among the 13 observed items, to find the underlying latent factors that influenced the observed items, and to see if the items needed revision. We found that a three model was the best fit for our data, given fit statistics and model interpretability. The factors are: Factor 1: Self-efficacy with respect to drawing specific objects; Factor 2: Self-efficacy with respect to drawing practically to solve problems, communicating with others, and brainstorming ideas; Factor 3: Self-efficacy with respect to drawing to create, express ideas, and use one’s imagination. An alternative four-factor model is also discussed. The purpose of our study is to inform interventions that increase self-efficacy. We believe that this assessment will be valuable especially for education researchers who implement AI-based tools to measure drawing skills.This initial validity study shows promising results for a new measure of drawing self-efficacy. Further validation with new populations and drawing classes is needed to support its use, and further psychometric testing of item-level performance. In the future, this self-efficacy assessment could be used by teachers and researchers to guide instructional interventions meant to increase drawing self-efficacy. 

   more » « less
5. Towards a Survey Instrument for Use In Proactive Advising

   https://doi.org/10.18260/1-2--48167  

   West, Kenneth; Carroll, Bruce; Shin, Jinnie; Crippen, Kent (June 2024, ASEE Conferences)

   This paper introduces the pilot implementation of the Evidence Based Personas survey instrument for assessing non-cognitive attributes of relevance from undergraduate students at different stages of their engineering degree for the purpose of informing proactive advising processes. The survey instrument was developed with two key objectives: first, to assess its potential for streamlining and shortening existing instruments, and second, to explore the possibility of consolidating items from different surveys that measure the same or closely related constructs. A proactive advising system is being developed that uses the Mediation Model of Research Experiences (MMRE) as a framework. Within this framework, participation in various educational activities is linked to increased Commitment to Engineering via three mediating parameters: Self-Efficacy, Teamwork/Leadership Self-Efficacy, and Engineering Identity. The existing, validated MMRE survey instrument was used as a starting point for development of the current instrument with a goal of streamlining / shortening the number of questions. Ultimately, we envision augmenting the shortened instrument with items related to broader non-cognitive and affective constructs from the SUCCESS instrument. Noting that both the MMRE and SUCCESS instruments include measures of Self-Efficacy and Engineering Identity, selected questions from both were included and compared. Data was collected from 395 total respondents, and subsequent data analysis was based on 337 valid participants. Factor Analysis techniques, both exploratory and confirmatory, were employed to uncover underlying or latent variables within the results, particularly in the areas of Self-Efficacy where the combined items of the SUCCESS instrument and the MMRE instrument were used. Cronbach’s alpha analysis was employed to assess the internal consistency of the survey instrument. The Teamwork, Engineering Identity, and Commitment to Engineering constructs all produced a Cronbach’s alpha value in excess of 0.80. The Self-Efficacy construct fell below the 0.80 threshold at 0.77 which is considered to be respectable but is indicative of some short comings compared to that of the other constructs. The results of the EFA four-factor pattern matrix show the SUCCESS instrument items breaking out into their own components while the MMRE items merge with some of the items from the Engineering Identity construct suggesting a distinction in the underlying concepts these items may be measuring. This finding is further supported in the CFA through an assessment of the Goodness of Fit (GFI), Tucker-Lewis Index (TLI), and Root Mean Square Error of Approximation (RMSEA) of these constructs. The initial groupings of the four constructs produced a robust CFI value of 0.853, robust TLI value of 0.838, and a robust RMSEA value of 0.075. Self-Efficacy is broken out into two sub-scales one defined by the three items from the SUCCESS instrument and the other defined by the four remaining items from the MMRE instrument. Engineering Identity was also broken into two sub-scales. The robust CFI and TLI report values of 0.928 and 0.919 respectively, and the robust RMSEA is reported to be 0.053. The findings of the factor analyses indicate that a shortened form of the MMRE survey instrument will provide reliable measures of the underlying constructs. Additionally, the results suggest that the self-efficacy as measured by items from the MMRE and from the SUCCESS instruments are related to two separate aspects of self-efficacy and do not load well into a single factor. 

   more » « less