More Like this

1. A Revised Measure of Acceptance of the Theory of Evolution: Introducing the MATE 2.0

   https://doi.org/10.1187/cbe.21-05-0127  

   Barnes, M. Elizabeth; Misheva, Taya; Supriya, K.; Rutledge, Michael; Brownell, Sara E. (March 2022, CBE—Life Sciences Education)

   Romine, William (Ed.)

   Hundreds of articles have explored the extent to which individuals accept evolution, and the Measure of Acceptance of the Theory of Evolution (MATE) is the most often used survey. However, research indicates the MATE has limitations, and it has not been updated since its creation more than 20 years ago. In this study, we revised the MATE using information from cognitive interviews with 62 students that revealed response process errors with the original instrument. We found that students answered items on the MATE based on constructs other than their acceptance of evolution, which led to answer choices that did not fully align with their actual acceptance. Students answered items based on their understanding of evolution and the nature of science and different definitions of evolution. We revised items on the MATE, conducted 29 cognitive interviews on the revised version, and administered it to 2881 students in 22 classes. We provide response process validity evidence for the new measure through cognitive interviews with students, structural validity through a Rasch dimensionality analysis, and concurrent validity evidence through correlations with other measures of evolution acceptance. Researchers can now measure student evolution acceptance using this new version of the survey, which we have called the MATE 2.0. 

   more » « less
2. Developing a Measure of Engineering Students’ Makerspace Learning, Perceptions, and Interactions

   Lanci, Sarah; Nadelson, Louis; Villanueva, Idalis; Bouwma-Gearhart, Jana; Youmans, Kate; Lenz, Adam (June 2018, American Society of Engineering Education)

   Makerspaces have become a rather common structure within engineering education programs. The spaces are used in a wide range of configurations but are typically intended to facilitate student collaboration, communication, creativity, and critical thinking, essentially giving students the opportunity to learn 21st century skills and develop deeper understanding of the processes of engineering. Makerspace structure, layout, and use has been fairly well researched, yet the impact of makerspaces on student learning is understudied, somewhat per a lack of tools to measure student learning in these spaces. We developed a survey tool to assess undergraduate engineering students’ perceptions and learning in makerspaces, considering levels of students’ motivation, professional identity, engineering knowledge, and belongingness in the context of makerspaces. Our survey consists of multiple positively-phrased (supporting a condition) and some negatively-phrased (refuting a condition) survey items correlated to each of our four constructs. Our final survey contained 60 selected response items including demographic data. We vetted the instrument with an advisory panel for an additional level of validation and piloted the survey with undergraduate engineering students at two universities collecting completed responses from 196 participants. Our reliability analysis and additional statistical calculations revealed our tool was statistically sound and was effectively gathering the data we designed the instrument to measure. 

   more » « less
3. Towards a Survey Instrument for Use In Proactive Advising

   https://doi.org/10.18260/1-2--48167  

   West, Kenneth; Carroll, Bruce; Shin, Jinnie; Crippen, Kent (June 2024, ASEE Conferences)

   This paper introduces the pilot implementation of the Evidence Based Personas survey instrument for assessing non-cognitive attributes of relevance from undergraduate students at different stages of their engineering degree for the purpose of informing proactive advising processes. The survey instrument was developed with two key objectives: first, to assess its potential for streamlining and shortening existing instruments, and second, to explore the possibility of consolidating items from different surveys that measure the same or closely related constructs. A proactive advising system is being developed that uses the Mediation Model of Research Experiences (MMRE) as a framework. Within this framework, participation in various educational activities is linked to increased Commitment to Engineering via three mediating parameters: Self-Efficacy, Teamwork/Leadership Self-Efficacy, and Engineering Identity. The existing, validated MMRE survey instrument was used as a starting point for development of the current instrument with a goal of streamlining / shortening the number of questions. Ultimately, we envision augmenting the shortened instrument with items related to broader non-cognitive and affective constructs from the SUCCESS instrument. Noting that both the MMRE and SUCCESS instruments include measures of Self-Efficacy and Engineering Identity, selected questions from both were included and compared. Data was collected from 395 total respondents, and subsequent data analysis was based on 337 valid participants. Factor Analysis techniques, both exploratory and confirmatory, were employed to uncover underlying or latent variables within the results, particularly in the areas of Self-Efficacy where the combined items of the SUCCESS instrument and the MMRE instrument were used. Cronbach’s alpha analysis was employed to assess the internal consistency of the survey instrument. The Teamwork, Engineering Identity, and Commitment to Engineering constructs all produced a Cronbach’s alpha value in excess of 0.80. The Self-Efficacy construct fell below the 0.80 threshold at 0.77 which is considered to be respectable but is indicative of some short comings compared to that of the other constructs. The results of the EFA four-factor pattern matrix show the SUCCESS instrument items breaking out into their own components while the MMRE items merge with some of the items from the Engineering Identity construct suggesting a distinction in the underlying concepts these items may be measuring. This finding is further supported in the CFA through an assessment of the Goodness of Fit (GFI), Tucker-Lewis Index (TLI), and Root Mean Square Error of Approximation (RMSEA) of these constructs. The initial groupings of the four constructs produced a robust CFI value of 0.853, robust TLI value of 0.838, and a robust RMSEA value of 0.075. Self-Efficacy is broken out into two sub-scales one defined by the three items from the SUCCESS instrument and the other defined by the four remaining items from the MMRE instrument. Engineering Identity was also broken into two sub-scales. The robust CFI and TLI report values of 0.928 and 0.919 respectively, and the robust RMSEA is reported to be 0.053. The findings of the factor analyses indicate that a shortened form of the MMRE survey instrument will provide reliable measures of the underlying constructs. Additionally, the results suggest that the self-efficacy as measured by items from the MMRE and from the SUCCESS instruments are related to two separate aspects of self-efficacy and do not load well into a single factor. 

   more » « less
4. Barriers instructors experience in adopting active learning: Instrument development.

   https://doi.org/10.1002/jee.20577  

   Carroll, Laura J. Reeping (January 2024, Journal of Engineering Education)

   Abstract Background Despite well-documented benefits, instructor adoption of active learning has been limited in engineering education. Studies have identified barriers to instructors’ adoption of active learning, but there is no well-tested instrument to measure instructors perceptions of these barriers. Purpose We developed and tested an instrument to measure instructors’ perceptions of barriers to adopting active learning and identify the constructs that coherently categorize those barriers. Method We used a five-phase process to develop an instrument to measure instructors’ perceived barriers to adopting active learning. In Phase 1, we built upon the Faculty Instructional Barriers and Identity Survey (FIBIS) to create a draft instrument. In Phases 2 and 3, we conducted exploratory factor analysis (EFA) on an initial 45-item instrument and a refined 21-item instrument, respectively. We conducted confirmatory factor analysis (CFA) in Phases 4 and 5 to test the factor structure identified in Phases 2 and 3. Results Our final instrument consists of 17 items and four factors: (1) student preparation and engagement; (2) instructional support; (3) instructor comfort and confidence; and (4) institutional environment/rewards. Instructor responses indicated that time considerations do not emerge as a standalone factor. Conclusions Our 17-item instrument exhibits a sound factor structure and is reliable, enabling the assessment of perceived barriers to adopting active learning in different contexts. The four factors align with an existing model of instructional change in science, technology, engineering, and mathematics (STEM). Although time is a substantial instructor concern that did not comprise a standalone factor, it is closely related to multiple constructs in our final model. 

   more » « less
5. A Measure of Affect toward Key Elements of Engineering Professional Practice.

   Patrick, A D; Choe, N H; Martins, L L; Borrego, M J; Kendall, M R; Seepersad, C C (June 2017, ASEE annual conference & exposition proceedings)

   Identity, or how people choose to define themselves, is emerging as an attractive explanation for who persists in engineering. Many studies of engineering identity build off of prior work in math and science identity, emphasizing the academic aspects of engineering. However, affect towards professional practice is also central to engineering identity development. This paper describes the methods used to create a new survey measure of individuals’ affect toward elements of engineering practice. We followed the item generation, refinement, and instrument validation steps required for psychometric validation of a new survey measure. We generated items deductively using the literature on engineering professional skills and practice and inductively based on interviews with practicing engineers, engineering graduate students, and engineering undergraduate students. We blended the inductively and deductively derived item lists to create a list of initial items for the measure. We circulated this list of items to a set of engineering and professional identity experts to establish face validity and made modifications based on their feedback. The final list included 34 items. These 34 items were administered in a questionnaire survey in the fall of 2016 to 1465 engineering undergraduates in three majors at two institutions. We conducted an exploratory factor analysis (EFA) and established internal consistency using Cronbach’s alpha on a subset of the analytical sample data (n=384). The resulting factors fit our a priori assumption of the factors theorized to characterize affect towards engineering professional practice. Using the remaining data (n=904), we conducted a confirmatory factor analysis on the reduced set of items resulting from EFA. The results indicate an emergent factor structure for affect towards elements of engineering practice. 

   more » « less