skip to main content


Title: Towards Development of an Engineering Design Value-expectancy Scale (EDVES)
As high school programs are increasingly incorporating engineering content into their curricula, a question is raised as to the impacts of those programs on student attitudes towards engineering, in particular engineering design. From a collegiate perspective, there is a related question as to how first-year engineering programs at the college level should adapt to a greater percentage of incoming students with prior conceptions about engineering design and how to efficaciously uncover what those conceptions may be. Further, there is a broader question within engineering design as to how various design experiences, especially introductory experiences, may influence student attitudes towards the subject and towards engineering more broadly. Student attitudes is a broad and well-studied area and a wide array of instruments have been shown to be valid and reliable assessments of various aspects of student motivation, self-efficacy, and interests. In terms of career interests, the STEM Career Interest Survey (STEM-CIS) has been widely used in grade school settings to gauge student intentions to pursue STEM careers, with a subscale focused on engineering. In self-efficacy and motivation, the Value-Expectancy STEM Assessment Scale (VESAS) is a STEM-focused adaptation of the broader Values, Interest, and Expectations Scale (VIES), which in turns builds upon Eccles’ Value-Expectancy model of self-efficacy. When it comes to engineering design, there have been a few attempts to develop more focused instruments, such as Carberry’s Design Self-Efficacy Instrument. For the purposes of this work, evaluating novice and beginning designer attitudes about engineering design, the available instruments were not found to assess the desired attributes. Design-focused instruments such as Carberry’s were too narrowly focused on the stages of the design process, many of which required a certain a priori knowledge to effectively evaluate. Broader instruments such as the VESAS were too focused on working and studying engineering, rather than doing or identifying with engineering. A new instrument, the Engineering Design Value-Expectancy Scale (EDVES) was developed to meet this need. In its current form the EDVES includes 38 items across several subscales covering expectancy of success in, perceived value of, and identification with engineering and design. This work presents the EDVES and discusses the development process of the instrument. It presents validity evidence following the Cook validation evidence model, including scoring, generalization, and extrapolation validity evidence. This validation study was conducted using pre- and post-course deployment with 192 first-year engineering students enrolled in a foundational engineering design course.  more » « less
Award ID(s):
1812823
NSF-PAR ID:
10286684
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
2021 ASEE Virtual Annual Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. As more institutions create first year engineering programs that teach an engineering design process, there is a growing desire to prepare students for this coursework in the high school setting. When exposing such a broad population to these ideas, a primary question arises regarding student attitudes toward engineering and how these attitudes develop over time. That is, how does this exposure to engineering design influence student attitudes toward engineering? Moreover, answering this question will allow educators to better understand what motivates students to learn, how much their motivation impacts their overall mastery of these skills, and how these aspects of engineering self-efficacy and engineering design may differ between those who are on a pre-engineering track and those who are not. To begin answering this question, high school students enrolled in the Olathe City school system of Olathe, Kansas completed Engineering Problem-Framing Design Activities (EPDAs) in participating science courses (AP physics, physics, advanced biotechnology, chemistry, honors chemistry, biology, honors biology, and physical science specifically) of the traditional science and engineering academy curriculums offered by the district. Student engineering self-efficacy and motivation was also measured at the beginning and end of their coursework. This was conducted via a new instrument, the Engineering Design Value-Expectancy Scale (EDVES), which includes 38 items across three primary subscales: expectancy of success in, perceived value of, and identification with engineering and design. The development of this tool was presented and discussed in a previous study where the EDVES instrument was analyzed for validity among first-year undergraduate engineering students. In this work, the responses of high school students on the EDVES were analyzed to establish validity in this new population and to begin exploring trends in student responses based on their sub-population. Validity testing was completed via Cook’s validation evidence model with respect to scoring, generalization, and extrapolation evidence. The pre-course EDVES responses obtained were used to complete validation and trend analysis (note that post-course data was not readily available at the time of analysis). 
    more » « less
  2. There is little research or understanding of curricular differences between two- and four-year programs, career development of engineering technology (ET) students, and professional preparation for ET early career professionals [1]. Yet, ET credentials (including certificates, two-, and four-year degrees) represent over half of all engineering credentials awarded in the U.S [2]. ET professionals are important hands-on members of engineering teams who have specialized knowledge of components and engineering systems. This research study focuses on how career orientations affect engineering formation of ET students educated at two-year colleges. The theoretical framework guiding this study is Social Cognitive Career Theory (SCCT). SCCT is a theory which situates attitudes, interests, and experiences and links self-efficacy beliefs, outcome expectations, and personal goals to educational and career decisions and outcomes [3]. Student knowledge of attitudes toward and motivation to pursue STEM and engineering education can impact academic performance and indicate future career interest and participation in the STEM workforce [4]. This knowledge may be measured through career orientations or career anchors. A career anchor is a combination of self-concept characteristics which includes talents, skills, abilities, motives, needs, attitudes, and values. Career anchors can develop over time and aid in shaping personal and career identity [6]. The purpose of this quantitative research study is to identify dimensions of career orientations and anchors at various educational stages to map to ET career pathways. The research question this study aims to answer is: For students educated in two-year college ET programs, how do the different dimensions of career orientations, at various phases of professional preparation, impact experiences and development of professional profiles and pathways? The participants (n=308) in this study represent three different groups: (1) students in engineering technology related programs from a medium rural-serving technical college (n=136), (2) students in engineering technology related programs from a large urban-serving technical college (n=52), and (3) engineering students at a medium Research 1 university who have transferred from a two-year college (n=120). All participants completed Schein’s Career Anchor Inventory [5]. This instrument contains 40 six-point Likert-scale items with eight subscales which correlate to the eight different career anchors. Additional demographic questions were also included. The data analysis includes graphical displays for data visualization and exploration, descriptive statistics for summarizing trends in the sample data, and then inferential statistics for determining statistical significance. This analysis examines career anchor results across groups by institution, major, demographics, types of educational experiences, types of work experiences, and career influences. This cross-group analysis aids in the development of profiles of values, talents, abilities, and motives to support customized career development tailored specifically for ET students. These findings contribute research to a gap in ET and two-year college engineering education research. Practical implications include use of findings to create career pathways mapped to career anchors, integration of career development tools into two-year college curricula and programs, greater support for career counselors, and creation of alternate and more diverse pathways into engineering. Words: 489 References [1] National Academy of Engineering. (2016). Engineering technology education in the United States. Washington, DC: The National Academies Press. [2] The Integrated Postsecondary Education Data System, (IPEDS). (2014). Data on engineering technology degrees. [3] Lent, R.W., & Brown, S.B. (1996). Social cognitive approach to career development: An overivew. Career Development Quarterly, 44, 310-321. [4] Unfried, A., Faber, M., Stanhope, D.S., Wiebe, E. (2015). The development and validation of a measure of student attitudes toward science, technology, engineeirng, and math (S-STEM). Journal of Psychoeducational Assessment, 33(7), 622-639. [5] Schein, E. (1996). Career anchors revisited: Implications for career development in the 21st century. Academy of Management Executive, 10(4), 80-88. [6] Schein, E.H., & Van Maanen, J. (2013). Career Anchors, 4th ed. San Francisco: Wiley. 
    more » « less
  3. Need/Motivation (e.g., goals, gaps in knowledge) The ESTEEM implemented a STEM building capacity project through students’ early access to a sustainable and innovative STEM Stepping Stones, called Micro-Internships (MI). The goal is to reap key benefits of a full-length internship and undergraduate research experiences in an abbreviated format, including access, success, degree completion, transfer, and recruiting and retaining more Latinx and underrepresented students into the STEM workforce. The MIs are designed with the goals to provide opportunities for students at a community college and HSI, with authentic STEM research and applied learning experiences (ALE), support for appropriate STEM pathway/career, preparation and confidence to succeed in STEM and engage in summer long REUs, and with improved outcomes. The MI projects are accessible early to more students and build momentum to better overcome critical obstacles to success. The MIs are shorter, flexibly scheduled throughout the year, easily accessible, and participation in multiple MI is encouraged. ESTEEM also establishes a sustainable and collaborative model, working with partners from BSCS Science Education, for MI’s mentor, training, compliance, and building capacity, with shared values and practices to maximize the improvement of student outcomes. New Knowledge (e.g., hypothesis, research questions) Research indicates that REU/internship experiences can be particularly powerful for students from Latinx and underrepresented groups in STEM. However, those experiences are difficult to access for many HSI-community college students (85% of our students hold off-campus jobs), and lack of confidence is a barrier for a majority of our students. The gap between those who can and those who cannot is the “internship access gap.” This project is at a central California Community College (CCC) and HSI, the only affordable post-secondary option in a region serving a historically underrepresented population in STEM, including 75% Hispanic, and 87% have not completed college. MI is designed to reduce inequalities inherent in the internship paradigm by providing access to professional and research skills for those underserved students. The MI has been designed to reduce barriers by offering: shorter duration (25 contact hours); flexible timing (one week to once a week over many weeks); open access/large group; and proximal location (on-campus). MI mentors participate in week-long summer workshops and ongoing monthly community of practice with the goal of co-constructing a shared vision, engaging in conversations about pedagogy and learning, and sustaining the MI program going forward. Approach (e.g., objectives/specific aims, research methodologies, and analysis) Research Question and Methodology: We want to know: How does participation in a micro-internship affect students’ interest and confidence to pursue STEM? We used a mixed-methods design triangulating quantitative Likert-style survey data with interpretive coding of open-responses to reveal themes in students’ motivations, attitudes toward STEM, and confidence. Participants: The study sampled students enrolled either part-time or full-time at the community college. Although each MI was classified within STEM, they were open to any interested student in any major. Demographically, participants self-identified as 70% Hispanic/Latinx, 13% Mixed-Race, and 42 female. Instrument: Student surveys were developed from two previously validated instruments that examine the impact of the MI intervention on student interest in STEM careers and pursuing internships/REUs. Also, the pre- and post (every e months to assess longitudinal outcomes) -surveys included relevant open response prompts. The surveys collected students’ demographics; interest, confidence, and motivation in pursuing a career in STEM; perceived obstacles; and past experiences with internships and MIs. 171 students responded to the pre-survey at the time of submission. Outcomes (e.g., preliminary findings, accomplishments to date) Because we just finished year 1, we lack at this time longitudinal data to reveal if student confidence is maintained over time and whether or not students are more likely to (i) enroll in more internships, (ii) transfer to a four-year university, or (iii) shorten the time it takes for degree attainment. For short term outcomes, students significantly Increased their confidence to continue pursuing opportunities to develop within the STEM pipeline, including full-length internships, completing STEM degrees, and applying for jobs in STEM. For example, using a 2-tailed t-test we compared means before and after the MI experience. 15 out of 16 questions that showed improvement in scores were related to student confidence to pursue STEM or perceived enjoyment of a STEM career. Finding from the free-response questions, showed that the majority of students reported enrolling in the MI to gain knowledge and experience. After the MI, 66% of students reported having gained valuable knowledge and experience, and 35% of students spoke about gaining confidence and/or momentum to pursue STEM as a career. Broader Impacts (e.g., the participation of underrepresented minorities in STEM; development of a diverse STEM workforce, enhanced infrastructure for research and education) The ESTEEM project has the potential for a transformational impact on STEM undergraduate education’s access and success for underrepresented and Latinx community college students, as well as for STEM capacity building at Hartnell College, a CCC and HSI, for students, faculty, professionals, and processes that foster research in STEM and education. Through sharing and transfer abilities of the ESTEEM model to similar institutions, the project has the potential to change the way students are served at an early and critical stage of their higher education experience at CCC, where one in every five community college student in the nation attends a CCC, over 67% of CCC students identify themselves with ethnic backgrounds that are not White, and 40 to 50% of University of California and California State University graduates in STEM started at a CCC, thus making it a key leverage point for recruiting and retaining a more diverse STEM workforce. 
    more » « less
  4. The United Nation’s Sustainable Development Goals state climate change could irreversibly affect future generations and is one of the most urgent issues facing society. To date, most education research on climate change examines middle and high school students’ knowledge without considering the link between understanding and interest to address such issues in their career. In research on students’ attitudes about sustainability, we found that half of first-year college engineering students, in our nationally representative sample of all college students at 4-year institutions (n = 937), do not believe climate change is caused by humans. This lack of belief in human-caused climate change is a significant problem in engineering education because our results also indicate engineering students who do not believe in human caused climate change are less likely to want to address climate change in their careers. This dismal finding highlights a need for improving student understanding and attitudes toward climate change in order to produce engineers prepared and interested in solving complex global problems in sustainability. To advance understanding about students’ understanding of climate change and their agency to address the issue, we developed the CLIMATE survey to measure senior undergraduate engineering students’ Climate change literacy, engineering identity, career motivations, and agency through engineering. The survey was designed for students in their final senior design, or capstone course, just prior to entering the workforce. We developed the survey using prior national surveys and newly written questions categorized into six sections: (1) career goals and motivation, (2) college experiences, (3) agency, (4) climate literacy, (5) people and the planet, and (6) demographic information. We conducted focus groups with students to establish face and content validity of the survey. We collected pilot data with 200 engineering students in upper-level engineering courses to provide validity evidence for the use of these survey items to measure students and track changes across the undergraduate curriculum for our future work. In this paper, we narrate the development of the survey supported by literature and outline the next step for further validation and distribution on a national scale. Our intent is to receive feedback and input about the questions being asked and the CLIMATE instrument. Our objective is to share the nationally representative non-identifiable responses (the estimated goal is 4,000 responses) openly with education researchers interested in students understanding about climate change, their engineering identity, career motivations, and agency through engineering. Ultimately, we want this research to become a catalyst for teaching about topics related to climate change in engineering and its implications for sustainability. 
    more » « less
  5. The United Nation’s Sustainable Development Goals state climate change could irreversibly affect future generations and is one of the most urgent issues facing society. To date, most education research on climate change examines middle and high school students’ knowledge without considering the link between understanding and interest to address such issues in their career. In research on students’ attitudes about sustainability, we found that half of first-year college engineering students, in our nationally representative sample of all college students at 4-year institutions (n = 937), do not believe climate change is caused by humans. This lack of belief in human-caused climate change is a significant problem in engineering education because our results also indicate engineering students who do not believe in human caused climate change are less likely to want to address climate change in their careers. This dismal finding highlights a need for improving student understanding and attitudes toward climate change in order to produce engineers prepared and interested in solving complex global problems in sustainability. To advance understanding about students’ understanding of climate change and their agency to address the issue, we developed the CLIMATE survey to measure senior undergraduate engineering students’ Climate change literacy, engineering identity, career motivations, and agency through engineering. The survey was designed for students in their final senior design, or capstone course, just prior to entering the workforce. We developed the survey using prior national surveys and newly written questions categorized into six sections: (1) career goals and motivation, (2) college experiences, (3) agency, (4) climate literacy, (5) people and the planet, and (6) demographic information. We conducted focus groups with students to establish face and content validity of the survey. We collected pilot data with 200 engineering students in upper-level engineering courses to provide validity evidence for the use of these survey items to measure students and track changes across the undergraduate curriculum for our future work. In this paper, we narrate the development of the survey supported by literature and outline the next step for further validation and distribution on a national scale. Our intent is to receive feedback and input about the questions being asked and the CLIMATE instrument. Our objective is to share the nationally representative non-identifiable responses (the estimated goal is 4,000 responses) openly with education researchers interested in students understanding about climate change, their engineering identity, career motivations, and agency through engineering. Ultimately, we want this research to become a catalyst for teaching about topics related to climate change in engineering and its implications for sustainability. 
    more » « less