Instructional experimental and demonstration laboratories have been an essential part of the undergraduate engineering curriculum to varying degrees throughout the history of the engineering profession in the United States. Since the founding of the engineering discipline in the U.S. in 1802 at the U.S. Military Academy at West Point, NY, engineering instructional laboratories have been the foundation of undergraduate engineering education. In the middle and nineteenth centuries, these laboratories were coupled with fieldwork, drafting, mathematics, and science as more schools joined the practice of training engineers, e.g., Norwich University (1821), Rensselaer Institute (1835), Yale (1852), MIT (1865), Union College (1845), and Cornell (1830) [1]- [3]. During these early years, universities built physical infrastructures to house engineering laboratories aligned with and reflected realistic work environments. This form of practical and real-world informed instruction continued until the end of World War II when it was discovered that Int J Eval & Res Educ ISSN: 2252-8822
Student perceived value of engineering labs: a lab assessment instrument (Kimberly Cook-Chennault) 2149 scientists, rather than engineers, had developed most inventions during the war. As a result of this discovery, the American Society of Engineering Education (ASEE) formed a committee to study the problem. This committee concluded that the lack of innovation from engineers during World War II was because the engineering course curriculum was too "practically oriented" and not designed to give future engineers the skills needed to solve engineering problems using first principles. The committee summarized its findings within an infamous report called the Grinter Report [4], which called for strengthening the requirements for engineers in basic sciences, mathematics, chemistry, and physics.
In response to the Grinter Report, universities increased the theoretical content included within the engineering curriculum. This report also led to the establishment of two distinct disciplines: engineering technologists and engineers, whose course curriculum was regulated by the engineers' council for professional development, which was the precursor to the Accreditation Board for Engineering and Technology (ABET) [5]. Though well-intentioned, the heightened emphasis on theoretical concepts in the engineering curriculum and diminished investment in instruction labs during this period led to the graduation of many engineers with little practical or laboratory experience, which led to challenges for matriculated students entering the engineering field. Since then, ASEE has produced several reports affirming the importance of laboratory instruction for undergraduate curriculum, along with recommendations for best practices, e.g., reports in 1967, 1986, 1987 [2], [4], [6]. However, in the late 1980s, many scholars found that the undergraduate foundational engineering laboratories needed an overhaul [7]. Despite this need, studies pertaining to instructional in-person physical engineering labs waned [1] and have continued through the 2000s. Instead, laboratory research has primarily focused on developing virtual lab tools and technologies [8]- [13]. Though instructional labs are required for all engineering disciplines, mechanisms, and assessment instruments for understanding students' perspectives of these learning experiences for in-person and virtual labs of the 21st century, they have not caught up with the new forms of communication, tools, and educational norms. Hence, this work is novel because it initiates the steps toward assessing 21st labs in higher education for students engaged in engineering educational laboratories.
The role that instructional engineering labs play in developing engineers is essential as these labs reaffirm theoretical and foundational engineering concepts/principles and should provide a meaningful link to aspects of the engineering profession [14]. Cultivating students' authentic knowledge of the engineering profession is vital because many undergraduate engineering students have higher self-proclaimed levels of professional engineering identity than their developmental levels actually are [15]. This misalignment of skill proficiency with perceived knowledge can stifle students' continuation and success in engineering. Further, the literature suggests that students' misunderstanding of the scope and work of 21st engineers decrease the likelihood of them staying in the engineering field after matriculation [15].
While engineering laboratories are needed in higher education to affirm students' learning and application of theoretical concepts, there are few (if any) validated assessment instruments that can be used to examine the usefulness and effectiveness of these types of labs in conventional 21st century educational inperson laboratory environments. Thus, this work addresses the need for an educational assessment tool for 21st [15]- [20] in-person engineering educational laboratories, where there is a gap in the literature. This article describes the development and validation of an assessment tool, the Student Perceived Value of an Engineering Laboratory (SPVEL) assessment instrument, for application to in-person laboratory environments. The purpose of this study is to validate this instrument for in-person physical laboratories, so that it may be used by laboratory instructors and researchers to garner students' perceptions of in-person laboratories taken as part of an engineering curriculum. The theoretical frameworks that inform this educational laboratory assessment instrument are the technology acceptance model (TAM) [21], [22], Astin's input-environment-output conceptual model [23], [24], and the conceptual model of engineering role identity [25]. In particular, this assessment instrument is unique because it may be used to compare the effectiveness of in-person and virtual laboratory environments and is premised on three frameworks that have not been used prior to this work. An exploratory factor analysis (EFA) was conducted on a questionnaire informed by these three theoretical models to validate it as an assessment instrument. The responses from 208 undergraduate mechanical and aerospace engineering students were used in this study. This study also builds upon previous work [8] that found that traditional course evaluation instruments for 21st engineering labs generally lacked meaningful information about students' experiences of the laboratory environment and focused primarily on the assessment of the instructor. The general course evaluation surveys did not recognize students' selfefficacy and prior experiences with engineering equipment and technology. The student perceived value of an engineering laboratory (SPVEL) questionnaire developed for this study was used as a feedback mechanism for mechanical and engineering virtual and in-person labs that took place in the School of Engineering at a university in the Northeastern region of the United States. This study was approved by a university Internal Review Board (IRB) for students to participate in a multiple-year study about their experiences in educational labs that covered multiple topics over an academic year. The benefit of this assessment tool is that it allows for comparison of strengths and weaknesses of learning environments and can be used to tailor experiential learning labs to meet student needs and diverse learning platforms. This study is novel because it describes the design and validation of an assessment instrument for engineering laboratories that provides a mechanism for: i) elucidating student's perceptions of their laboratory experiences; ii) examining how these experiences influence their engineering role identities; and iii) elucidating how students value laboratory experiences as preparatory and reflective of the skills they need for professional careers in engineering, which has not been done for in-person 21st century laboratories to date.
The technology acceptance model (TAM), developed by Davis [21], [22], posits that peoples' adoption of information technological systems is related to their belief in a system's perceived usefulness and their perceptions of the system being easy to use. In other words, people will use or not use an application/tool to the degree that they deem the tool will help them do their jobs better [21]. The TAM further postulates that if people believe the effort required to use a tool is too high or consider the benefits of its use less than the effort of use, they will abandon the use of the technology. The TAM has been used to explore undergraduate students' acceptance of engineering games within the classroom [26], mobile and elearning tools [27], [28] in higher education, and hybrid and virtual laboratories [12], [29]- [31].
Most researchers assert that the TAM is most effective when other variables are considered. When studying virtual laboratories, Raikar et al. [30] concluded that undergraduates (UGs) chose to engage with virtual labs based on their ease of use, perceived usefulness, in addition to their prior knowledge of materials related to the virtual labs. It was also concluded that UGs with more prior experience achieved better grades in the course that incorporated virtual labs and associated higher value to the use of VLs, than those who did not have similar prior knowledge. Likewise, Estiegana et al. [12] used the TAM to examine students' acceptance of VLs and interactive activities and concluded that perceived efficiency, expectation, and satisfaction were crucial factors to consider when using the TAM. Other scholars have found that undergraduate engineering students associate more value, i.e., usefulness from educational technologies that allow them to connect their real-world experiences and theoretical knowledge to their perceptions of the realworld engineering profession [32]. Few researchers (if any) have used this model to understand students' acceptance of in-person laboratory technologies and equipment until this work. However, understanding students' perceptions of value associated with learning in physical laboratories helps instructors and employers anticipate the needs of students as they prepare them for the engineering profession.
In the 1970's Astin established the input-environment-output conceptual model [23], [24] to help educators reduce biases in assessing the outputs, e.g., academic performance resulting from college environments and educational interventions. Astin posited that differences in inputs, i.e., what students bring to the learning situation, or social identities should be considered in predicted and understanding outputs because these things influence how an intervention or environment is experienced, facilitated, or perceived by students. Since then, the IEO conceptual model has been used to evaluate and assess programmatic interventions and their relationship to student success as a function of input variables such as learning disabilities [33], [34], amount and quality of time of involvement [35], perceived academic ability and drive to achieve [36], in undergraduate and postsecondary levels. The role of gender and race in the prediction of gender-role traditionalism [37], feminist identity and program characteristic roles in social advocacy [38] and differences in the transition of black and white students from high school to college [39]. Less than a handful of researchers have used this model to understand outcomes in engineering, though the engineering community is beginning to understand the importance of considering student inputs and environment as described by the IEO model in assessment of engineering curriculum. For example, Broeck et al. [40] used the IEO model to explore differences in dropout and academic achievement of traditional versus lateral entrance engineering students at Katholieke Universiteit Leuven in Belgium. In this study, the input variables were prior education and study patterns, and it was concluded that both groups had similar drop-out rates and academic achievement. It was determined that these similar trends were due to mandatory curriculum course work that was required for lateral (bridged) students to enter the program [40].
Understanding how the IEO inputs and environments influence students' outcomes as a result of engaging in undergraduate engineering laboratories is important for 21st century engineering curriculum development that appropriate addresses the needs of students who have been exposed to different forms of social media and electronic technologies, compared to the early years of engineering curricular development in the late 1990's and early 2000's. Int J Eval & Res Educ ISSN: 2252-8822 Student perceived value of engineering labs: a lab assessment instrument (Kimberly Cook-Chennault) 2151
The concept of engineering identity, e.g., a form of role identity that students form as they gain experiences working in a community of practice and in the college environment [41] is an area of intense study as scholars seek to better understand how engineers are formed, and to what extent social, environmental, and educational factors influence "Who gets to be an engineer?" [42]. Godwin and Kirn [25] defined engineering role identity as how students describe themselves and are positioned by others into the role of an engineer. Engineering role identity is premised on three elements: dialogic [43] communication that relies on the social perspective of the student, the student's interest in the subject and beliefs about their competence relating to the subject [44], [45]; and the student's comprehension of concepts, and ability to connect new knowledge to prior information (cognitive perspective) [46], [47].
Many studies have shown engineering identity as a predictor of students' educational and professional persistence. Others have extended this model to include resilience to consider intersectional social identities, e.g., black women engineers [48]. Most of these studies have focused on how students' perception of their engineering role identity is related to their culture and enacting the qualities, they believe are required for being an engineer [44], [45]. In this context, the development of an instrument that considers students' identity in the design and evaluation of a learning environment/tool/technology has immense value.
Relating the formation of one's engineering identity to the value obtained from a curricular intervention should in theory helps the educator illuminate the ways students describe themselves and their experiences with educational laboratories, how they value/or not the laboratories in their learning, and how they affirm/build upon engineering concepts as they engage in educational laboratory.
Understanding the interrelationship between one's identity and their persistence in the science, technology, engineering, and mathematics (STEM) educational process and formation into an engineer has been a subject of many researchers over several decades, where differences between subgroups (race, gender, socioeconomic, sexuality) and the traditional stereotypical white/Asian masculine culture of engineering have been noted [49], [50]. For example, Pierrakos et al. [44], [45] used the social identity theory described by Deaux et al. [51], [52] to understand how students identify as engineers as a function of gender. It was found that there are significant gender differences in how first-year students identify with engineering and becoming an engineer, where fewer women were exposed to the engineering field through applied or building experiences (0% women to 26% men); interactions with relatives who were engineers (20% women to 26% men) and STEM activities (10% women to 26% men) [45]. Thus, the SPVEL instrument was developed to relate how students grow/or do not grow their engineering role identities, accept/or not new technology and systems of the educational lab, and use their prior experiences in and outside of the academe to learn in the engineering college environment.
A mixed-method convergent research design method [53] was proposed and approved by the primary Institutional Review Board of the first author. The study took place at a Research-1 [54], researchintensive institution in the Northeastern region of the United States. The data described herein represents phases of a multi-year study in the 2020 to 2022. Participants in the study (N=208) were recruited to participate from a mechanical and aerospace undergraduate engineering laboratory course that took place in the 2021 to 2022 academic school year when the laboratory was offered in-person following the COVID-19 pandemic.
Students who participated in this study were all undergraduate engineering students who were enrolled in a mechanical and aerospace engineering laboratory. The labs were designed to be either demonstration labs or hands-on labs that students engaged in during the lab section. There were 217 students participated in the study by submitting responses to a pre-lab and/or a post-lab questionnaire. A total of nine participants neglected to complete either the pre-or post-lab questionnaires and were removed from the analysis, resulting in 208 participants for the data analysis.
Due to the considerable number of students enrolled in the course, students were divided into multiple sections and were rotated to different labs that occurred simultaneously through the course semester. Students participated in one introductory laboratory lecture that discussed course objectives, design, and expectations. Before engaging in laboratory activities, students completed a pre-lab questionnaire. After finishing the pre-lab questionnaire, students downloaded and observed a pre-recorded video lecture that described the theoretical concepts covered in each lab. These recorded lectures were created by instructors who taught the theory associated with the lab in the technical courses. These technical courses were prerequisites to the senior educational engineering lab. Students were also provided with equipment manuals This study focuses on an in-person hands-on laboratory where students were paired into groups of 3 to 4 and seated at tables equipped with a computer and lab equipment (data acquisition board and function generator). Students followed the instructions for the lab as described by the teaching assistant (TA) during the lab and used the lab manual as needed to troubleshoot difficulties associated with carrying out operations described in the lab manual. A schematic of the lab set up is provided in Figure 1. As shown in this figure, students were placed into teams at lab station tables, where they recreated the demonstration lab along with the TA. The data evaluated in this work comes from students who participated in a LabView laboratory in an in-person setting.
An ideal exploratory load factor analysis requires "strong data" to obtain high strength item loadings, uniformity of the communalities, and a suitable number of items per factor [55], [56]. These previous items are vital for the stability, reliability, and replicability of the factor solution predicted by the analysis. To ascertain the minimum number of participants for this study, the N:p ratio, i.e., ratio of the number of participants (N) to the number of items in the questionnaire (p) was used [57]. According to previous studies [57], [58], the higher the ratio (more data per item), the higher the power to detect meaningful relationships between items, where the minimum acceptable ratio is 5:1. In this study, 175 participants are required for an exploratory load factor analysis of a questionnaire consisting of 35 questions to achieve a N:p=5:1. However, 208 participants engaged in this study where the original questionnaire consisted of 35 items. For a 35-item questionnaire, with 208 participants, N:p=6:1 ratio, which is higher than the minimum suggested ratio of 5:1. In addition, after questions with factor loading coefficients less than |0.4| were removed from the 35 questions, only 25 items remained in the questionnaire, resulting in a N:p ratio equal to 8:1. A N:p equal to 8:1 also adheres to the minimum participant number (100-250) guidance posited by previous studies [57], [59].
A multiple item questionnaire was created for this project called the student perceived value of an engineering laboratory (SPVEL) assessment questionnaire that was designed to leverage three theoretical models, i.e., the technology acceptance model [21], [22], inputs-environment-outcome (IEO) conceptual 2153 model [35], [60], and engineering role identity [25], [48], [61]. The original draft of the questionnaire (prior to the application of the exploratory load factor analysis comprised 35 items from a combined pre-and postlaboratory questionnaire. Questions 8-35 were rated on a Likert-type scale that ranged from 1 to 5, where 1, 2, 3, 4, and 5, referred to "strongly disagree", "somewhat disagree", "neither agree nor disagree", "somewhat agree", and "strongly agree", respectively. The other items in the questionnaire were scaled according to number of occurrences/experiences and hours of participation. The descriptive statistics, e.g., mean, and standard deviations for the responses to the questions are presented in the results and discussion section.
The process for validating the SPVEL instrument for in-person laboratories consisted of applying an exploratory factor analysis (EFA) on the questionnaire items using an IBM SPSS (version 28) to perform the statistical calculations. Principle axis factoring was used to extract the factors under the Varimax rotation method. Under the coefficient format, the absolute value for loadings was set to 0.40 and weak loadings below this cut-off number were neglected. Additionally, any factors with less than 3 items were considered weak and unstable [55], [62] and thus, excluded from further analyses for the Cronbach's alpha calculations.
As a dimension reduction procedure, exploratory factor analysis was conducted to investigate the factor structure underlying the responses from the questionnaire. Principal axis factoring was used to extract the factors, and the squared multiple correlations were used as prior communality estimates. A Kaiser-Meyer-Olkin (KMO) test was also performed to validate that an appropriate number of sampling sizes were used in the study. In particular, this statistic (ranges from 0.0 to 1.0) was used to measure the proportion of variance among variables that may be common variance, which determines if the data is suitable for factor analysis, where values greater than or equal to 0.7 indicate suitable data [63]. A Barlett's test for sphericity was performed to determine whether the data had an adequate number of correlations. In other words, this test was conducted to check whether there was redundancy between variables, where a value of less than or equal to 0.05 indicates that the correlation matrix is not the identity matrix [63]. Finally, a scree plot containing the eigenvalues of the factors arranged in descending order of magnitude was used to ascertain the most meaningful factors of the structure [64].
The reliability of the entire questionnaire and subsequent factor loadings was assessed via Cronbach's alpha () to ascertain the strength of the consistency in the questionnaire questions and the loadings for measuring the concepts. To interpret Cronbach's alpha, a score between 0.7-0.95 considered very high and indicative of questionnaire items and loading factors that possess high test-retest reliability and internal consistency (connected to the inter-relatedness of the items in the test). While Cronbach alpha scores between 0.55 and 0.70 are considered acceptable, those that are less than 0.55 are not [65], [66]. Specifically, a Cronbach alpha score that is less than 0.55 may indicate two common issues: i) low number of questions and hence poor inter-relatedness between the items; and ii) multiple-choice questions that have only two or three choices of responses, which generally have lower reliability scores compared to Likert style questions that have five to seven response choices [66].
The racial and ethnic demographics of the students who participated in this study are provided in Table 1 and Table 2. The demographics of the student population presented in these tables demonstrate that the racial and ethnic groups are similar in percentage to the national averages recorded by the ASEE (engineering by the numbers report) [67]. Women represent ~14% of the participants in this study, which is slightly less than the national average values for mechanical engineering (15.7%) women graduates. Similarly, the percentage of LatinX participants in this study, e.g., 11%, is close to the percentage of graduating students nationally for all engineering majors, i.e., 12.1%. Lastly, the number of black/African American participants, e.g., 2%, is less the national average values (4.5%). The demographical, social identity information presented in Table 1 and Table 2 were captured as part of the I-E-O Conceptual model as inputs that describe aspects of student identity that inform their academic and personal environment, access, and previous experiences.
The descriptive statistics, i.e., the statistical mean (Mean, M) and standard deviation (STDEV) for the questionnaire questions based on the responses from 208 participants are provided in Table 3 and Table 4. In these tables, the notation, "+", indicates that approximately two thirds of the population responses will lie (i.e., have mean values) within plus (+) or minus (-) one standard deviation of the mean. In addition, the theoretical model associated with questionnaire item is also presented in these tables. Since these students were re-entering the classroom after nearly two years of remote learning, questions pertaining to their virtual lab and in-person laboratory experiences are also included within the questionnaire items in the pre-lab questionnaire. The post-lab questions all focused on the students' hands-on laboratory experiences and perceptions.
An exploratory factor analysis was conducted to investigate the factor structure underlying the responses to the questionnaire that originally comprised 35 items and then was reduced to 25 items. The remaining 25 items loaded into five factors. Descriptive statistics for the pre-and post-lab questions, i.e., the mean and standard deviations for each response are provided in Table 3 and Table 4.
A normality test was conducted for each item in the questionnaire, and it was determined that the distribution of the responses was skewed and did not follow a normal distribution. Hence, a maximum likelihood estimator (used for normal distribution responses) was not used for estimating parameters. Instead, the data was treated as categorical data, which are ordered and non-normal [64]. The factor structure of the latent variables was estimated with the aid of SPSS software where squared multiple correlations were used as prior communality estimates. Polychoric correlation factors [68] were calculated from the categorical variables. This correlation matrix indicated that both positive and negative correlations existed in the data, where they ranged from -0.578 to 0.835. The range of the correlation coefficients indicated that the putative factors from the EFA were not independent. None of the correlations in the original matrix exceeded 0.85, thus multicollinearity was not observed, i.e., no two items measured the same aspect of the construct. Also, the determinant of the matrix was found to be greater than 0.0001 [64], [69], which supports the further use of the data set for EFA and principal component analysis reduction methods for this study. Three additional tests, i.e., Kaiser-Meyer-Olkin, Bartlett, and Scree Plot, were conducted to affirm the viability of using the data set for EFA and Principal Component Analysis (PCA) analyses. The results from the Kaiser-Meyer-Olkin (KMO) and Bartlett test are provided in Table 5.
A KMO test was performed to validate that an appropriate number of sample sizes were used in the study, e.g., sampling adequacy. A total of 217 students participated, however, only 208 of the participant data was used as incomplete surveys were discarded from the analysis. The KMO for this work was calculated to be 0.866 as shown in Table 5. Since KMO is equal to 0.866, this indicates that sample size is sufficient for factor analysis. Bartlett's Test for Sphericity was conducted to test the null hypothesis that the correlation matrix is an identity matrix. As shown in Table 5, sphericity significance was determined to be <0.001, which confirms that there are an adequate number of correlations between variables to conduct an EFA [63]. To extract the number of factors underlying the data, two criteria were used: the point of inflection from the scree plot [70] and the number of eigenvalues greater than 1.0 [70], [71]. The Scree Plot containing the eigenvalues of the factors arranged in descending order of magnitude for the data for this study is provided in Figure 2 and was used to ascertain the most meaningful factors of the structure [64]. Five factors were identified using this extraction method, which is used to define the putative factor structure for the SPVEL instrument. Once the putative factor structure was identified, factor loadings were analyzed and reduced using the principal component analysis (PCA) method [72].
Figure 2. Scree plot of the questionnaire questions A principal component analysis (PCA) method was used to extract, define, and reduce the factor loadings, where the squared multiple correlations were used as prior communality estimates to extract the factors for this analysis. A rotated orthogonal matrix (Varimax with Kaiser Normalization) [73] and communalities were used to ascertain the loading of factors, where items with factor loading coefficients greater than |0.4| were considered significant for a specific factor, and those less than |0.40|, were removed. This process of analysis was repeated to optimize loading coefficient values and communality values and minimize the instances of cross-loading of variables onto multiple factors. The final rotation converged in ten iterations, where questions were removed using this reduction and extraction method to enhance overall reliability and robustness of the instrument. Several questions were removed because they contained a limited number of choice options for participant responses, i.e., less than five response choices. The percentage of variance and rotation sums of squared loading for the validated instrument are presented in Table 6, where 25 items are structured along five factors representing 71.41% of variance. Cronbach's alpha was calculated for each factor to assess the reliability of the loading associated with the group. The final loading factor structure for the labs is presented in the Table 7, where five factors were observed.
The analysis of the data initiated by ascertaining the reliability of the entire questionnaire via Cronbach's alpha to ascertain the strength of the consistency in the questionnaire factor loadings. Since all remaining questions within the SPVEL instrument are based on a 5-point Likert Scale, a valid reliability analysis could be conducted. The scores for each of the factor loadings are provided in Table 7. These factors range from 0.65 to 0.930. These high scores provide sufficient evidence that the test-retest reliability of the combined questionnaire is remarkably high, and the internal consistency of the items are high as well. .830 Q30: This Lab helped me make the connections between previous course concepts.
.828 Q28: This Lab helped me to understand concepts from my previous courses.
.822 Q35: I think the skills I learned in this lab will be useful in my future career.
.776 Q34: The LabView Lab was a good learning experience.
.747 .409 Q21: This Lab adequately prepared me to write my final report.
.538 .466 Q22: TAs effectively answered questions during the lab.
.851 Q19: I was able to communicate with the TAs during the lab.
.850 Q16: The lab was easy to understand.
.836 Q17: I could follow the steps in the lab.
.818 Q18: The lab held my attention for the full duration of the time.
.434 .709 Q23: The operations performed in the lab were easy to follow.
.407 .592 .468 Q32: I was able to interpret the data from the lab using only resources provided in the class.
.
.571
Q27: I can learn as much in virtual labs as in hands-on-labs.
.828 Q8: Virtual labs can be good learning tools.
.820 Q9: Virtual labs can replace hands-on-labs.
.803 Q12: The skills from virtual labs will be useful in my career.
.788 Q11: I think I learned as much from this hands-on as I would have learned in a virtual (remote) lab.
.503 Q26: I was able to ask the TA questions orally (live) during the lab.
.564 .706 Q20: Class ran smoothly with no technical glitches.
.706 Q25: I was able to ask questions in during the lab.
.617 .675 Q15: My friends see me as an engineer.
.874 Q14: Being an engineer is an important part of my self-image.
.820 Q13: I understand concepts that I have studied in engineering.
.613 Extraction method: Principal Component Analysis; Rotation Method: Varimax with Kaiser Normalization. a. Rotation converged in 5 iterations.
An exploratory factor analysis approach was used to decipher five primary factors including 25 questions from the original set of 35. Load factor one describes student perceptions of laboratory educational value towards enhancing students' skillset and reinforcement/enhancement of theoretical content taught in previous engineering classes. The second load factor describes how students accepted/or not the laboratory environment, ease of use of equipment, software, educational tools, which are informed from the technology acceptance model. Inclusive in this factor is the instructor's ability to effectively guide and answer questions about the lab. The third load factor describes students' perception of the viability of virtual lab learning environment as a learning tool. While the labs that took place for this study were in person, the perception of the virtual/remote laboratory learning experience is examined via the questions loading under this factor. Load factor four describes the effectiveness of the communication between students and the instructor and the operation of the laboratory equipment and environment. The fifth load factor describes students' engineering role identities.
In our previous work [8], questions from a conventional course evaluation instrument were examined, where it was concluded that these forms of traditional course assessment instruments tended to be more instructor focused, rather than equal observation of student engagement and perceived value towards achieving career and learning goals. Hence, it was concluded that many traditional course instruments did not fully explore students' perceptions regarding in-person learning laboratory environments to the extent needed to understand how they inform the professional development of engineering students of the 21st century. Hence, the goal of this work is to validate an assessment instrument that both examines the effectiveness of the environment learning experience, and ability to encourage/motivate further exploration of the engineering discipline, while also affirming and enhancing students' engineering role identities, which are all needed for students to continue in the engineering field. Examination of these factors is important as it relates to the US' needs for science and engineering professionals to promote a knowledge-driven economy, long-term productivity, and human capital, and global environmental health [74], [75].
The first load factor had six factors loaded into it, contributing to 21.590% of the variable after the rotation, with a Cronbach's alpha value equal to 0.93. This factor refers to how students perceived the laboratory experience in terms of value in enhancing their existing skills and/or technical knowledge. This factor also examines if the laboratory experience enhanced students' motivation to learn more about the laboratory topic outside of the classroom environment. In this way, the factor helps the researcher understand the tendency of the learner to allocate time towards gaining more knowledge, which is part of the I-O-E model. The factor also illustrates the connection between usefulness of the lab in preparing course work materials to motivate lifelong learning. This factor illustrates and confirms the work of Felder et al. [76] that posits that students achieve better learning outcomes when they are introduced to topics in a way that leverages their existing knowledge as it asks students about the lab's connection to previous course work. This factor also illustrates the importance of creating curriculum that facilitates students' ability to build upon and affirm their prior knowledge, which connects to the perceptions of the learning process in preparing them for their engineering careers. This factor helps the practitioner to understand how students value or rank experiences that adequately prepare them for their career and the ability to exercise softer, yet relevant skills such as writing the lab report that describes the lab process and evaluation of the results.
Similar to the first load factor, the second load factor has a high Cronbach's alpha score of 0.931, where the questions loading onto this factor contribute to 20.96% of the total variance. The second factor has seven variables loading onto it and refers to the attentiveness of the students in the lab environment, as well as the ease of use (TAM) of the in-person laboratory environment. This factor informs the instructor or instruction team/technologist, about factors that influence students' ease of observing (visually) and hearing the lab as performed by the instructor. The high value of Cronbach's alpha suggests a high reliability of this load factor to predict students' opinions regarding how easy/or not it was to engage with the in-person laboratory environment and gain insight/answers to questions from the lab instructor. The high correlation between the variables in this group reinforce our previous work [8], where qualitative responses from students indicated the importance of instructors being prepared to articulate the process of lab procedure and technical content, materials/guides being easy to interpret and follow, along with appropriate equipment and instruments that represent aspects of the discipline relevant to industry. It is expected that this instrument will provide a unique opportunity to garner students' evolving perceptions of the engineering profession and their personalized educational needs, which have been identified by the National Academy as a grand challenge in engineering [77].
The third factor has five variables loading into it. This factor contributes to 11.77% of the total variance after rotation, where the questions focus on students' expectations regarding virtual lab environments. As expected, this factor captured questions pertaining to students' expectations versus their impressions of the actual lab since the laboratory environment for this work was in-person. This factor had a high Cronbach's alpha value of 0.80. The inclusion of virtual lab expectations within the SPVEL instrument allows the instructor and engineering education practitioner to consider what aspects of the laboratory experiences may or may not be appropriate for virtual learning environments, which could allow for enhancement of educational access to those who may not be able to participate in all aspects of physical "hands-on" activities. It also sheds light on what aspects of the lab students associate with having more value when conducted in person versus others that may not.
The fourth factor has a total of three factors loading into it and is associated with 9.195% of the variable after rotation. The fourth factor examines the students' perception of the communication between the instructor and themselves, along with the general operation and flow of the laboratory. This factor has a Cronbach's alpha score of 0.76, which affirms the reliability of the questions, and gives insight into the importance of real-time feedback on the laboratory flow and process, which is related to how students perceive its technical organization and overall effectiveness. For instructors, this metric allows them to better understand how much or little students may need to interact with them during a lab period, and even perhaps the number of assistants needed to address questions during a laboratory session.
The fifth factor has three variables that pertain to engineering role identity [25], [61]. Factor five contributed 7.9% of the total variance after rotation and has a Cronbach's alpha equal to 0.65. The slightly lower Cronbach's alpha (in comparison to the other factors) is attributed to the lower connection between students' beliefs in their understanding of concepts studied in engineering and how they and others see them as engineers. This lower connection with the other variables indicates an opportunity for this instrument to garner evolving perceptions of students' affection for their chosen field, and confidence in their ties to appreciate and use skills acquired in coursework and laboratories. This disconnect in personal confidence in engineering skillset and actual performance has been noted by Villanueva and Nadelson [15]. Also, variability in student experiences, e.g., mentorship [78], parental support [79], [80], and exposure to others in engineering like themselves [81], [82], may contribute to confidence, which are elements not included in this instrument, but found to relate to engineering role identity, engineering formation, and persistence in the engineering field, which undoubtedly influence the effectiveness of educational resources and learning tools. This question may also have lower inter-relatedness since it may be interpreted differently by the students or not provide enough context for students within the same department but with different specific interests, e.g., thermal science, design, and composites. In addition, variability in confidence regarding one's abilities in a subject could be influenced by sentiments of imposter phenomenon [83], which were not explored as a part of this study instrument. Thus, it may be beneficial to allow students to include explanations regarding their choice selections when this instrument is used in future studies.
It is anticipated that the SPVEL assessment instrument can be used by researchers and instructors who facilitate and design engineering laboratories for 21st century engineering undergraduate and pre-college high science students in both remote/virtual and in-person hands on engineering laboratory settings. For example, the SPVEL instrument provides a meaningful way to assess how laboratory content relates to and affirms theoretical content taught in prior courses, which is critical to enhancing the learning outcomes of engineering undergraduate students [76]. This instrument also allows engineering education practitioners to examine the effectiveness of the communication and interaction between students and instructors, which is different from traditional assessment tools that focus on student assessment of instructor preparedness, which diminishes the importance of student accountability in the learning process. This instrument also illustrates the connection between student motivation to learn, and course relation to career goals, and incremental building of content knowledge through previous courses and STEM informal and formal exposure. The SPVEL instrument allows the instructor and researcher to examine how diverse types of laboratory environments, equipment, and tools are accepted (or not) as being useful for realistic professional skill development as interpreted by the student, along with the responsibility of the instructor to have knowledge of and ability to illustrate the relationship of lab materials to real world applications. Lastly, given the important relationship between students' association with their engineering role identity and persistence in the field, learning how laboratory environments affirm (or not) students positionality within the engineering field is vital as educators contemplate evidence-based practices for updating and modernizing laboratory equipment, protocols, and subject matter in innovative novel ways.
An exploratory factor analysis was used to validate the SPVEL instrument or use in understanding the perceptions of students engaging with in -person hands on laboratories. In this process, underlying factors within the questionnaire were identified and appropriate Cronbach alpha scores were achieved. Several questions were eliminated from the instrument due to low communality scores, i.e., lower than 0.4, or loading of two or fewer questions within one factor. Understanding how to design remote and in-person labs is a meaningful step towards developing personalized learning tools for engineering education as described by the National Academy of Engineering. Also, this work provides an initial glimpse into how students align practical demonstration and hands-on labs with skills they anticipate needing for their engineering careers.
Understanding ways of preparing 21st engineering students for the engineering profession will require critical analysis of existing norms and ways of doing, fundamental engineering theory, teaching, and mechanisms/tools for assessment as the connection between coursework and practical application of theory. As the identity and expectations of the engineering curriculum evolves, so too will the profession and research as the engineering education field become more convergent in practice.
ISSN: 2252-8822