This project was designed to address three major challenges faced by undergraduate engineering students (UES) and pre-service teachers (PSTs): 1) retention for UESs after the first year, and continued engagement when they reach more difficult concepts, 2) to prepare PSTs to teach engineering, which is a requirement in the Next Generation Science Standards as well as many state level standards of learning, and 3) to prepare both groups of students to communicate and collaborate in a multi-disciplinary context, which is a necessary skill in their future places of work. This project was implemented in three pairs of classes: 1) an introductory mechanical engineering class, fulfilling a general education requirement for information literacy and a foundations class in education, 2) fluid mechanics in mechanical engineering technology and a science methods class in education, and 3) mechanical engineering courses requiring programming (e.g., computational methods and robotics) with an educational technology class. All collaborations taught elementary level students (4th or 5th grade). For collaborations 1 and 2, the elementary students came to campus for a field trip where they toured engineering labs and participated in a one hour lesson taught by both the UESs and PSTs. In collaboration 3, the UESs and PSTs worked with the upper-elementary students in their school during an after school club. In collaborations 1 and 2, students were assigned to teams and worked remotely on some parts of the project. A collaboration tool, built in Google Sites and Google Drive, was used to facilitate the project completion. The collaboration tool includes a team repository for all the project documents and templates. Students in collaboration 3 worked together directly during class time on smaller assignments. In all three collaborations lesson plans were implemented using the BSCS 5E instructional model, which was aligned to the engineering design process. Instruments were developed to assess knowledge in collaborations 1 (engineering design process) and 3 (computational thinking), while in collaboration 2, knowledge was assessed with questions from the fundamentals of engineering exam and a science content assessment. Comprehensive Assessment of Team Member Effectiveness (CATME) was also used in all 3 collaborations to assess teamwork across the collaborations. Finally, each student wrote a reflection on their experiences, which was used to qualitatively assess the project impact. The results from the first full semester of implementation have led us to improvements in the implementation and instrument refinement for year 2.
more »
« less
Content Agnostic Game Engineering: Impact of Stealth Assessment and Content Order on Player Engagement
Content Agnostic Game Engineering (CAGE) architecture utilizes content agnostic mechanics to create educational games that can teach multiple contents. However, the player engagement goes down when second content is played using the same game mechanics. A content agnostic stealth assessment can aid a CAGE game in sustaining the engagement level of its players. A potentially generalizable method for this was tested using Chem-o-crypt, a CAGE game that can teach chemistry and cryptography contents. The game automatically detects frustration, flow, and boredom using the Affdex SDK from Affectiva. A randomized controlled experiment incorporating real-time game adaptation revealed that using stealth assessment can help sustain engagement in a CAGE game when playing multiple contents.
more »
« less
- Award ID(s):
- 1828010
- NSF-PAR ID:
- 10346355
- Date Published:
- Journal Name:
- Proceedings of the Future Technologies Conference
- Volume:
- 360
- Page Range / eLocation ID:
- 455–470
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In an attempt to predict the learning of a player during a content agnostic educational video game session, this study used a dynamic bayesian network in which participants’ game play interactions were continuously recorded. Their actions were captured and used to make real-time inferences of the learning performance using a dynamic bayesian network. The predicted learning was then correlated with the post-test scores to establish the validity of assessment. The assessment was moderately positively correlated with the post-test scores demonstrating support for its validity.more » « less
-
Major challenges in engineering education include retention of undergraduate engineering students (UESs) and continued engagement after the first year when concepts increase in difficulty. Additionally, employers, as well as ABET, look for students to demonstrate non-technical skills, including the ability to work successfully in groups, the ability to communicate both within and outside their discipline, and the ability to find information that will help them solve problems and contribute to lifelong learning. Teacher education is also facing challenges given the recent incorporation of engineering practices and core ideas into the Next Generation Science Standards (NGSS) and state level standards of learning. To help teachers meet these standards in their classrooms, education courses for preservice teachers (PSTs) must provide resources and opportunities to increase science and engineering knowledge, and the associated pedagogies. To address these challenges, Ed+gineering, an NSF-funded multidisciplinary collaborative service learning project, was implemented into two sets of paired-classes in engineering and education: a 100 level mechanical engineering class (n = 42) and a foundations class in education (n = 17), and a fluid mechanics class in mechanical engineering technology (n = 23) and a science methods class (n = 15). The paired classes collaborated in multidisciplinary teams of 5-8 undergraduate students to plan and teach engineering lessons to local elementary school students. Teams completed a series of previously tested, scaffolded activities to guide their collaboration. Designing and delivering lessons engaged university students in collaborative processes that promoted social learning, including researching and planning, peer mentoring, teaching and receiving feedback, and reflecting and revising their engineering lesson. The research questions examined in this pilot, mixed-methods research study include: (1) How did PSTs’ Ed+gineering experiences influence their engineering and science knowledge?; (2) How did PSTs’ and UESs’ Ed+gineering experiences influence their pedagogical understanding?; and (3) What were PSTs’ and UESs’ overall perceptions of their Ed+gineering experiences? Both quantitative (e.g., Engineering Design Process assessment, Science Content Knowledge assessment) and qualitative (student reflections) data were used to assess knowledge gains and project perceptions following the semester-long intervention. Findings suggest that the PSTs were more aware and comfortable with the engineering field following lesson development and delivery, and often better able to explain particular science/engineering concepts. Both PSTs and UESs, but especially the latter, came to realize the importance of planning and preparing lessons to be taught to an audience. UESs reported greater appreciation for the work of educators. PSTs and UESs expressed how they learned to work in groups with multidisciplinary members—this is a valuable lesson for their respective professional careers. Yearly, the Ed+gineering research team will also request and review student retention reports in their respective programs to assess project impact.more » « less
-
Foundational engineering courses are critical to student success in engineering programs. The conceptually challenging content of these courses establishes the requisite knowledge for future classes. Thus, it is no surprise that such courses can serve as barriers or gatekeepers to successful student progress through the undergraduate curriculum. Although the difficulty of the courses may be necessary, often other features of the course delivery such as large class environments or a few very high-stakes assessments can further exacerbate these challenges. And especially problematic, past studies have shown that grade penalties associated with these courses and environments may disproportionately impact women. On the faculty side, institutions often turn to non-tenure track instructional faculty to teach multiple sections of foundational courses each semester. Although having faculty whose sole role is dedicated to quality teaching is an asset, benefits would likely be maximized when such faculty have clear metrics for paths to promotion, some autonomy and ownership regarding the curriculum, and overall job satisfaction. However, literature suggests that faculty, like students, note ill effects from large classes, such as challenges connecting and building rapport with students and having time to offer individualized feedback to students. Our NSF IUSE project focuses on instructors of large foundational engineering students with the belief that by better understanding the educational environment from their perspective we can improve the quality of the teaching and learning environment for all engineering students. Our project regularly convenes faculty teaching an array of core courses (e.g,. Mathematics, Chemistry, Mechanics, Physics) and uses insights from these meetings and individual interviews to identify possible leverage points where our project or the institution more broadly might affect change. Parallel to this effort, we have been working with data stewards on campus to gain access to institutional data (e.g., student course and grade histories, student evaluations of faculty teaching) to link and provide aggregate deidentified results to faculty to feed more information in to their decision-making. We are demonstrating that regular engagement between faculty and institutional leaders around analyzed and curated data is essential to continuous and systematic improvement. Efforts to date have included building an institutional data explorer dashboard (e.g., influences of pre-requisite courses on future courses) and drafting reports to be sent to department heads and associate deans which gather priorities identified in the first year of our research. For example, participating instructors identified that clarity of promotion paths across non-tenure track teaching faculty from different departments varied greatly, and the institution as a whole could benefit from clarified university-wide guidance. While some findings may be institution-specific (NSF IUSE Institutional Transformation track), as a large public research institution, peer-institutions with high engineering enrollments often face similar challenges and so findings from our change efforts potentially have broad applicability.more » « less
-
A key untapped feature of game-based learning environments is their capacity to generate a rich stream of fine-grained learning interaction data. The learning behaviors captured in these data provide a wealth of information on student learning, which stealth assessment can utilize to unobtrusively draw inferences about student knowledge to provide tailored problem-solving support. In this paper, we present a long short-term memory network (LSTM)-based stealth assessment framework that takes as input an observed sequence of raw game-based learning environment interaction data along with external pre-learning measures to infer students’ post-competencies. The framework is evaluated using data collected from 191 middle school students interacting with a game-based learning environment for middle grade computational thinking. Results indicate that LSTM-based stealth assessors induced from student game-based learning interaction data outperform comparable models that required labor-intensive hand-engineering of input features. The findings suggest that the LSTM-based approach holds significant promise for evidence modeling in stealth assessment.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1007/978-3-030-89912-7_35
