More Like this

1. Understanding Context: Propagation and Effectiveness of the Concept Warehouse in Mechanical Engineering at Five Diverse Institutions and Beyond – Results from Year 2

   https://doi.org/10.18260/1-2--35419  

   Self, B. P. ; Koretsky, M. D. ; Papadopoulos, C. ; Widmann, J. M. ( July 2021 , ASEE annual conference exposition)

   It has been well-established that concept-based active learning strategies increase student retention, improve engagement and student achievement, and reduce the performance gap of underrepresented students. Despite the evidence supporting concept-based instruction, many faculty continue to stress algorithmic problem solving. In fact, the biggest challenge to improving STEM education is not the need to develop more effective instructional practices, but to find ways to get faculty to adopt the evidence-based pedagogies that already exist. Our project aims to propagate the Concept Warehouse (CW), an online innovation tool that was developed in the Chemical Engineering community, into Mechanical Engineering (ME). A portion of our work focuses on content development in mechanics, and includes statics, dynamics, and to a lesser extent strength of materials. Our content development teams had created 170 statics and 253 dynamics questions. Additionally, we have developed four different simulations to be embedded in online Instructional Tools – these are interactive modules that provided different physical scenarios to help students understand important concepts in mechanics. During initial interviews, we found that potential adopters needed coaching on the benefits of concept-based instruction, training on how to use the CW, and support on how to best implement the different affordances offered by the CW. This caused a slight shift in our initial research plans, and much of our recent work has concentrated on using faculty development activities to help us advertise the CW and encourage evidence-based practices. From these activities, we are recruiting participants for surveys and interviews to help us investigate how different contexts affect the adoption of educational innovations. A set of two summer workshops attracted over 270 applicants, and over 60 participants attended each synchronous offering. Other applicants were provided links to recordings of the workshop. From these participants, we recruited 20 participants to join our Community of Practice (CoP). These members are sharing how they use the CW in their classes, especially in the virtual environment. Community members discuss using evidence-based practices, different things that the CW can do, and suggest potential improvements to the tool. They will also be interviewed to help us determine barriers to adoption, how their institutional contexts and individual epistemologies affect adoption, and how they have used the CW in their classes. Our research will help us formulate strategies that others can use when attempting to propagate pedagogical innovations. 

   more » « less
2. WIP: Instances of Dynamic Pedagogical Decision Making in the Uptake of a Technology Tool

   Harpreet Auby ; John Galisky ; Susan Nolen ; Milo Koretsky ( June 2022 , ASEE Annual Conference proceedings)

   In this work-in-progress paper, we continue investigation into the propagation of the Concept Warehouse within mechanical engineering (Friedrichsen et al., 2017; Koretsky et al., 2019a). Even before the pandemic forced most instruction online, educational technology was a growing element in classroom culture (Koretsky & Magana, 2019b). However, adoption of technology tools for widespread use is often conceived from a turn-key lens, with professional development focused on procedural competencies and fidelity of implementation as the goal (Mills & Ragan, 2000; O’Donnell, 2008). Educators are given the tool with initial operating instructions, then left on their own to implement it in particular instructional contexts. There is little emphasis on the inevitable instructional decisions around incorporating the tool (Hodge, 2019) or on sustainable incorporation of technologies into existing instructional practice (Forkosh-Baruch et al., 2021). We consider the take-up of a technology tool as an emergent, rather than a prescribed process (Henderson et al., 2011). In this WIP paper, we examine how two instructors who we call Al and Joe reason through their adoption of a technology tool, focusing on interactions among instructors, tool, and students within and across contexts. The Concept Warehouse (CW) is a widely-available, web-based, open educational technology tool used to facilitate concept-based active learning in different contexts (Friedrichsen et al., 2017; Koretsky et al., 2014). Development of the CW is ongoing and collaboration-driven, where user-instructors from different institutions and disciplines can develop conceptual questions (called ConcepTests) and other learning and assessment tools that can be shared with other users. Currently there are around 3,500 ConcepTests, 1,500 faculty users, and 36,000 student users. About 700 ConcepTests have been developed for mechanics (statics and dynamics). The tool’s spectrum of affordances allows different entry points for instructor engagement, but also allows their use to grow and change as they become familiar with the tool and take up ideas from the contexts around them. Part of a larger study of propagation and use across five diverse institutions (Nolen & Koretsky, 2020), instructors were introduced to the tool, offered an introductory workshop and opportunity to participate in a community of practice (CoP), then interviewed early and later in their adoption. For this paper, we explore a bounded case study of the two instructors, Al and Joe, who took up the CW to teach Introductory Statics. Al and Joe were experienced instructors, committed to active learning, who presented examples from their ongoing adaptation of the tool for discussion in the community of practice. However, their decisions about how to integrate the tool fundamentally differed, including the aspects of the tool they took up and the ways they made sense of their use. In analyzing these two cases, we begin to uncover how these instructors navigated the dynamic nature of pedagogical decision making in and across contexts. 

   more » « less
3. WIP: Using Machine Learning to Automate Coding of Student Explanations to Challenging Mechanics Concept Questions

   Harpreet Auby ; Namrata Shivagunde ; Anna Rumshisky ; Milo Koretsky ( June 2022 , ASEE Annual Conference proceedings)

   This work-in-progress paper describes a collaborative effort between engineering education and machine learning researchers to automate analysis of written responses to conceptually challenging questions in mechanics. These qualitative questions are often used in large STEM classes to support active learning pedagogies; they require minimum calculations and focus on the application of underlying physical phenomena to various situations. Active learning pedagogies using this type of questions has been demonstrated to increase student achievement (Freeman et al., 2014; Hake, 1998) and engagement (Deslauriers, et al., 2011) of all students (Haak et al., 2011). To emphasize reasoning and sense-making, we use the Concept Warehouse (Koretsky et al., 2014), an audience response system where students provide written justifications to concept questions. Written justifications better prepare students for discussions with peers and in the whole class and can also improve students’ answer choices (Koretsky et al., 2016a, 2016b). In addition to their use as a tool to foster learning, written explanations can also provide valuable information to concurrently assess that learning (Koretsky and Magana, 2019). However, in practice, there has been limited deployment of written justifications with concept questions, in part, because they provide a daunting amount of information for instructors to process and for researchers to analyze. In this study, we describe the initial evaluation of large pre-trained generative sequence-to-sequence language models (Raffel et al., 2019; Brown et al., 2020) to automate the laborious coding process of student written responses. Adaptation of machine learning algorithms in this context is challenging since each question targets specific concepts which elicit their own unique reasoning processes. This exploratory project seeks to utilize responses collected through the Concept Warehouse to identify viable strategies for adapting machine learning to support instructors and researchers in identifying salient aspects of student thinking and understanding with these conceptually challenging questions. 

   more » « less
4. Progress in the Nationwide Dissemination and Assessment of Low-Cost Desktop Learning Modules and Adaptation of Pedagogy to a Virtual Era

   Van Wie, BJ ( July 2021 , American Society for Engineering Education)

   null (Ed.)

   The development of tools that promote active learning in engineering disciplines is critical. It is widely understood that students engaged in active learning environments outperform those taught using passive methods. Previously, we reported on the development and implementation of hands-on Low-Cost Desktop Learning Modules (LCDLMs) that replicate real-world industrial equipment which serves to create active learning environments. Thus far, miniaturized venturi meter, hydraulic loss, and double-pipe and shell & tube heat exchanger DLMs have been utilized by hundreds of students across the country. It was demonstrated that the use of DLMs in face-to-face classrooms results in statistically significant improvements in student performance as well as increases in student motivation compared to students taught in a traditional lecture-only style classroom. Last year, participants in the project conducted 45 implementations including over 600 DLMs at 24 universities across the country reaching more than 1,000 students. In this project, we report on the significant progress made in broad dissemination of DLMs and accompanying pedagogy. We demonstrate that DLMs serve to increase student learning gains not only in face-to-face environments but also in virtual learning environments. Instructional videos were developed to aid in DLM-based learning during the COVID-19 pandemic when instructors were limited to virtual instruction. Preliminary results from this work show that students working with DLMs even in a virtual setting significantly outperform those taught without DLM-associated materials. Significant progress has also been made on the development of a new DLM cartridge: a see-through 3D-printed miniature fluidized bed. The new 3D printing methodology will allow for rapid prototyping and streamlined development of DLMs. A 3D-printed evaporative cooling tower DLM will also be developed in the coming year. In October 2020, the team held a virtual implementers workshop to train new participating faculty in DLM use and implementation. In total, 13 new faculty participants from 10 universities attended the 6-hour, 2-day workshop and plan to implement DLMs in their classrooms during this academic year. In the last year, this project was disseminated in 8 presentations at the American Society for Engineering Education (ASEE) Virtual Conference (June 2020) and American Institute of Chemical Engineers Annual Conference (November 2019) as well as the AIChE virtual Community of Practice Labs Group and a seminar at a major university, ultimately disseminating DLM pedagogy to approximately 200 individuals including approximately 120 university faculty. Further, the former group postdoc has accepted an instructor faculty position at University of Wisconsin Madison where she will teach unit operations among other subjects; she and the remainder of the team believe the LCDLM project has prepared her well for that position. In the remaining 2.5 years of the project, we will continue to evaluate the effectiveness of DLMs in teaching key heat transfer and fluid dynamics concepts thru implementations in the rapidly expanding pool of participating universities. Further, we continue our ongoing efforts in creating the robust support structure necessary for large-scale adoption of hands-on educational tools for promotion of hands-on interactive student learning. 

   more » « less
5. Progress in the Nationwide Dissemination and Assessment of Low-Cost Desktop Learning Modules and Adaptation of Pedagogy to a Virtual Era

   Van Wie, Bernard ; Bryant, Kristin ; Kaiphanliam, Kitana ; Khan, Aminul ; Olufunso, Oje ; Dutta, Prashanta ; Gartner, Jacqueline ; Thiessen, David ( July 2021 , ASEE Annual Conference proceedings)

   null (Ed.)

   The development of tools that promote active learning in engineering disciplines is critical. It is widely understood that students engaged in active learning environments outperform those taught using passive methods. Previously, we reported on the development and implementation of hands-on Low-Cost Desktop Learning Modules (LCDLMs) that replicate real-world industrial equipment which serves to create active learning environments. Thus far, miniaturized venturi meter, hydraulic loss, and double-pipe and shell & tube heat exchanger DLMs have been utilized by hundreds of students across the country. It was demonstrated that the use of DLMs in face-to-face classrooms results in statistically significant improvements in student performance as well as increases in student motivation compared to students taught in a traditional lecture-only style classroom. Last year, participants in the project conducted 45 implementations including over 600 DLMs at 24 universities across the country reaching more than 1,000 students. In this project, we report on the significant progress made in broad dissemination of DLMs and accompanying pedagogy. We demonstrate that DLMs serve to increase student learning gains not only in face-toface environments but also in virtual learning environments. Instructional videos were developed to aid in DLM-based learning during the COVID-19 pandemic when instructors were limited to virtual instruction. Preliminary results from this work show that students working with DLMs even in a virtual setting significantly outperform those taught without DLM-associated materials. Significant progress has also been made on the development of a new DLM cartridge: a see-through 3Dprinted miniature fluidized bed. The new 3D printing methodology will allow for rapid prototyping and streamlined development of DLMs. A 3D-printed evaporative cooling tower DLM will also be developed in the coming year. In October 2020, the team held a virtual implementers workshop to train new participating faculty in DLM use and implementation. In total, 13 new faculty participants from 10 universities attended the 6-hour, 2- day workshop and plan to implement DLMs in their classrooms during this academic year. In the last year, this project was disseminated in 8 presentations at the ASEE Virtual Conference (June 2020) and American Institute of Chemical Engineers Annual Conference (November 2019) as well as the AIChE virtual Community of Practice Labs Group and a seminar at a major university, ultimately disseminating DLM pedagogy to approximately 200 individuals including approximately 120 university faculty. Further, the former group postdoc has accepted an instructor faculty position at University of Wisconsin Madison where she will teach unit operations among other subjects; she and the remainder of the team believe the LCDLM project has prepared her well for that position. In the remaining 2.5 years of the project, we will continue to evaluate the effectiveness of DLMs in teaching key heat transfer and fluid dynamics concepts thru implementations in the rapidly expanding pool of participating universities. Further, we continue our ongoing efforts in creating the robust support structure necessary for large-scale adoption of hands-on educational tools for promotion of hands-on interactive student learning. 

   more » « less