More Like this

1. Board 418: Understanding Context: Propagation and Effectiveness of the Concept Warehouse in Mechanical Engineering at Five Diverse Institutions and Beyond – Results from Year 4

   Self, B. P. ; Koretsky, M. ; Nolen, S. B. ; Dal Bello, D. J. ; Widmann, J. M. ; Prince, M. J. ; Papadopoulos, C. ( June 2023 , 2023 ASEE Annual Conference & Exposition)

   Several consensus reports cite a critical need to dramatically increase the number and diversity of STEM graduates over the next decade. They conclude that a change to evidence-based instructional practices, such as concept-based active learning, is needed. Concept-based active learning involves the use of activity-based pedagogies whose primary objectives are to make students value deep conceptual understanding (instead of only factual knowledge) and then to facilitate their development of that understanding. Concept-based active learning has been shown to increase academic engagement and student achievement, to significantly improve student retention in academic programs, and to reduce the performance gap of underrepresented students. Fostering students' mastery of fundamental concepts is central to real world problem solving, including several elements of engineering practice. Unfortunately, simply proving that these instructional practices are more effective than traditional methods for promoting student learning, for increasing retention in academic programs, and for improving ability in professional practice is not enough to ensure widespread pedagogical change. In fact, the biggest challenge to improving STEM education is not the need to develop more effective instructional practices, but to find ways to get faculty to adopt the evidence-based pedagogies that already exist. In this project we seek to propagate themore »Concept Warehouse, a technological innovation designed to foster concept-based active learning, into Mechanical Engineering (ME) and to study student learning with this tool in five diverse institutional settings. The Concept Warehouse (CW) is a web-based instructional tool that we developed for Chemical Engineering (ChE) faculty. It houses over 3,500 ConcepTests, which are short questions that can rapidly be deployed to engage students in concept-oriented thinking and/or to assess students’ conceptual knowledge, along with more extensive concept-based active learning tools. The CW has grown rapidly during this project and now has over 1,600 faculty accounts and over 37,000 student users. New ConcepTests were created during the current reporting period; the current numbers of questions for Statics, Dynamics, and Mechanics of Materials are 342, 410, and 41, respectively. A detailed review process is in progress, and will continue through the no-cost extension year, to refine question clarity and to identify types of new questions to fill gaps in content coverage. There have been 497 new faculty accounts created after June 30, 2018, and 3,035 unique students have answered these mechanics questions in the CW. We continue to analyze instructor interviews, focusing on 11 cases, all of whom participated in the CW Community of Practice (CoP). For six participants, we were able to compare use of the CW both before and after participating in professional development activities (workshops and/or a community or practice). Interview results have been coded and are currently being analyzed. To examine student learning, we recruited faculty to participate in deploying four common questions in both statics and dynamics. In statics, each instructor agreed to deploy the same four questions (one each for Rigid Body Equilibrium, Trusses, Frames, and Friction) among their overall deployments of the CW. In addition to answering the question, students were also asked to provide a written explanation to explain their reasoning, to rate the confidence of their answers, and to rate the degree to which the questions were clear and promoted deep thinking. The analysis to date has resulted in a Work-In-Progress paper presented at ASEE 2022, reporting a cross-case comparison of two instructors and a Work-In-Progress paper to be presented at ASEE 2023 analyzing students’ metacognitive reflections of concept questions.« less
2. Motivating youth to learn STEM through a gender inclusive digital forensic science program

   https://doi.org/10.1186/s40561-022-00213-x  

   Casey, Eoghan ; Jocz, Jennifer ; Peterson, Karen A. ; Pfeif, Daryl ; Soden, Cassy ( January 2023 , Smart Learning Environments)

   This paper describes the design, implementation and research of the Cyber Sleuth Science Lab (CSSL), an innovative educational program and supporting virtual learning environment, that combines pedagogical theory, gender inclusive instruction strategies, scientific principles/practices, gamification methods, computational thinking, and real-world problem solving. This program provides underrepresented youth, especially girls, with digital forensic knowledge, skills and career pathways, challenging them to explore complex social issues related to technology and to become cyber sleuths using real-world digital forensic methods and tools to solve investigative scenarios. Students also learn about related careers while improving their cyber street smarts. The CSSL incorporates additional “outside of the computer” activities to strengthen students’ engagement such as structured in-classroom discussions, mock trials, and in-person interactions with practitioner role models. The CSSL was piloted in various forms to assess the suitability for in-school and out-of-school settings, and the students predominantly represented racial minorities. Research in this project relied on a mixed methods approach for data collection and analysis, including qualitative and quantitative methods, reinforced using learning analytics generated from the students clicking through the interface and interacting with the system. Analysis of gathered data indicate that the virtual learning environment developed in this project is highly effectivemore »for teaching digital forensic knowledge, skills, and abilities that are directly applicable in the workplace. Furthermore, the strategies for gender inclusive STEM instruction implemented in CSSL are effective for engaging girls without being harmful to boys’ engagement. Learning STEM through digital forensic science taps into girls’ motivations to address real-world problems that have direct relevance to their lives, and to protect and serve their community. After participating in the educational program, girls expressed a significantly greater increase in interest, relative to boys, in learning more about careers related to digital forensics and cybersecurity.

   « less
3. Experiences during the implementation of two different project-based learning assignments in a fluid mechanics course.

   Ayala, O. ( August 2022 , Zone 1 Conference of the American Society for Engineering Education)

   There is growing evidence of the effectiveness of project-based learning (PBL) in preparing students to solve complex problems. In PBL implementations in engineering, students are treated as professional engineers facing projects centered around real-world problems, including the complexity and uncertainty that influence such problems. Not only does this help students to analyze and solve an authentic real-world task, promoting critical thinking, but also students learn from each other, learning valuable communication and teamwork skills. Faculty play an important part by assuming non-conventional roles (e.g., client, senior professional engineer, consultant) to help students throughout this instructional and learning approach. Typically in PBLs, students work on projects over extended periods of time that culminate in realistic products or presentations. In order to be successful, students need to learn how to frame a problem, identify stakeholders and their requirements, design and select concepts, test them, and so on. Two different implementations of PBL projects in a fluid mechanics course are presented in this paper. This required, junior-level course has been taught since 2014 by the same instructor. The first PBL project presented is a complete design of pumped pipeline systems for a hypothetical plant. In the second project, engineering students partnered with pre-servicemore »teachers to design and teach an elementary school lesson on fluid mechanics concepts. With the PBL implementations, it is expected that students: 1) engage in a deeper learning process where concepts can be reemphasized, and students can realize applicability; 2) develop and practice teamwork skills; 3) learn and practice how to communicate effectively to peers and to those from other fields; and 4) increase their confidence working on open-ended situations and problems. The goal of this paper is to present the experiences of the authors with both PBL implementations. It explains how the projects were scaffolded through the entire semester, including how the sequence of course content was modified, how team dynamics were monitored, the faculty roles, and the end products and presentations. Students' experiences are also presented. To evaluate and compare students’ learning and satisfaction with the team experience between the two PBL implementations, a shortened version of the NCEES FE exam and the Comprehensive Assessment of Team Member Effectiveness (CATME) survey were utilized. Students completed the FE exam during the first week and then again during the last week of the semester in order to assess students’ growth in fluid mechanics knowledge. The CATME survey was completed mid-semester to help faculty identify and address problems within team dynamics, and at the end of the semester to evaluate individual students’ teamwork performance. The results showed that no major differences were observed in terms of the learned fluid mechanics content, however, the data showed interesting preliminary observations regarding teamwork satisfaction. Through reflective assignments (e.g., short answer reflections, focus groups), student perceptions of the PBL implementations are discussed in the paper. Finally, some of the challenges and lessons learned from implementing both projects multiple times, as well as access to some of the PBL course materials and assignments will be provided.« less
4. Machine learning for the educational sciences

   https://doi.org/10.1002/rev3.3310  

   Hilbert, Sven ; Coors, Stefan ; Kraus, Elisabeth ; Bischl, Bernd ; Lindl, Alfred ; Frei, Mario ; Wild, Johannes ; Krauss, Stefan ; Goretzko, David ; Stachl, Clemens ( November 2021 , Review of Education)

   Machine learning (ML) provides a powerful framework for the analysis of high‐dimensional datasets by modelling complex relationships, often encountered in modern data with many variables, cases and potentially non‐linear effects. The impact of ML methods on research and practical applications in the educational sciences is still limited, but continuously grows, as larger and more complex datasets become available through massive open online courses (MOOCs) and large‐scale investigations. The educational sciences are at a crucial pivot point, because of the anticipated impact ML methods hold for the field. To provide educational researchers with an elaborate introduction to the topic, we provide an instructional summary of the opportunities and challenges of ML for the educational sciences, show how a look at related disciplines can help learning from their experiences, and argue for a philosophical shift in model evaluation. We demonstrate how the overall quality of data analysis in educational research can benefit from these methods and show how ML can play a decisive role in the validation of empirical models. Specifically, we (1) provide an overview of the types of data suitable for ML and (2) give practical advice for the application of ML methods. In each section, we provide analyticalmore »examples and reproducible R code. Also, we provide an extensive Appendix on ML‐based applications for education. This instructional summary will help educational scientists and practitioners to prepare for the promises and threats that come with the shift towards digitisation and large‐scale assessment in education.

   In 2020, the worldwide SARS‐COV‐2 pandemic forced the educational sciences to perform a rapid paradigm shift with classrooms going online around the world—a hardly novel but now strongly catalysed development. In the context of data‐driven education, this paper demonstrates that the widespread adoption of machine learning techniques is central for the educational sciences and shows how these methods will become crucial tools in the collection and analysis of data and in concrete educational applications. Helping to leverage the opportunities and to avoid the common pitfalls of machine learning, this paper provides educators with the theoretical, conceptual and practical essentials.

   The process of teaching and learning is complex, multifaceted and dynamic. This paper contributes a seminal resource to highlight the digitisation of the educational sciences by demonstrating how new machine learning methods can be effectively and reliably used in research, education and practical application.

   The progressing digitisation of societies around the globe and the impact of the SARS‐COV‐2 pandemic have highlighted the vulnerabilities and shortcomings of educational systems. These developments have shown the necessity to provide effective educational processes that can support sometimes overwhelmed teachers to digitally impart knowledge on the plan of many governments and policy makers. Educational scientists, corporate partners and stakeholders can make use of machine learning techniques to develop advanced, scalable educational processes that account for individual needs of learners and that can complement and support existing learning infrastructure. The proper use of machine learning methods can contribute essential applications to the educational sciences, such as (semi‐)automated assessments, algorithmic‐grading, personalised feedback and adaptive learning approaches. However, these promises are strongly tied to an at least basic understanding of the concepts of machine learning and a degree of data literacy, which has to become the standard in education and the educational sciences.

   Demonstrating both the promises and the challenges that are inherent to the collection and the analysis of large educational data with machine learning, this paper covers the essential topics that their application requires and provides easy‐to‐follow resources and code to facilitate the process of adoption.

   « less
5. Engaging preschoolers in data collection and analysis with manipulatives, body movement, and digital technology

   Lewis Presser, A. E. ( February 2021 , Early Childhood STEM Conference)

   This workshop will focus on how to teach data collection and analysis to preschoolers. Our project aims to promote preschoolers’ engagement with, and learning of, mathematics and computational thinking (CT) with a set of classroom activities that engage preschoolers in a data collection and analysis (DCA) process. To do this, the project team is engaging in an iterative cycle of development and testing of hands-on, play-based, curricular investigations with feedback from teachers. A key component of the intervention is a teacher-facing digital app (for teachers to use with students on touch-screen tablets) to support the collaboration of preschool teachers and children in collecting data, creating simple graphs, and using the graphs to answer real-world questions. The curricular investigations offer an applied context for using mathematical knowledge (i.e., counting, sorting, classifying, comparing, contrasting) to engage with real-world investigations and lay the foundation for developing flexible problem-solving skills. Each investigation follows a series of instructional tasks that scaffold the problem-solving process and includes (a) nine hands-on and play-based problem-solving investigations where children answer real-world questions by collecting data, creating simple graphs, and interpreting the graphs and. (b) a teacher- facing digital app to support specific data collection and organization steps (i.e., collecting,more »recording, visualizing). This workshop will describe: (1) the rationale and prior research conducted in this domain, (2) describe an intervention in development focused on data collection and analysis content for preschoolers that develop mathematical (common core standards) and computational thinking skills (K-12 Computational Thinking Framework Standards), (3) demonstrates an app in development that guides teacher and preschoolers through the investigation process and generates graphs to answer questions (NGSS practice standards), (4) report on feedback from a pilot study conducted virtually in preschool classrooms; and (5) describe developmentally appropriate practices for engaging young children in investigations, data collection, and data analysis.« less