More Like this

1. Bridging the Gap Between Academia and Industry in Approaches for Solving Ill-Structured Problems: Problem Formulation and Protocol Development

   Akinci-Ceylan, Secil ; Cetin, Kristen Sara ; Fleming, Renee ; Ahn, Benjamin ; Surovek, Andrea ; Cetin, Bora ; Taylor, Paige ( January 2018 , ASEE Annual Conference proceedings)

   One of the main skills of engineers is to be able to solve problems. It is generally recognized that real-world engineering problems are inherently ill-structured in that they are complex, defined by non-engineering constraints, are missing information, and contain conflicting information. Therefore, it is very important to prepare future engineering students to be able to anticipate the occurrence of such problems, and to be prepared to solve them. However, most courses are taught by academic professors and lecturers whose focus is on didactic teaching of fundamental principles and code-based design approaches leading to predetermined “right” answers. Most classroom-taught methods tomore »solve well-structured problems and the methods needed to solve ill-structured problems are strikingly different. The focus of our current effort is to compare and contrast the problem solving approaches employed by students, academics and practicing professionals in an attempt to determine if students are developing the necessary skills to tackle ill-structured problems. To accomplish this, an ill-structured problem is developed, which will later be used to determine, based on analysis of oral and written responses of participants in semi-structured interviews, attributes of the gap between student, faculty, and professional approaches to ill-structured problem solving. Based on the results of this analysis, we will identify what pedagogical approaches may limit and help students’ abilities to develop fully-formed solutions to ill-structured problems. This project is currently ongoing. This work-in-progress paper will present the study and proposed methods. Based on feedback obtained at the conference from the broader research community, the studies will be refined. The current phase includes three parts, (1) problem formulation; (2) protocol development; and (3) pilot study. For (1), two different ill-structured problems were developed in the Civil Engineering domain. The problem difficulty assessment method was used to determine the appropriateness of each problem developed for this study. For (2), a protocol was developed in which participants will be asked to first solve a simple problem to become familiar with the interview format, then are given 30 minutes to solve the provided ill-structured problem, following a semi-structured interview format. Participants will be encouraged to speak out loud and also write down what they are thinking and their thought processes throughout the interview period. Both (1) and (2) will next be used for (3) the pilot study. The pilot study will include interviewing three students, three faculty members and three professional engineers. Each participant will complete both problems following the same protocol developed. Post-interview discussion will be held with the pilot study participants individually to inquire if there were any portions of the tasks that are unclearly worded or could be improved to clarify what was being asked. Based on these results the final problem will be chosen and refined.« less
2. Changing Homework Achievement with Mechanix Pedagogy: Increasing the Efficacy of a Measurement Tool for Construction Majors,” American Society for Engineering Education Annual Conference, Minneapolis, Minnesota.

   Talley, K. ( January 2022 , American Society for Engineering Education Annual Conference)

   In online or large in-person course sections, instructors often adopt an online homework tool to alleviate the burden of grading. While these systems can quickly tell students whether they got a problem correct for a multiple-choice or numeric answer, they are unable to provide feedback on students’ free body diagrams. As the process of sketching a free body diagram correctly is a foundational skill to solving engineering problems, the loss of feedback to the students in this area is a detriment to students. To address the need for rapid feedback on students’ free body diagram sketching, the research team developedmore »an online, sketch-recognition system called Mechanix. This system allows students to sketch free body diagrams, including for trusses, and receive instant feedback on their sketches. The sketching feedback is ungraded. After the students have a correct sketch, they are then able to enter in the numeric answers for the problem and submit those for a grade. Thereby, the platform offers the grading convenience of other online homework systems but also helps the students develop their free body diagram sketching skills. To assess the efficacy of this experimental system, standard concept inventories were administered pre- and post-semester for both experimental and control groups. The unfamiliarity or difficulty of some advanced problems in the Statics Concept Inventory, however, appeared to discourage students, and many would stop putting in any effort after a few problems that were especially challenging to solve. This effect was especially pronounced with the Construction majors versus the Mechanical Engineering majors in the test group. To address this tendency and therefore collect more complete pre- and post-semester concept inventory data, the research group worked on reordering the Statics Concept Inventory questions from more familiar to more challenging, based upon the past performance of the initial students taking the survey. This paper describes the process and results of the effort to reorder this instrument in order to increase Construction student participation and, therefore, the researchers’ ability to measure the impact of the Mechanix system.« less
3. Changing Homework Achievement with Mechanix Pedagogy: Increasing the Efficacy of a Measurement Tool for Construction Majors

   Talley, Kimberly G. ; Hurt, Josh ; Linsey, Julie S. ( June 2022 , ASEE Annual Conference proceedings)

   In online or large in-person course sections, instructors often adopt an online homework tool to alleviate the burden of grading. While these systems can quickly tell students whether they got a problem correct for a multiple-choice or numeric answer, they are unable to provide feedback on students’ free body diagrams. As the process of sketching a free body diagram correctly is a foundational skill to solving engineering problems, the loss of feedback to the students in this area is a detriment to students. To address the need for rapid feedback on students’ free body diagram sketching, the research team developedmore »an online, sketch-recognition system called Mechanix. This system allows students to sketch free body diagrams, including for trusses, and receive instant feedback on their sketches. The sketching feedback is ungraded. After the students have a correct sketch, they are then able to enter in the numeric answers for the problem and submit those for a grade. Thereby, the platform offers the grading convenience of other online homework systems but also helps the students develop their free body diagram sketching skills. To assess the efficacy of this experimental system, standard concept inventories were administered pre- and post-semester for both experimental and control groups. The unfamiliarity or difficulty of some advanced problems in the Statics Concept Inventory, however, appeared to discourage students, and many would stop putting in any effort after a few problems that were especially challenging to solve. This effect was especially pronounced with the Construction majors versus the Mechanical Engineering majors in the test group. To address this tendency and therefore collect more complete pre- and post-semester concept inventory data, the research group worked on reordering the Statics Concept Inventory questions from more familiar to more challenging, based upon the past performance of the initial students taking the survey. This paper describes the process and results of the effort to reorder this instrument in order to increase Construction student participation and, therefore, the researchers’ ability to measure the impact of the Mechanix system.« less
4. Partnering Undergraduate Engineering Students with Preservice Teachers to Design and Teach an Elementary Engineering Lesson Through Ed+gineering

   Gutierrez, K. ; Ringleb, S. ; Kidd, J. ; Ayala, O. ; Pazos, P. ; Kaipa, K. ( June 2020 , 2020 ASEE Virtual Annual Conference Content Access, Virtual Online)

   Major challenges in engineering education include retention of undergraduate engineering students (UESs) and continued engagement after the first year when concepts increase in difficulty. Additionally, employers, as well as ABET, look for students to demonstrate non-technical skills, including the ability to work successfully in groups, the ability to communicate both within and outside their discipline, and the ability to find information that will help them solve problems and contribute to lifelong learning. Teacher education is also facing challenges given the recent incorporation of engineering practices and core ideas into the Next Generation Science Standards (NGSS) and state level standards ofmore »learning. To help teachers meet these standards in their classrooms, education courses for preservice teachers (PSTs) must provide resources and opportunities to increase science and engineering knowledge, and the associated pedagogies. To address these challenges, Ed+gineering, an NSF-funded multidisciplinary collaborative service learning project, was implemented into two sets of paired-classes in engineering and education: a 100 level mechanical engineering class (n = 42) and a foundations class in education (n = 17), and a fluid mechanics class in mechanical engineering technology (n = 23) and a science methods class (n = 15). The paired classes collaborated in multidisciplinary teams of 5-8 undergraduate students to plan and teach engineering lessons to local elementary school students. Teams completed a series of previously tested, scaffolded activities to guide their collaboration. Designing and delivering lessons engaged university students in collaborative processes that promoted social learning, including researching and planning, peer mentoring, teaching and receiving feedback, and reflecting and revising their engineering lesson. The research questions examined in this pilot, mixed-methods research study include: (1) How did PSTs’ Ed+gineering experiences influence their engineering and science knowledge?; (2) How did PSTs’ and UESs’ Ed+gineering experiences influence their pedagogical understanding?; and (3) What were PSTs’ and UESs’ overall perceptions of their Ed+gineering experiences? Both quantitative (e.g., Engineering Design Process assessment, Science Content Knowledge assessment) and qualitative (student reflections) data were used to assess knowledge gains and project perceptions following the semester-long intervention. Findings suggest that the PSTs were more aware and comfortable with the engineering field following lesson development and delivery, and often better able to explain particular science/engineering concepts. Both PSTs and UESs, but especially the latter, came to realize the importance of planning and preparing lessons to be taught to an audience. UESs reported greater appreciation for the work of educators. PSTs and UESs expressed how they learned to work in groups with multidisciplinary members—this is a valuable lesson for their respective professional careers. Yearly, the Ed+gineering research team will also request and review student retention reports in their respective programs to assess project impact.« less
5. Assessing the Reliability of a Chemical Engineering Problem-solving Rubric when Using Multiple Raters

   https://doi.org/10.18260/1-2--32124  

   Duckett, Timothy ; Liberatore, Matthew ; Asogwa, Uchenna ; Mentzer, Gale ; Malefyt, Amanda ( January 2019 , ASEE Annual Meeting)

   This evidence-based practices paper discusses the method employed in validating the use of a project modified version of the PROCESS tool (Grigg, Van Dyken, Benson, & Morkos, 2013) for measuring student problem solving skills. The PROCESS tool allows raters to score students’ ability in the domains of Problem definition, Representing the problem, Organizing information, Calculations, Evaluating the solution, Solution communication, and Self-assessment. Specifically, this research compares student performance on solving traditional textbook problems with novel, student-generated learning activities (i.e. reverse engineering videos in order to then create their own homework problem and solution). The use of student-generated learning activities tomore »assess student problem solving skills has theoretical underpinning in Felder’s (1987) work of “creating creative engineers,” as well as the need to develop students’ abilities to transfer learning and solve problems in a variety of real world settings. In this study, four raters used the PROCESS tool to score the performance of 70 students randomly selected from two undergraduate chemical engineering cohorts at two Midwest universities. Students from both cohorts solved 12 traditional textbook style problems and students from the second cohort solved an additional nine student-generated video problems. Any large scale assessment where multiple raters use a rating tool requires the investigation of several aspects of validity. The many-facets Rasch measurement model (MFRM; Linacre, 1989) has the psychometric properties to determine if there are any characteristics other than “student problem solving skills” that influence the scores assigned, such as rater bias, problem difficulty, or student demographics. Before implementing the full rating plan, MFRM was used to examine how raters interacted with the six items on the modified PROCESS tool to score a random selection of 20 students’ performance in solving one problem. An external evaluator led “inter-rater reliability” meetings where raters deliberated rationale for their ratings and differences were resolved by recourse to Pretz, et al.’s (2003) problem-solving cycle that informed the development of the PROCESS tool. To test the new understandings of the PROCESS tool, raters were assigned to score one new problem from a different randomly selected group of six students. Those results were then analyzed in the same manner as before. This iterative process resulted in substantial increases in reliability, which can be attributed to increased confidence that raters were operating with common definitions of the items on the PROCESS tool and rating with consistent and comparable severity. This presentation will include examples of the student-generated problems and a discussion of common discrepancies and solutions to the raters’ initial use of the PROCESS tool. Findings as well as the adapted PROCESS tool used in this study can be useful to engineering educators and engineering education researchers.« less