skip to main content


Title: Defining and Assessing Systems Thinking in Diverse Engineering Populations
Engineers are called to play an important role in addressing the complex problems of our global society, such as climate change and global health care. In order to adequately address these complex problems, engineers must be able to identify and incorporate into their decision making relevant aspects of systems in which their work is contextualized, a skill often referred to as systems thinking. However, within engineering, research on systems thinking tends to emphasize the ability to recognize potentially relevant constituent elements and parts of an engineering problem, rather than how these constituent elements and parts are embedded in broader economic, sociocultural, and temporal contexts and how all of these must inform decision making about problems and solutions. Additionally, some elements of systems thinking, such as an awareness of a particular sociocultural context or the coordination of work among members of a cross-disciplinary team, are not always recognized as core engineering skills, which alienates those whose strengths and passions are related to, for example, engineering systems that consider and impact social change. Studies show that women and minorities, groups underrepresented within engineering, are drawn to engineering in part for its potential to address important social issues. Emphasizing the importance of systems thinking and developing a more comprehensive definition of systems thinking that includes both constituent parts and contextual elements of a system will help students recognize the relevance and value of these other elements of engineering work and support full participation in engineering by a diverse group of students. We provide an overview of our study, in which we are examining systems thinking across a range of expertise to develop a scenario-based assessment tool that educators and researchers can use to evaluate engineering students’ systems thinking competence. Consistent with the aforementioned need to define and study systems thinking in a comprehensive, inclusive manner, we begin with a definition of systems thinking as a holistic approach to problem solving in which linkages and interactions of the immediate work with constituent parts, the larger sociocultural context, and potential impacts over time are identified and incorporated into decision making. In our study, we seek to address two key questions: 1) How do engineers of different levels of education and experience approach problems that require systems thinking? and 2) How do different types of life, educational, and work experiences relate to individuals’ demonstrated level of expertise in solving systems thinking problems? Our study is comprised of three phases. The first two phases include a semi-structured interview with engineering students and professionals about their experiences solving a problem requiring systems thinking and a think-aloud interview in which participants are asked to talk through how they would approach a given engineering scenario and later reflect on the experiences that inform their thinking. Data from these two phases will be used to develop a written assessment tool, which we will test by administering the written instrument to undergraduate and graduate engineering students in our third study phase. Our paper describes our study design and framing and includes preliminary findings from the first phase of our study.  more » « less
Award ID(s):
1733665
NSF-PAR ID:
10105632
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the American Society for Engineering Education Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. One of the main skills of engineers is to be able to solve problems. It is generally recognized that real-world engineering problems are inherently ill-structured in that they are complex, defined by non-engineering constraints, are missing information, and contain conflicting information. Therefore, it is very important to prepare future engineering students to be able to anticipate the occurrence of such problems, and to be prepared to solve them. However, most courses are taught by academic professors and lecturers whose focus is on didactic teaching of fundamental principles and code-based design approaches leading to predetermined “right” answers. Most classroom-taught methods to solve well-structured problems and the methods needed to solve ill-structured problems are strikingly different. The focus of our current effort is to compare and contrast the problem solving approaches employed by students, academics and practicing professionals in an attempt to determine if students are developing the necessary skills to tackle ill-structured problems. To accomplish this, an ill-structured problem is developed, which will later be used to determine, based on analysis of oral and written responses of participants in semi-structured interviews, attributes of the gap between student, faculty, and professional approaches to ill-structured problem solving. Based on the results of this analysis, we will identify what pedagogical approaches may limit and help students’ abilities to develop fully-formed solutions to ill-structured problems. This project is currently ongoing. This work-in-progress paper will present the study and proposed methods. Based on feedback obtained at the conference from the broader research community, the studies will be refined. The current phase includes three parts, (1) problem formulation; (2) protocol development; and (3) pilot study. For (1), two different ill-structured problems were developed in the Civil Engineering domain. The problem difficulty assessment method was used to determine the appropriateness of each problem developed for this study. For (2), a protocol was developed in which participants will be asked to first solve a simple problem to become familiar with the interview format, then are given 30 minutes to solve the provided ill-structured problem, following a semi-structured interview format. Participants will be encouraged to speak out loud and also write down what they are thinking and their thought processes throughout the interview period. Both (1) and (2) will next be used for (3) the pilot study. The pilot study will include interviewing three students, three faculty members and three professional engineers. Each participant will complete both problems following the same protocol developed. Post-interview discussion will be held with the pilot study participants individually to inquire if there were any portions of the tasks that are unclearly worded or could be improved to clarify what was being asked. Based on these results the final problem will be chosen and refined. 
    more » « less
  2. As problems become more complex, global, and interdisciplinary, engineers need to develop novel solutions and utilize resources, information, and tools in strategic and creative ways. Divergent thinking describes a process where multiple options, pathways, alternatives, or ideas are developed. For engineering students, divergent thinking can facilitate flexibility and expand opportunities considered when solving problems. To develop divergent thinking skills in engineering, we must understand how it is (and is not) facilitated in current engineering education experiences. Current pedagogy and resources available in engineering education on divergent thinking are limited. Thus, our research focused on exploring educational experiences in which students felt they considered divergent thinking. In this paper, we describe the iterative development of an interview protocol to elicit student experiences related to opportunities for divergent thinking. From the initial round of piloting, we found student awareness of divergent thinking was limited. Our findings highlight the need to structure questions in ways that align with students’ existing understandings of their engineering experiences. Our team made modifications to the protocol to address this, including using accessible terms to describe divergent thinking, asking students to describe one example project they remembered well, and. focusing questions within one step of the project selected by the student as most relevant to their exploration of alternatives. This iterative development of the protocol was successful in eliciting divergent thinking experiences across their work. 
    more » « less
  3. There have been numerous demands for enhancements in the way undergraduate learning occurs today, especially at a time when the value of higher education continues to be called into question (The Boyer 2030 Commission, 2022). One type of demand has been for the increased integration of subjects/disciplines around relevant issues/topics—with a more recent trend of seeking transdisciplinary learning experiences for students (Sheets, 2016; American Association for the Advancement of Science, 2019). Transdisciplinary learning can be viewed as the holistic way of working equally across disciplines to transcend their own disciplinary boundaries to form new conceptual understandings as well as develop new ways in which to address complex topics or challenges (Ertas, Maxwell, Rainey, & Tanik, 2003; Park & Son, 2010). This transdisciplinary approach can be important as humanity’s problems are not typically discipline specific and require the convergence of competencies to lead to innovative thinking across fields of study. However, higher education continues to be siloed which makes the authentic teaching of converging topics, such as innovation, human-technology interactions, climate concerns, or harnessing the data revolution, organizationally difficult (Birx, 2019; Serdyukov, 2017). For example, working across a university’s academic units to collaboratively teach, or co-teach, around topics of convergence are likely to be rejected by the university systems that have been built upon longstanding traditions. While disciplinary expertise is necessary and one of higher education’s strengths, the structures and academic rigidity that come along with the disciplinary silos can prevent modifications/improvements to the roles of academic units/disciplines that could better prepare students for the future of both work and learning. The balancing of disciplinary structure with transdisciplinary approaches to solving problems and learning is a challenge that must be persistently addressed. These institutional challenges will only continue to limit universities seeking toward scaling transdisciplinary programs and experimenting with novel ways to enhance the value of higher education for students and society. This then restricts innovations to teaching and also hinders the sharing of important practices across disciplines. To address these concerns, a National Science Foundation Improving Undergraduate STEM Education project team, which is the topic of this paper, has set the goal of developing/implementing/testing an authentically transdisciplinary, and scalable educational model in an effort to help guide the transformation of traditional undergraduate learning to span academics silos. This educational model, referred to as the Mission, Meaning, Making (M3) program, is specifically focused on teaching the crosscutting practices of innovation by a) implementing co-teaching and co-learning from faculty and students across different academic units/colleges as well as b) offering learning experiences spanning multiple semesters that immerse students in a community that can nourish both their learning and innovative ideas. As a collaborative initiative, the M3 program is designed to synergize key strengths of an institution’s engineering/technology, liberal arts, and business colleges/units to create a transformative undergraduate experience focused on the pursuit of innovation—one that reaches the broader campus community, regardless of students’ backgrounds or majors. Throughout the development of this model, research was conducted to help identify institutional barriers toward creating such a cross-college program at a research-intensive public university along with uncovering ways in which to address these barriers. While data can show how students value and enjoy transdisciplinary experiences, universities are not likely to be structured in a way to support these educational initiatives and they will face challenges throughout their lifespan. These challenges can result from administration turnover whereas mutual agreements across colleges may then vanish, continued disputes over academic territory, and challenges over resource allotments. Essentially, there may be little to no incentives for academic departments to engage in transdisciplinary programming within the existing structures of higher education. However, some insights and practices have emerged from this research project that can be useful in moving toward transdisciplinary learning around topics of convergence. Accordingly, the paper will highlight features of an educational model that spans disciplines along with the workarounds to current institutional barriers. This paper will also provide lessons learned related to 1) the potential pitfalls with educational programming becoming “un-disciplinary” rather than transdisciplinary, 2) ways in which to incentivize departments/faculty to engage in transdisciplinary efforts, and 3) new structures within higher education that can be used to help faculty/students/staff to more easily converge to increase access to learning across academic boundaries. 
    more » « less
  4. This is a research study that investigates the range of conceptions of prototyping in engineering design courses through exploring the conceptions and implementations from the instructors’ perspective. Prototyping is certainly an activity central to engineering design. The context of prototyping to support engineering education and practice has a range of implementations in an undergraduate engineering curriculum, from first-year engineering to capstone engineering design experiences. Understanding faculty conceptions’ of the reason, purpose, and place of prototyping can help illustrate how teaching and learning of the engineering design process is realistically implemented across a curriculum and how students are prepared for work practice. We seek to understand, and consequently improve, engineering design teaching and learning, through transformations of practice that are based on engineering education research. In this exploratory study, we interviewed three faculty members who teach engineering design in project-based learning courses across the curriculum of an undergraduate engineering program. This builds on related work done by the authors that previously investigated undergraduate engineering students’ conceptions of prototyping activities and process. With our instructor participants, a similar interview protocol was followed through semi-structured qualitative interviews. Data analysis has been undertaken through an emerging thematic analysis of these interview transcripts. Early findings characterize the focus on teaching the design process; the kind of feedback that the educators provide on students’ prototypes; students’ behavior while working on design projects; and educators’ perspectives on the design course. Understanding faculty conceptions with students’ conceptions of prototyping can shed light on the efficacy of using prototyping as an authentic experience in design teaching and learning. In project-based learning courses, particular issues of authenticity and assessment are under consideration, especially across the curriculum. More specifically, “proportions of problems” inform “problem solving” as one of the key characteristics in design thinking, teaching and learning. More attention to prototyping as part of the study of problem-solving processes can be useful to enhance understanding of the impact of instructional design. Challenges for teaching engineering design exist, and may be due to difficulties in framing design problems, recognizing what expertise students possess, and assessing their expertise to help them reach their goals, all at an appropriate place and ambiguity with student learning goals. Initial findings show that prototyping activities can help students become more reflective on their design. Scaffolded activities in prototyping can support self-regulated learning by students. The range of support and facilities, such as campus makerspaces, may also help students and instructors alike develop industry-ready engineering students. 
    more » « less
  5. Engineering problem solving has become more complex and reliant on technology making engineering judgement an increasingly important and essential skill for engineers. Educators need to ensure that students do not become rote learners with little ability to critically analyze the result of solutions. This suggests that greater focus should be placed on developing engineering judgement, specifically engineering intuition, in our students who will be the future engineering workforce. This project is focused on the following four research questions: 1) What are practicing professional engineers’ perceptions of discipline specific intuition and its use in the workplace? 2) Where does intuition manifest in expert engineer decision-making and problem-solving processes? 3) How does the motivation and identity of practicing professional engineers relate to discipline-specific intuition? 4) What would an instrument designed to validly and reliably measure engineering intuition look like? Literature from the fields of nursing (Smith), management (Simon), and expertise development (Dreyfus) suggest intuition plays a role in both decision making and becoming an expert. This literature is used to support our definition of engineering intuition which is defined as the ability to: 1) assess the feasibility of a solution or response, and 2) predict outcomes and/or options within an engineering scenario (Authors). This paper serves as an update on the progress of our work to date. The first three research questions have been addressed through interviews with engineering practitioners at various stages in their careers, from early career to retired. Emergent findings have allowed us to construct a modified definition of engineering intuition, while also identifying related constructs. In Spring 2021, we created and tested an instrument to measure intuition. This instrument was re-deployed in Fall 2021. Preliminary results from the project’s qualitative and quantitative efforts will be presented. Our ultimate aim of this project is to inform the creation of classroom practices that improve students’ ability to develop, recognize, and improve their own engineering intuition. Select References: Authors (2020). Dreyfus, Stuart E., and Hubert L. Dreyfus. A five-stage model of the mental activities involved in directed skill acquisition. No. ORC-80-2. California Univ Berkeley Operations Research Center, 1980. Smith, Anita. "Exploring the legitimacy of intuition as a form of nursing knowledge." Nursing Standard (through 2013) 23.40 (2009): 35. Simon, Herbert A. "Making management decisions: The role of intuition and emotion." Academy of Management Perspectives 1.1 (1987): 57-64. 
    more » « less