skip to main content

Title: Assessing Interdisciplinary Competency in the Disaster Resilience and Risk Management Graduate Program using Concept Maps
Concept maps have emerged as a valid and reliable method for assessing deep conceptual understanding in engineering education within disciplines as well as interdisciplinary knowledge integration across disciplines. Most work on concept maps, however, focuses on undergraduates. In this paper, we use concept maps to examine changes in graduate students’ conceptual understanding and knowledge integration resulting from an interdisciplinary graduate program. Our study context is pair of foundational, team-taught courses in an interdisciplinary Disaster Resilience and Risk Management (DRRM) graduate program. The courses include a 3-hour research course and a 1-hour seminar that aim to build student understanding within and across Urban Affairs and Planning, Civil and Environmental Engineering, Geosciences, and Business Information Technology. The courses introduce core principles of DRRM and relevant research methods in these disciplines, and drive students to understand the intersections of these disciplines in the context of planning for and responding to natural and human-made disasters. To understand graduate student growth from disciplinary-based to interdisciplinary scholars, we pose the research questions: 1) In what ways do graduate students’ understandings of DRRM change as a result of their introduction to an interdisciplinary graduate research program? and 2) To what extent and in what ways do concept more » maps serve as a tool to capture interdisciplinary learning in this context? Data includes pre/post concept maps centered on disaster resilience and risk management, a one-page explanation of the post-concept map, and ethnographic field notes gathered from class and faculty meetings. Pre-concept maps were collected on the first day of class; post-concept maps will be collected as part of the final course assignment. We assess the students’ concept maps for depth of conceptual understanding within disciplines and interdisciplinary competency across disciplines, using the field notes to provide explanatory context. The results presented in this paper support the inclusion of an explanation component to concept maps, and also suggest that concept maps alone may not be the best measure of student understanding of concepts within and across disciplines in this specific context. If similar programs wish to use concept maps as an assessment method, we suggest the inclusion of an explanation component and suggest providing explicit instructions that specify the intended audience. We also suggest using a holistic scoring method, as it is more likely to capture nuances in the concept maps than traditional scoring methods, which focus solely on counting factors like hierarchies and number of cross-links. « less
; ; ; ;
Award ID(s):
Publication Date:
Journal Name:
2019 ASEE Annual Conference and Exposition
Sponsoring Org:
National Science Foundation
More Like this
  1. Disasters are becoming more frequent as the global climate changes, and recovery efforts require the cooperation and collaboration of experts and community members across disciplines. The DRRM program, funded through the National Science Foundation (NSF) Research Traineeship (NRT), is an interdisciplinary graduate program that brings together faculty and graduate students from across the university to develop new, transdisciplinary ways of solving disaster-related issues. The core team includes faculty from business, engineering, education, science, and urban planning fields. The overall objective of the program is to create a community of practice amongst the graduate students and faculty to improve understanding and support proactive decision-making related to disasters and disaster management. The specific educational objectives of the program are (1) context mastery and community building, (2) transdisciplinary integration and professional development, and (3) transdisciplinary research. The program’s educational research and assessment activities include program development, trainee learning and development, programmatic educational research, and institutional transformation. The program is now in its fourth year of student enrollment. Core courses on interdisciplinary research methods in disaster resilience are in place, engaging students in domain-specific research related to natural hazards, resilience, and recovery, and in methods of interdisciplinary and transdisciplinary collaboration. In addition to courses,more »the program offers a range of professional development opportunities through seminars and workshops. Since the program’s inception, the core team has expanded both the numbers of faculty and students and the range of academic disciplines involved in the program, including individuals from additional science and engineering fields as well as those from natural resources and the social sciences. At the same time, the breadth of disciplines and the constraints of individual academic programs have posed substantial structural challenges in engaging students in the process of building interdisciplinary research identities and in building the infrastructure needed to sustain the program past the end of the grant. Our poster and paper will identify major program accomplishments, but also draw on interviews with students to examine the structural challenges and potential solution paths associated with a program of this breadth. Critical opportunities for sustainability and engagement have emerged through integration with a larger university-level center as well as through increased flexibility in program requirements and additional mechanisms for student and faculty collaboration.« less
  2. Abstract Background

    Engineering education traditionally emphasizes technical skills, sometimes at the cost of under‐preparing graduates for the real‐world engineering context. In recent decades, attempts to address this issue include increasing project‐based assignments and engineering design courses in curricula; however, a skills gap between education and industry remains.


    This study aims to understand how undergraduate engineering students perceive product design before and after an upper‐level project‐based design course, as measured through concept maps. The purpose is to measure whether and how students account for the technical and nontechnical elements of design, as well as how a third‐year design course influences these design perceptions.


    Concept maps about product design were collected from 105 third‐year engineering students at the beginning and end of a design course. Each concept map's content and structure were quantitatively analyzed to evaluate the students' conceptual understandings and compare them across disciplines in the before and after conditions.


    The analyses report on how student conceptions differ by discipline at the outset and how they changed after taking the course. Mechanical Engineering students showed a decrease in business‐related content and an increased focus on societal content, while students in the Engineering Management and Industrial and Systems Engineering programs showed an increasemore »in business topics, specifically market‐related content.


    This study reveals how undergraduate students conceptualize product design, and specifically to what extent they consider engineering, business, and societal factors. The design courses were shown to significantly shape student conceptualizations of product design, and they did so in a way that mirrored the topics in the course syllabi. The findings offer insights into the education‐practice skills gap and may help future educators to better prepare engineering students to meet industry needs.

    « less
  3. The CUAHSI Virtual University is an interinstitutional graduate training framework that was developed to increase access to specialized hydrology courses for graduate students from participating US institutions. The program was designed to capitalize on the benefits of collaborative teaching, allowing students to differentiate their learning and access subject matter experts at multiple institutions, while enrolled in a single course at their home institution, through a framework of reciprocity. Although the CUAHSI Virtual University was developed prior to the COVID-19 pandemic, the resilience of its online education model to such disruptions to classroom teaching increases the urgency of understanding how effective such an approach is at achieving its goals and what challenges multi-institutional graduate training faces for sustainability and expansion within the water sciences or in other disciplines. To gain faculty perspectives on the program, we surveyed (1) water science graduate program faculty who had served as instructors in the program, (2) water science graduate program faculty who were aware of the program, but had not participated, and (3) departmental chairs of participating instructors. Our data show widespread agreement across respondent types that the program is positive for students, diversifying their educational opportunities and increasing access to subject matter experts. Concernsmore »and factors limiting faculty involvement revolved around faculty workload and administrative barriers, including low enrollment at individual institutions. If these barriers can be surmounted, the CUAHSI Virtual University has the potential for wider participation within hydrology and adoption in other STEM disciplines.« less
  4. Abstract

    Interdisciplinary teams are on the rise as scientists attempt to address complex environmental issues. While the benefits of team science approaches are clear, researchers often struggle with its implementation, particularly for new team members. The challenges of large projects often weigh on the most vulnerable members of a team: trainees, including undergraduate students, graduate students, and post‐doctoral researchers. Trainees on big projects have to navigate their role on the team, with learning project policies, procedures, and goals, all while also training in key scientific tasks such as co‐authoring papers. To address these challenges, we created and participated in a project‐specific, graduate‐level team science course. The purposes of this course were to: (1) introduce students to the goals of the project, (2) build trainees' understanding of how big projects operate, and (3) allow trainees to explore how their research interests dovetailed with the overall project. Additionally, trainees received training regarding: (1) diversity, equity & inclusion, (2) giving and receiving feedback, and (3) effective communication. Onboarding through the team science course cultivated psychological safety and a collaborative student community across disciplines and institutions. Thus, we recommend a team science course for onboarding students to big projects to help students establish themore »skills necessary for collaborative research. Project‐based team science classes can benefit student advancement, enhance the productivity of the project, and accelerate the discovery of solutions to ecological issues by building community, establishing a shared project vocabulary, and building a workforce with collaborative skills to better answer ecological research questions.

    « less
  5. There is a critical need for more students with engineering and computer science majors to enter into, persist in, and graduate from four-year postsecondary institutions. Increasing the diversity of the workforce by inclusive practices in engineering and science is also a profound identified need. According to national statistics, the largest groups of underrepresented minority students in engineering and science attend U.S. public higher education institutions. Most often, a large proportion of these students come to colleges and universities with unique challenges and needs, and are more likely to be first in their family to attend college. In response to these needs, engineering education researchers and practitioners have developed, implemented and assessed interventions to provide support and help students succeed in college, particularly in their first year. These interventions typically target relatively small cohorts of students and can be managed by a small number of faculty and staff. In this paper, we report on “work in progress” research in a large-scale, first-year engineering and computer science intervention program at a public, comprehensive university using multivariate comparative statistical approaches. Large-scale intervention programs are especially relevant to minority serving institutions that prepare growing numbers of students who are first in their family tomore »attend college and who are also under-resourced, financially. These students most often encounter academic difficulties and come to higher education with challenging experiences and backgrounds. Our studied first-year intervention program, first piloted in 2015, is now in its 5th year of implementation. Its intervention components include: (a) first-year block schedules, (b) project-based introductory engineering and computer science courses, (c) an introduction to mechanics course, which provides students with the foundation needed to succeed in a traditional physics sequence, and (d) peer-led supplemental instruction workshops for calculus, physics and chemistry courses. This intervention study responds to three research questions: (1) What role does the first-year intervention’s components play in students’ persistence in engineering and computer science majors across undergraduate program years? (2) What role do particular pedagogical and cocurricular support structures play in students’ successes? And (3) What role do various student socio-demographic and experiential factors play in the effectiveness of first-year interventions? To address these research questions and therefore determine the formative impact of the firstyear engineering and computer science program on which we are conducting research, we have collected diverse student data including grade point averages, concept inventory scores, and data from a multi-dimensional questionnaire that measures students’ use of support practices across their four to five years in their degree program, and diverse background information necessary to determine the impact of such factors on students’ persistence to degree. Background data includes students’ experiences prior to enrolling in college, their socio-demographic characteristics, and their college social capital throughout their higher education experience. For this research, we compared students who were enrolled in the first-year intervention program to those who were not enrolled in the first-year intervention. We have engaged in cross-sectional 2 data collection from students’ freshman through senior years and employed multivariate statistical analytical techniques on the collected student data. Results of these analyses were interesting and diverse. Generally, in terms of backgrounds, our research indicates that students’ parental education is positively related to their success in engineering and computer science across program years. Likewise, longitudinally (across program years), students’ college social capital predicted their academic success and persistence to degree. With regard to the study’s comparative research of the first-year intervention, our results indicate that students who were enrolled in the first-year intervention program as freshmen continued to use more support practices to assist them in academic success across their degree matriculation compared to students who were not in the first-year program. This suggests that the students continued to recognize the value of such supports as a consequence of having supports required as first-year students. In terms of students’ understanding of scientific or engineering-focused concepts, we found significant impact resulting from student support practices that were academically focused. We also found that enrolling in the first-year intervention was a significant predictor of the time that students spent preparing for classes and ultimately their grade point average, especially in STEM subjects across students’ years in college. In summary, we found that the studied first-year intervention program has longitudinal, positive impacts on students’ success as they navigate through their undergraduate experiences toward engineering and computer science degrees.« less