skip to main content

Title: Assessing Interdisciplinary Competency in the Disaster Resilience and Risk Management Graduate Program using Concept Maps
Concept maps have emerged as a valid and reliable method for assessing deep conceptual understanding in engineering education within disciplines as well as interdisciplinary knowledge integration across disciplines. Most work on concept maps, however, focuses on undergraduates. In this paper, we use concept maps to examine changes in graduate students’ conceptual understanding and knowledge integration resulting from an interdisciplinary graduate program. Our study context is pair of foundational, team-taught courses in an interdisciplinary Disaster Resilience and Risk Management (DRRM) graduate program. The courses include a 3-hour research course and a 1-hour seminar that aim to build student understanding within and across Urban Affairs and Planning, Civil and Environmental Engineering, Geosciences, and Business Information Technology. The courses introduce core principles of DRRM and relevant research methods in these disciplines, and drive students to understand the intersections of these disciplines in the context of planning for and responding to natural and human-made disasters. To understand graduate student growth from disciplinary-based to interdisciplinary scholars, we pose the research questions: 1) In what ways do graduate students’ understandings of DRRM change as a result of their introduction to an interdisciplinary graduate research program? and 2) To what extent and in what ways do concept more » maps serve as a tool to capture interdisciplinary learning in this context? Data includes pre/post concept maps centered on disaster resilience and risk management, a one-page explanation of the post-concept map, and ethnographic field notes gathered from class and faculty meetings. Pre-concept maps were collected on the first day of class; post-concept maps will be collected as part of the final course assignment. We assess the students’ concept maps for depth of conceptual understanding within disciplines and interdisciplinary competency across disciplines, using the field notes to provide explanatory context. The results presented in this paper support the inclusion of an explanation component to concept maps, and also suggest that concept maps alone may not be the best measure of student understanding of concepts within and across disciplines in this specific context. If similar programs wish to use concept maps as an assessment method, we suggest the inclusion of an explanation component and suggest providing explicit instructions that specify the intended audience. We also suggest using a holistic scoring method, as it is more likely to capture nuances in the concept maps than traditional scoring methods, which focus solely on counting factors like hierarchies and number of cross-links. « less
Authors:
; ; ; ;
Award ID(s):
1735139
Publication Date:
NSF-PAR ID:
10108834
Journal Name:
2019 ASEE Annual Conference and Exposition
Sponsoring Org:
National Science Foundation
More Like this
  1. Disasters are becoming more frequent as the global climate changes, and recovery efforts require the cooperation and collaboration of experts and community members across disciplines. The DRRM program, funded through the National Science Foundation (NSF) Research Traineeship (NRT), is an interdisciplinary graduate program that brings together faculty and graduate students from across the university to develop new, transdisciplinary ways of solving disaster-related issues. The core team includes faculty from business, engineering, education, science, and urban planning fields. The overall objective of the program is to create a community of practice amongst the graduate students and faculty to improve understanding andmore »support proactive decision-making related to disasters and disaster management. The specific educational objectives of the program are (1) context mastery and community building, (2) transdisciplinary integration and professional development, and (3) transdisciplinary research. The program’s educational research and assessment activities include program development, trainee learning and development, programmatic educational research, and institutional transformation. The program is now in its fourth year of student enrollment. Core courses on interdisciplinary research methods in disaster resilience are in place, engaging students in domain-specific research related to natural hazards, resilience, and recovery, and in methods of interdisciplinary and transdisciplinary collaboration. In addition to courses, the program offers a range of professional development opportunities through seminars and workshops. Since the program’s inception, the core team has expanded both the numbers of faculty and students and the range of academic disciplines involved in the program, including individuals from additional science and engineering fields as well as those from natural resources and the social sciences. At the same time, the breadth of disciplines and the constraints of individual academic programs have posed substantial structural challenges in engaging students in the process of building interdisciplinary research identities and in building the infrastructure needed to sustain the program past the end of the grant. Our poster and paper will identify major program accomplishments, but also draw on interviews with students to examine the structural challenges and potential solution paths associated with a program of this breadth. Critical opportunities for sustainability and engagement have emerged through integration with a larger university-level center as well as through increased flexibility in program requirements and additional mechanisms for student and faculty collaboration.« less
  2. Biodiversity is a complex, yet essential, concept for undergraduate students in ecology and other natural sciences to grasp. As beginner scientists, students must learn to recognize, describe, and interpret patterns of biodiversity across various spatial scales and understand their relationships with ecological processes and human influences. It is also increasingly important for undergraduate programs in ecology and related disciplines to provide students with experiences working with large ecological datasets to develop students’ data science skills and their ability to consider how ecological processes that operate at broader spatial scales (macroscale) affect local ecosystems. To support the goals of improving studentmore »understanding of macroscale ecology and biodiversity at multiple spatial scales, we formed an interdisciplinary team that included grant personnel, scientists, and faculty from ecology and spatial sciences to design a flexible learning activity to teach macroscale biodiversity concepts using large datasets from the National Ecological Observatory Network (NEON). We piloted this learning activity in six courses enrolling a total of 109 students, ranging from midlevel ecology and GIS/remote sensing courses, to upper-level conservation biology. Using our classroom experiences and a pre/post assessment framework, we evaluated whether our learning activity resulted in increased student understanding of macroscale ecology and biodiversity concepts and increased familiarity with analysis techniques, software programs, and large spatio-ecological datasets. Overall, results suggest that our learning activity improved student understanding of biological diversity, biodiversity metrics, and patterns of biodiversity across several spatial scales. Participating faculty reflected on what went well and what would benefit from changes, and we offer suggestions for implementation of the learning activity based on this feedback. This learning activity introduced students to macroscale ecology and built student skills in working with big data (i.e., large datasets) and performing basic quantitative analyses, skills that are essential for the next generation of ecologists.« less
  3. There is a critical need for more students with engineering and computer science majors to enter into, persist in, and graduate from four-year postsecondary institutions. Increasing the diversity of the workforce by inclusive practices in engineering and science is also a profound identified need. According to national statistics, the largest groups of underrepresented minority students in engineering and science attend U.S. public higher education institutions. Most often, a large proportion of these students come to colleges and universities with unique challenges and needs, and are more likely to be first in their family to attend college. In response to thesemore »needs, engineering education researchers and practitioners have developed, implemented and assessed interventions to provide support and help students succeed in college, particularly in their first year. These interventions typically target relatively small cohorts of students and can be managed by a small number of faculty and staff. In this paper, we report on “work in progress” research in a large-scale, first-year engineering and computer science intervention program at a public, comprehensive university using multivariate comparative statistical approaches. Large-scale intervention programs are especially relevant to minority serving institutions that prepare growing numbers of students who are first in their family to attend college and who are also under-resourced, financially. These students most often encounter academic difficulties and come to higher education with challenging experiences and backgrounds. Our studied first-year intervention program, first piloted in 2015, is now in its 5th year of implementation. Its intervention components include: (a) first-year block schedules, (b) project-based introductory engineering and computer science courses, (c) an introduction to mechanics course, which provides students with the foundation needed to succeed in a traditional physics sequence, and (d) peer-led supplemental instruction workshops for calculus, physics and chemistry courses. This intervention study responds to three research questions: (1) What role does the first-year intervention’s components play in students’ persistence in engineering and computer science majors across undergraduate program years? (2) What role do particular pedagogical and cocurricular support structures play in students’ successes? And (3) What role do various student socio-demographic and experiential factors play in the effectiveness of first-year interventions? To address these research questions and therefore determine the formative impact of the firstyear engineering and computer science program on which we are conducting research, we have collected diverse student data including grade point averages, concept inventory scores, and data from a multi-dimensional questionnaire that measures students’ use of support practices across their four to five years in their degree program, and diverse background information necessary to determine the impact of such factors on students’ persistence to degree. Background data includes students’ experiences prior to enrolling in college, their socio-demographic characteristics, and their college social capital throughout their higher education experience. For this research, we compared students who were enrolled in the first-year intervention program to those who were not enrolled in the first-year intervention. We have engaged in cross-sectional 2 data collection from students’ freshman through senior years and employed multivariate statistical analytical techniques on the collected student data. Results of these analyses were interesting and diverse. Generally, in terms of backgrounds, our research indicates that students’ parental education is positively related to their success in engineering and computer science across program years. Likewise, longitudinally (across program years), students’ college social capital predicted their academic success and persistence to degree. With regard to the study’s comparative research of the first-year intervention, our results indicate that students who were enrolled in the first-year intervention program as freshmen continued to use more support practices to assist them in academic success across their degree matriculation compared to students who were not in the first-year program. This suggests that the students continued to recognize the value of such supports as a consequence of having supports required as first-year students. In terms of students’ understanding of scientific or engineering-focused concepts, we found significant impact resulting from student support practices that were academically focused. We also found that enrolling in the first-year intervention was a significant predictor of the time that students spent preparing for classes and ultimately their grade point average, especially in STEM subjects across students’ years in college. In summary, we found that the studied first-year intervention program has longitudinal, positive impacts on students’ success as they navigate through their undergraduate experiences toward engineering and computer science degrees.« less
  4. Wyoming recently mandated that computer science instruction be provided in K-12 schools by 2022, and there is an urgent need for designing instruction that can integrate computer science into the teaching of other subjects. This project assembles a network improvement community comprised of partners from the University of Wyoming, community colleges, Wyoming school districts, the Wyoming Library System, the Wyoming Department of Education, and local software development firms. The community meets once monthly over the duration of the project to collaborate stakeholder agendas for meeting the project goals. The community enlists K-8 teachers from across the state to experience professionalmore »development and collaborate on integrating computer science into their instruction of STEM and social science topics. The project is producing units for teachers, who are implementing these units with support from master teachers and educational scholars. The community serves as a forum for teachers to debrief and learn from each other about ways to improve their instruction and design of the curricular units. Libraries in the state system act as partners for dissemination to rural areas of the innovative instructional approaches. WySLICE prepares 150 K-8 teachers and state librarians from all disciplines to integrate computer science into their teaching. The project is reaching almost half of all K-8 students in Wyoming. The research questions address how teachers use modeling practices as supports for student understanding of algorithms and coding in a variety of ways. The curricula involve cybersecurity as well as other topics relevant to measurement in mathematics and social studies topics that involve social concerns like voting. Data sources include teacher lesson plans and recordings of their instructional implementation, scoring of each of these according to a rubric, meeting notes of monthly meetings, and results from pre-post student assessments. The evaluation focuses on the meeting of project goals and the quality of the management of the network improvement community. This project is jointly funded by CS for All and the Established Program to Stimulate Competitive Research (EPSCoR). This work is supported by the National Science Foundation under DRL Grant #1923542 "CS For All:RPP - Booting Up Computer Science in Wyoming."« less
  5. Graduate training often takes a monodisciplinary approach that is not informed by best practices, ignores the needs and preferences of students, and overlooks the increasingly interdisciplinary and international nature of research. This is unfortunate, particularly since graduate education that is fully integrated with interdisciplinary research can help students become part of a trained and diverse workforce equipped to meet society’s many challenges. Against this backdrop, a National Science Foundation Research Traineeship (NRT) program is being established at the University of Kentucky leveraging the most effective instruments for the training of STEM professionals, such as network-based graduate student mentoring and careermore »preparation encompassing both technical and professional skillsets. Briefly, the training graduate students will receive – in a way that is fully integrated with the research they perform – includes: 1) tools such as individual development plans and developmental network maps; 2) a multi-departmental and interdisciplinary course on research-related content; 3) a seminar course on transferrable skills (ethics, research, communication, teaching, mentoring, entrepreneurship, teamwork, management, leadership, outreach, etc.); 4) a certificate to be awarded once students complete the two courses above and garner additional credits from an interdisciplinary curriculum of research-related courses; 5) summer internships at other departments and at external institutions (other universities, industry, national laboratories) nationwide or abroad; 6) an annual research-related symposium including all elements of a scientific conference; 7) internal collaborative research grants for participants to fund and pursue their own ideas; 8) fields trips to facilities related to the research; and 9) coaching on job hunting as well as résumé, motivation letter and interview preparation. Since a workforce equipped to meet society’s challenges must be both well trained and diverse, multiple initiatives will ensure that this NRT will broaden participation in STEM. Recruitment-wise, close collaboration with a number of entities will provide this NRT with a broad recruitment pool of talented and diverse students. Moreover, collaboration with these entities will provide trainees with ample opportunities to acquire, practice and refine their professional skills, as trainees present their results and recruit in conferences, meetings and outreach events organized by these entities, become members and/or join their leadership, and expand their professional and mentoring network in the process. In addition, minority trainees will be surveyed periodically to probe their feelings of well-being, preparation, acceptance, belonging and distress, as well as their perception of how well structured their departments and programs are. According to recent literature, these factors determine whether or not they perform (i.e., publish) at rates comparable to their male majority peers. Saliently, the evaluation of the educational model employed will afford a comprehensive understanding not only of the academy components that were more utilized and impactful, but will reveal the individual mentoring and skill-building facets of the program driving its successful implementation. The evaluation plan includes outcomes, performance measures, an evaluation timetable, benchmarks and a description of how formative evaluation will improve practice, the evaluation process also extending to research activities.« less