skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: A multi‐dimensional index of evaluating systems thinking skills from textual data
Abstract Systems thinking (ST) includes a set of critical skills and approaches for addressing today's complex societal problems. Therefore, it has been introduced into the curricula of many educational programmes around the world. Despite all the attention to ST, there is less consensus when it comes to evaluating and assessing ST skills. Particularly, a quantitative assessment approach that captures ST's multi‐dimensionality is crucial when evaluating the degree to which one has learned and utilizes ST. This paper proposes a systematic approach to create such a multi‐dimensional Index of ST from textual data. Initially, we provide an overview of the theoretical background as it pertains to different measurement approaches of ST skills. Then we provide a conceptual framework based on ST skill measures and transform this framework into a quantifiable model. Finally, using student data, we provide an illustration of an integrated index of ST skills. We compute this index by using a mixed methods approach, including robust principal component analysis, data envelopment analysis and two‐staged bootstrapping approach. The results show that (i) our model serves as a systematic multi‐dimensional ST approach by including multiple measures of ST skills and (ii) international students and self‐reported math skills are found as significant predictors of one's level of ST in the graduate student dataset (N = 30), however no significant factors are found in the first‐year engineering student dataset (N = 144).  more » « less
Award ID(s):
1824594
PAR ID:
10560789
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
International Federation for Systems Research
Date Published:
Journal Name:
Systems Research and Behavioral Science
ISSN:
1092-7026
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Skyline path queries (SPQs) extend skyline queries to multi-dimensional networks, such as multi-cost road networks (MCRNs). Such queries return a set of non-dominated paths between two given network nodes. Despite the existence of extensive works on evaluating different SPQ variants, SPQ evaluation is still very inefficient due to the nonexistence of efficient index structures to support such queries. Existing index building approaches for supporting shortest-path query execution, when directly extended to support SPQs, use an unreasonable amount of space and time to build, making them impractical for processing large graphs. In this paper, we propose a novel index structure,backbone index, and a corresponding index construction method that condenses an initial MCRN to multiple smaller summarized graphs with different granularity. We present efficient approaches to find approximate solutions to SPQs by utilizing the backbone index structure. Furthermore, considering making good use of historical query and query results, we propose two models,SkylinePathGraphNeuralNetwork (SP-GNN) andTransfer SP-GNN (TSP-GNN), to support effective SPQ processing. Our extensive experiments on real-world large road networks show that the backbone index can support finding meaningful approximate SPQ solutions efficiently. The backbone index can be constructed in a reasonable time, which dramatically outperforms the construction of other types of indexes for road networks. As far as we know, this is the first compact index structure that can support efficient approximate SPQ evaluation on large MCRNs. The results on the SP-GNN and TSP-GNN models also show that both models can help get approximate SPQ answers efficiently. 
    more » « less
  2. null (Ed.)
    Systems thinking (ST) skills are often the foundation of sustainability science curricula. Though ST skill sets are used as a basic approach to reasoning about complex environmental problems, there are gaps in our understanding regarding the best ways to promote and assess ST learning in classrooms. Since ST learning provides Science, Technology, Engineering, and Mathematics (STEM) students’ important skills and awareness to participate in environmental problem-solving, addressing these gaps is an important STEM learning contribution. We have created guidelines for teaching and measuring ST skills derived from a hybrid of a literature review and through case study data collection. Our approach is based on semi-quantitative cognitive mapping techniques meant to support deep reasoning about the complexities of social–ecological issues. We begin by arguing that ST should be evaluated on a continuum of understanding rather than a binary of correct/incorrect or present/absent. We then suggest four fundamental dimensions of teaching and evaluating ST which include: (1) system structure, (2) system function, (3) identification of leverage points for change, and (4) trade-off analysis. Finally, we use a case study to show how these ideas can be assessed through cognitive maps to help students develop deep system understanding and the capacity to propose innovative solutions to sustainability problems. 
    more » « less
  3. We present a novel deep learning framework to automatically compute independently salient networks in the brain that characterize the underlying changes in the brain in association with clinically observed assessments. Unsupervised approaches for high-dimensional neuroimaging data focus on computing low-dimensional brain components for subsequent analysis, while supervised learning approaches aim for predictive performance and yielding a single list of associative feature importance, thus making it hard to interpret at the level of brain subsystems. Our approach integrates the goals of decomposition into lower dimensional subspaces and, identifying salient brain subsystems into a single automated framework. We first train a convolutional neural network on structural brain features to predict clinical assessments, followed by a multi-step decomposition in the saliency space to compute salient brain networks that intrinsically characterize the brain changes associated with the assessment. Through a repeated training procedure on an Alzheimer’s disease (AD) dataset, we show that our method effectively computes AD-related salient brain subsystems directly from high-dimensional neuroimaging data, while maintaining predictive performance. Such approaches are crucial for data-driven biomarker development for brain disorders. 
    more » « less
  4. Abstract In our interconnected world, Systems Thinking (ST) is increasingly being recognized as a key learning goal for science education to help students make sense of complex phenomena. To support students in mastering ST, educators are advocating for using computational modeling programs. However, studies suggest that students often have challenges with using ST in the context of computational modeling. While previous studies have suggested that students have challenges modeling change over time through collector and flow structures and representing iterative processes through feedback loops, most of these studies investigated student ST through pre and post tests or through interviews. As such there is a gap in the literature regarding how student ST approaches develop and change throughout a computational modeling unit. In this case study, we aimed to determine which aspects of ST students found challenging during a computational modeling unit, how their approaches to ST changed over time, and how the learning environment was supporting students with ST. Building on prior frameworks, we developed a seven-category analysis tool that enabled us to use a mixture of student discourse, writing, and screen actions to categorize seven ST behaviors in real time. Through using this semi-quantitative tool and subsequent narrative analysis, we found evidence for all seven behavior categories, but not all categories were equally represented. Meanwhile our results suggest that opportunities for students to engage in discourse with both their peers and their teacher supported them with ST. Overall, this study demonstrates how student discourse and student writing can be important evidence of ST and serve as a potential factor to evaluate ST application as part of students’ learning progression. The case study also provides evidence for the positive impact that the implementation of a social constructivist approach has in the context of constructing computational system models. 
    more » « less
  5. The Python Foundations seminar emphasizes a hands-on approach and is designed to reach a diverse student population to provide them with the entry-level programming skills required in data science. The creation of the seminar materials has been funded by a three-year federal grant and has followed a systematic approach to enhance the student learning experience and raise student awareness of data science. The seminar materials—structured in three parts: pre-seminar, live seminar, and post-seminar—are licensed under the Creative Commons Attribution-ShareAlike 4.0 International license to encourage adoption by any institution. To foster interest in the seminar, students have the possibility to earn a digital badge attesting to their newly acquired skills. 
    more » « less