Large‐scale complex engineered systems are designed in large organizations consisting of hundreds or thousands of individuals making design decisions at different stages in the process. These complex systems are inherently multidisciplinary and uncertain, involving diverse disciplines spanning geographical locations. Without a means to guide their decisions, these many decision‐makers fall back on selecting a design according to their own value and risk preferences. The primary focus of this paper is to enable consistency in design decision‐making with respect to both value and risk preferences of the stakeholder. A new value‐based systems engineering (VBSE) framework is presented that enables physics‐based system consistency as well as consistency in communication of value and risk preferences for design decision‐making in a hierarchically decomposed system. This framework embraces principles of value‐driven design and multidisciplinary design optimization, while also ensuring consistency in risk preferences throughout the system to address uncertainty. Improper communication of risk preferences could result in nonoptimal designs, which are risk biased by the design decision‐makers at different levels of the hierarchy, with respect to the stakeholder's preferences. Consistency in risk preferences in the context of the VBSE framework is addressed in this research by incorporating decision analysis techniques.
In this article, a new framework for improving risk assessments of novel genetically engineered organisms (GEOs) is developed and applied. The Procedurally Robust Risk Assessment Framework (PRRAF) provides a set of principles and criteria for assessing and enhancing risk assessment protocols for GEOs under conditions of high uncertainty. The application of PRRAF is demonstrated using the case of a genetically engineered mosquito designed to kill its wild population and therefore decrease disease transmission. Assessments for regulatory approval of this genetically engineered insect fall short of several PPRAF criteria under the principles of humility, procedural validity, inclusion, anticipation, and reflexivity. With the emergence of GEOs designed to spread in ecosystems, such as those with gene drives, it will become increasingly important for regulatory agencies and technology developers to bolster their risk analysis methods and processes prior to field testing. PRRAF can be used as a flexible guide for doing so within a variety of institutional, regulatory, and governance contexts.
more » « less- NSF-PAR ID:
- 10359765
- Publisher / Repository:
- Wiley-Blackwell
- Date Published:
- Journal Name:
- Regulation & Governance
- Volume:
- 15
- Issue:
- 4
- ISSN:
- 1748-5983
- Page Range / eLocation ID:
- p. 1144-1165
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
more » « less
Abstract Background and objectives Universities throughout the USA increasingly offer undergraduate courses in evolutionary medicine (EvMed), which creates a need for pedagogical resources. Several resources offer course content (e.g. textbooks) and a previous study identified EvMed core principles to help instructors set learning goals. However, assessment tools are not yet available. In this study, we address this need by developing an assessment that measures students’ ability to apply EvMed core principles to various health-related scenarios.
Methodology The EvMed Assessment (EMA) consists of questions containing a short description of a health-related scenario followed by several likely/unlikely items. We evaluated the assessment’s validity and reliability using a variety of qualitative (expert reviews and student interviews) and quantitative (Cronbach’s α and classical test theory) methods. We iteratively revised the assessment through several rounds of validation. We then administered the assessment to undergraduates in EvMed and Evolution courses at multiple institutions.
Results We used results from the pilot to create the EMA final draft. After conducting quantitative validation, we deleted items that failed to meet performance criteria and revised items that exhibited borderline performance. The final version of the EMA consists of six core questions containing 25 items, and five supplemental questions containing 20 items.
Conclusions and implications The EMA is a pedagogical tool supported by a wide range of validation evidence. Instructors can use it as a pre/post measure of student learning in an EvMed course to inform curriculum revision, or as a test bank to draw upon when developing in-class assessments, quizzes or exams.
-
more » « less
Societal Impact Statement The current rate of global biodiversity loss creates a pressing need to increase efficiency and throughput of extinction risk assessments in plants. We must assess as many plant species as possible, working with imperfect knowledge, to address the habitat loss and extinction threats of the Anthropocene. Using the biodiversity database, Botanical Information and Ecology Network (BIEN), and the Andropogoneae grass tribe as a case study, we demonstrate that large‐scale, preliminary conservation assessments can play a fundamental role in accelerating plant conservation pipelines and setting priorities for more in‐depth investigations.
Summary The International Union for the Conservation of Nature (IUCN) Red List criteria are widely used to determine extinction risks of plant and animal life. Here, we used The Red List's criterion B, Geographic Range Size, to provide preliminary conservation assessments of the members of a large tribe of grasses, the Andropogoneae, with ~1100 species, including maize, sorghum, and sugarcane and their wild relatives.
We used georeferenced occurrence data from the Botanical Information and Ecology Network (BIEN) and automated individual species assessments using ConR to demonstrate efficacy and accuracy in using time‐saving tools for conservation research. We validated our results with those from the IUCN‐recommended assessment tool, GeoCAT.
We discovered a remarkably large gap in digitized information, with slightly more than 50% of the Andropogoneae lacking sufficient information for assessment. ConR and GeoCAT largely agree on which taxa are of least concern (>90%) or possibly threatened (<10%), highlighting that automating assessments with ConR is a viable strategy for preliminary conservation assessments of large plant groups. Results for crop wild relatives are similar to those for the entire dataset.
Increasing digitization and collection needs to be a high priority. Available rapid assessment tools can then be used to identify species that warrant more comprehensive investigation.
-
Knowledge quality assessment (KQA) has been developed in order to analyze the role of knowledge in situations of high stakes and urgency when characterized by deep uncertainty and ignorance. Governing coastal flood risk in the face of climate change is typical of such situations. These are situations which limit the ability to establish objective, reliable, and valid facts. This paper aims to identify the moral frameworks that stakeholders use to judge flood risk situations under climate change, and infer from these knowledge legitimacy criteria. Knowledge legitimacy, defined as being respectful of stakeholders' divergent values and beliefs, is one of the three broad quality criteria that have been proposed in order to assess knowledge quality in such situations; credibility (as scientific adequacy) and salience (relevance to the needs of decision makers) being the two others. Knowledge legitimacy is essentially the subject of a literature analyzing, ex-post (once knowledge has been deployed), how stakeholders' participation is a factor contributing to knowledge legitimacy. Very little is known about ex-ante characteristics (i.e.: that can be observed, determined, before knowledge is deployed) that would make some types of knowledge more legitimate (i.e., respectful of stakeholders' divergent values and beliefs) than others. We see this as a significant blind spot in the analysis of knowledge and its role under deep uncertainty. In this paper we posit that this blind spot may be addressed, in part. In order to achieve this we first identify the ethical frameworks that stakeholders use to judge a situation of risk under rapidly changing conditions. We then associate these ethical frameworks to characteristics of knowledge. We tested this conceptualization through a case study approach centered on flood risk on the French Atlantic coast. We have adopted a narrative approach to the analysis of two diachronic corpora consisting of interviews conducted in 2010–2012 (33 interviews) and 2020 (15 interviews). These were approached as narratives of a risk situation. We thematically coded these along themes considered as metanarratives. These metanarratives are associated with predefined (deontology, consequentialism, virtue ethics) and emerging (discourse ethics, connection ethics, and a naturalistic ethic) ethical theories. Our results show that, when faced with flood risks, stakeholders tell stories that mobilizes several metaethical frameworks as guiding principles in the form of both procedural and substantive injunctions. In order to respect what we interpret as manifestations of the moral stances of stakeholders, our results indicate that knowledge legitimacy may be assessed against the following criteria: lability, debatability and adaptability; degree of co-production invested; place-based approach; ability to include lessons that would be given by nature. The operationalization of these criteria is promising in a time when the knowledge that is used for decision making under certainty is increasingly contested on the ground of its legitimacy.more » « less
-
more » « less
Abstract There are over 700 aquatic ecological assessment approaches across the globe that meet specific institutional goals. However, in many cases, multiple assessment tools are designed to meet the same management need, resulting in a confusing array of overlapping options. Here, we look at six riverine wetland assessments currently in use in Montana, USA, and ask which tool (1) best captures the condition across a disturbance gradient and (2) has the most utility to meet the regulatory or management needs. We used descriptive statistics to compare wetland assessments (
n = 18) across a disturbance gradient determined by a landscape development intensity. Factor analysis showed that many of the tools had internal metrics that did not correspond well with overall results, hindering the tool’s ability to act as designed. We surveyed regional wetland managers (n = 56) to determine the extent of their use of each of the six tools and how well they trusted the information the assessment tool provided. We found that the Montana Wetland Assessment Methodology best measured the range of disturbance and had the highest utility to meet Clean Water Act (CWA§ 404) needs. Montana Department of Environmental Quality was best for the CWA§ 303(d) & 305(b) needs. The US Natural Resources Conservation Service’s Riparian Assessment Tool was the third most used by managers but was the tool that had the least ability to distinguish across a disturbance, followed by the US Bureau of Land Management’s Proper Functioning Condition.
