An official website of the United States government
The functions of an autonomous system can generally be partitioned into those concerned with perception and those concerned with action. Perception builds and maintains an internal model of the world (i.e., the system's environment) that is used to plan and execute actions to accomplish a goal established by human supervisors. Accordingly, assurance decomposes into two parts: a) ensuring that the model is an accurate representation of the world as it changes through time and b) ensuring that the actions are safe (and e ective), given the model. Both perception and action may employ AI, including machine learning (ML), and these present challenges to assurance. However, it is usually feasible to guard the actions with traditionally engineered and assured monitors, and thereby ensure safety, given the model. Thus, the model becomes the central focus for assurance. We propose an architecture and methods to ensure the accuracy of models derived from sensors whose interpretation uses AI and ML. Rather than derive the model from sensors bottom-up, we reverse the process and use the model to predict sensor interpretation. Small prediction errors indicate the world is evolving as expected and the model is updated accordingly. Large prediction errors indicate surprise, which may be due to errors in sensing or interpretation, or unexpected changes in the world (e.g., a pedestrian steps into the road). The former initiate error masking or recovery, while the latter requires revision to the model. Higher-level AI functions assist in diagnosis and execution of these tasks. Although this two-level architecture where the lower level does \predictive processing" and the upper performs more re ective tasks, both focused on maintenance of a world model, is derived by engineering considerations, it also matches a widely accepted theory of human cognition.
more »
« less
Minimizing and quantifying uncertainty in AI-informed decisions: Applications in medicine
AI is now a cornerstone of modern dataset analysis. In many real world applications, practitioners are concerned with controlling specific kinds of errors, rather than minimizing the overall number of errors. For example, biomedical screening assays may primarily be concerned with mitigating the number of false positives rather than false negatives. Quantifying uncertainty in AI-based predictions, and in particular those controlling specific kinds of errors, remains theoretically and practically challenging. We develop a strategy called multidimensional informed generalized hypothesis testing (MIGHT) which we prove accurately quantifies uncertainty and confidence given sufficient data, and concomitantly controls for particular error types. Our key insight was that it is possible to integrate canonical cross-validation and parametric calibration procedures within a nonparametric ensemble method. Simulations demonstrate that while typical AI based-approaches cannot be trusted to obtain the truth, MIGHT can be. We apply MIGHT to answer an open question in liquid biopsies using circulating cell-free DNA (ccfDNA) in individuals with or without cancer: Which biomarkers, or combinations thereof, can we trust? Performance estimates produced by MIGHT on ccfDNA data have coefficients of variation that are often orders of magnitude lower than other state of the art algorithms such as support vector machines, random forests, and Transformers, while often also achieving higher sensitivity. We find that combinations of variable sets often decrease rather than increase sensitivity over the optimal single variable set because some variable sets add more noise than signal. This work demonstrates the importance of quantifying uncertainty and confidence—with theoretical guarantees—for the interpretation of real-world data.
more »
« less
- PAR ID:
- 10636371
- Author(s) / Creator(s):
- ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more »
- Publisher / Repository:
- National Academy of Sciences
- Date Published:
- Journal Name:
- Proceedings of the National Academy of Sciences
- Volume:
- 122
- Issue:
- 34
- ISSN:
- 0027-8424
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Cook, S.; Infante, N. (Ed.)In this theoretical paper, our aim is to start a conversation about how "levels" in mathematics are operationalized and defined, with a specific focus on "college level." We approach this from a lens of developmental stages, using this to propose an initial framework for describing how learners might progress along a developmental continuum delineated by the kinds of reasoning/justification, generalization/abstraction, and types of conceptions that they hold, rather than by the particular computations learners are able to do, or the kinds of mathematical objects with which learners are engaging.more » « less
-
Abstract Root zone water storage capacity () sets the lower limit on the water available to plants and is critical for modeling terrestrial water and carbon fluxes. Due to subsurface heterogeneity in soils, roots, and geology, in‐situ measurements of are infeasible at regional to global scales. Instead, is often estimated using a water balance approach—calculating the difference between precipitation and evapotranspiration to approximate water deficits. This method can be applied using diverse data sets, including those from eddy covariance, satellite remote sensing, reanalysis, and land surface model outputs. However, each data product contains its own uncertainties, and the influence of data set choice on estimates has not been quantified. In this study, we assess the uncertainty in estimates across 90 FLUXNET sites in the continental United States with at least 5 years of data, comparing results from six different precipitation and evapotranspiration data sets. We find substantial variation in estimates across data sets, sites, and land cover types, with particularly high uncertainty for values exceeding 500 mm. Much of this uncertainty can be attributed to differences among evapotranspiration products rather than random errors. Our analysis shows that single data set approaches to quantifying underestimate the variability that emerges when multiple data sets are considered, especially in the Western US. These results underscore the need to account for data‐driven uncertainty when estimating subsurface water storage and provide a foundation for improving model parameterizations of ecosystem drought response and land‐atmosphere interactions.more » « less
-
Artificial intelligence (AI) and machine learning models are being increasingly deployed in real-world applications. In many of these applications, there is strong motivation to develop hybrid systems in which humans and AI algorithms can work together, leveraging their complementary strengths and weaknesses. We develop a Bayesian framework for combining the predictions and different types of confidence scores from humans and machines. The framework allows us to investigate the factors that influence complementarity, where a hybrid combination of human and machine predictions leads to better performance than combinations of human or machine predictions alone. We apply this framework to a large-scale dataset where humans and a variety of convolutional neural networks perform the same challenging image classification task. We show empirically and theoretically that complementarity can be achieved even if the human and machine classifiers perform at different accuracy levels as long as these accuracy differences fall within a bound determined by the latent correlation between human and machine classifier confidence scores. In addition, we demonstrate that hybrid human–machine performance can be improved by differentiating between the errors that humans and machine classifiers make across different class labels. Finally, our results show that eliciting and including human confidence ratings improve hybrid performance in the Bayesian combination model. Our approach is applicable to a wide variety of classification problems involving human and machine algorithms.more » « less
-
Recent advances in the study of artificial and biological neural networks support the power of dynamic representations-computing with information stored as nontrivial limit-sets rather than fixed-point attractors. Understanding and manipulating these computations in nonlinear networks requires a theory of control for abstract objective functions. Towards this end, we consider two properties of limit-sets: their topological dimension and orientation (covariance) in phase space and combine these abstract properties into a single well-defined objective: conic control-invariant sets in the derivative space (i.e., the vector field). Real-world applications, such as neural-medicine, constrain which control laws are feasible with less-invasive controllers being preferable. To this end, we derive a feedback control-law for conic invariance which corresponds to constrained restructuring of the network connections as might occur with pharmacological intervention (as opposed to a physically separate control unit). We demonstrate the ease and efficacy of the technique in controlling the orientation and dimension of limit sets in high-dimensional neural networks.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1073/pnas.2424203122
