Abstract. In the geosciences, recent attention has been paid to the influence of uncertainty on expert decision making. When making decisions under conditions of uncertainty, people tend to employ heuristics (rules of thumb) based on experience, relying on their prior knowledge and beliefs to intuitively guide choice. Over 50 years of decision making research in cognitive psychology demonstrates that heuristics can lead to less-than-optimal decisions, collectively referred to as biases. For example, a geologist who confidently interprets ambiguous data as representative of a familiar category form their research (e.g., strike slip faults for expert in extensional domains) is exhibiting the availability bias, which occurs when people make judgments based on what is most dominant or accessible in memory. Given the important social and commercial implications of many geoscience decisions, there is a need to develop effective interventions for removing or mitigating decision bias. In this paper, we summarize the key insights from decision making research about how to reduce bias and review the literature on debiasing strategies. First, we define an optimal decision, since improving decision making requires having a standard to work towards. Next, we discuss the cognitive mechanisms underlying decision biases and describe three biases that have been shown to influence geoscientists decision making (availability bias, framing bias, anchoring bias). Finally, we review existing debiasing strategies that have applicability in the geosciences, with special attention given to those strategies that make use of information technology and artificial intelligence (AI). We present two case studies illustrating different applications of intelligent systems for the debiasing of geoscientific decision making, where debiased decision making is an emergent property of the coordinated and integrated processing of human-AI collaborative teams.
Prediction with expert advice applied to the problem of prediction with expert advice
We often need to have beliefs about things on which we are not experts. Luckily, we often have access to expert judgements on such topics. But how should we form our beliefs on the basis of expert opinion when experts conflict in their judgments? This is the core of the novice/2-expert problem in social epistemology. A closely related question is important in the context of policy making: how should a policy maker use expert judgments when making policy in domains in which she is not herself an expert? This question is more complex, given the messy and strategic nature of politics. In this paper we argue that the prediction with expert advice (PWEA) framework from machine learning provides helpful tools for addressing these problems. We outline conditions under which we should expert PWEA to be helpful and those under which we should not expect these methods to perform well.
more » « less- Award ID(s):
- 1922424
- NSF-PAR ID:
- 10372342
- Publisher / Repository:
- Springer Science + Business Media
- Date Published:
- Journal Name:
- Synthese
- Volume:
- 200
- Issue:
- 4
- ISSN:
- 1573-0964
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
In response to chemical process incidents, the ABET criteria for chemical engineering programs has expanded to include an emphasis on the understanding of hazards associated with chemical processes. This requirement has oftentimes been met with a focus on system design and requirements. However, experts are coming to recognize that human error and judgements can be contributing factors in serious accidents. Poor judgements are a risk of individuals inaccurately predicting their actions, and engineers are not immune to these risks, especially when they are considering how to make tradeoffs with process safety criteria. Where engineers may believe to be prioritizing safety, their behaviors may demonstrate otherwise, which risks the well-being of others. For example, the Pryor Trust well blowout and Chevron refinery explosion may have both been exacerbated due to engineers inadequately making trade offs between safety and productivity demands. It is possible to minimize poor judgments caused by inaccurate predictions by reconciling self-held beliefs with actions actually taken. The purpose of this paper is to describe a pilot study with five senior level engineering students that aims to facilitate understanding whether they have any gaps between their beliefs and behaviors regarding competing criteria in a process safety context. The project is driven by the following four research questions: 1) What do engineers believe about how they make judgements; 2) How do they behave when actually making judgements; 3) What gap, if any, exists between their beliefs and behavior; and 4) How do they reconcile any gap between their beliefs and behavior? To begin answering these questions, we will interview subjects on their beliefs using a semi-structured interview format. We will then obtain data on subjects’ actual behaviors through a recently developed process safety digital game, Contents Under Pressure. Finally, we will compare the subjects’ responses to similar dilemmas in both contexts to then generate a Gap Profile that provides a visual of differences, if they exist. Subjects will then be asked to reconcile their Gap Profile in a subsequent interview.more » « less
-
In response to chemical process incidents, the ABET criteria for chemical engineering programs has expanded to include an emphasis on the understanding of hazards associated with chemical processes. This requirement has oftentimes been met with a focus on system design and requirements. However, experts are coming to recognize that human error and judgements can be contributing factors in serious accidents. Poor judgements are a risk of individuals inaccurately predicting their actions, and engineers are not immune to these risks, especially when they are considering how to make tradeoffs with process safety criteria. Where engineers may believe to be prioritizing safety, their behaviors may demonstrate otherwise, which risks the well-being of others. For example, the Pryor Trust well blowout and Chevron refinery explosion may have both been exacerbated due to engineers inadequately making trade offs between safety and productivity demands. It is possible to minimize poor judgments caused by inaccurate predictions by reconciling self-held beliefs with actions actually taken. The purpose of this paper is to describe a pilot study with five senior level engineering students that aims to facilitate understanding whether they have any gaps between their beliefs and behaviors regarding competing criteria in a process safety context. The projectis driven by the following four research questions: 1) What do engineers believe about how they make judgements; 2) How do they behave when actually making judgements; 3) What gap, if any, exists between their beliefs and behavior; and 4) How do they reconcile any gap between their beliefs and behavior? To begin answering these questions, we will interview subjects on their beliefs using a semi-structured interview format. We will then obtain data on subjects’ actual behaviors through a recently developed process safety digital game, Contents Under Pressure. Finally, we will compare the subjects’ responses to similar dilemmas in both contexts to then generate a Gap Profile that provides a visual of differences, if they exist. Subjects will then be asked to reconcile their Gap Profile in a subsequent interview.more » « less
-
In response to chemical process incidents, the ABET criteria for chemical engineering programs has expanded to include an emphasis on the understanding of hazards associated with chemical processes. This requirement has oftentimes been met with a focus on system design and requirements. However, experts are coming to recognize that human error and judgements can be contributing factors in serious accidents. Poor judgements are a risk of individuals inaccurately predicting their actions, and engineers are not immune to these risks, especially when they are considering how to make tradeoffs with process safety criteria. Where engineers may believe to be prioritizing safety, their behaviors may demonstrate otherwise, which risks the well-being of others. For example, the Pryor Trust well blowout and Chevron refinery explosion may have both been exacerbated due to engineers inadequately making trade offs between safety and productivity demands. It is possible to minimize poor judgments caused by inaccurate predictions by reconciling self-held beliefs with actions actually taken. The purpose of this paper is to describe a pilot study with five senior level engineering students that aims to facilitate understanding whether they have any gaps between their beliefs and behaviors regarding competing criteria in a process safety context. The project is driven by the following four research questions: 1) What do engineers believe about how they make judgements; 2) How do they behave when actually making judgements; 3) What gap, if any, exists between their beliefs and behavior; and 4) How do they reconcile any gap between their beliefs and behavior? To begin answering these questions, we will interview subjects on their beliefs using a semi-structured interview format. We will then obtain data on subjects’ actual behaviors through a recently developed process safety digital game, Contents Under Pressure. Finally, we will compare the subjects’ responses to similar dilemmas in both contexts to then generate a Gap Profile that provides a visual of differences, if they exist. Subjects will then be asked to reconcile their Gap Profile in a subsequent interview.more » « less
-
In response to chemical process incidents, the ABET criteria for chemical engineering programs has expanded to include an emphasis on the understanding of hazards associated with chemical processes. This requirement has oftentimes been met with a focus on system design and requirements. However, experts are coming to recognize that human error and judgements can be contributing factors in serious accidents. Poor judgements are a risk of individuals inaccurately predicting their actions, and engineers are not immune to these risks, especially when they are considering how to make tradeoffs with process safety criteria. Where engineers may believe to be prioritizing safety, their behaviors may demonstrate otherwise, which risks the well-being of others. For example, the Pryor Trust well blowout and Chevron refinery explosion may have both been exacerbated due to engineers inadequately making trade offs between safety and productivity demands. It is possible to minimize poor judgments caused by inaccurate predictions by reconciling self-held beliefs with actions actually taken. The purpose of this paper is to describe a pilot study with five senior level engineering students that aims to facilitate understanding whether they have any gaps between their beliefs and behaviors regarding competing criteria in a process safety context. The project is driven by the following four research questions: 1) What do engineers believe about how they make judgements; 2) How do they behave when actually making judgements; 3) What gap, if any, exists between their beliefs and behavior; and 4) How do they reconcile any gap between their beliefs and behavior? To begin answering these questions, we will interview subjects on their beliefs using a semi-structured interview format. We will then obtain data on subjects’ actual behaviors through a recently developed process safety digital game, Contents Under Pressure. Finally, we will compare the subjects’ responses to similar dilemmas in both contexts to then generate a Gap Profile that provides a visual of differences, if they exist. Subjects will then be asked to reconcile their Gap Profile in a subsequent interview.more » « less
