An official website of the United States government
Abstract The engineering of complex systems involves large number of individuals within multiple organizations spanning multiple years. Since it is challenging to perform empirical studies directly on real organizations at scale, some researchers in systems engineering and design have begun relying on abstracted model worlds that aim to be representative of the reference socio-technical system, but only preserve some aspects of it. However, there is a lack of corresponding knowledge on how to design representative model worlds for socio-technical research. Our objective is to create such knowledge through a reflective case study of the development of a model world. This “inner” study examines how two factors influence interdisciplinary communication during a concurrent design process. The reference real world system is a mission design laboratory (MDL) at NASA, and the model world is a simplified engine design problem in an undergraduate classroom environment. Our analysis focuses on the thought process followed, the key model world design decisions made, and a critical assessment of the extent to which communication phenomena in the model world (engine experiment) are representative of the real world (NASA's MDL). We find that the engine experiment preserves some but not all of the communication patterns of interest, and we present case-specific lessons learned for achieving and increasing representativeness in this type of study. More generally, we find that representativeness depends not on matching subjects, tasks, and context separately, but rather on the behavior that results from the interactions of these three dimensions.
more »
« less
Is it a judgment of representativeness? Re-examining the birth sequence problem
Although all birth orders in the “birth sequence problem” are equiprobable, most participants judge the less representative order as less likely than the more representative order. But this well-known problem confounds representativeness with the direction in which birth orders are compared. We hypothesized and corroborated in three experiments (total N = 1,136) that participants pragmatically infer the birth orders’ relative prevalence from the direction of comparison. Experiment 1 found that participants judged the less representative sequence as more common when we reversed the comparison. Experiment 2 reproduced these results despite removing representativeness as a cue. In Experiment 3, participants preferred to place the relatively common sequence as the referent in an inverted “speaker” problem. Our results turn the iconic problem’s interpretation on its head: Rather than indicating flawed human cognition, the birth sequence problem illustrates people’s ability to adaptively extract subtle linguistic meaning beyond the literal content.
more »
« less
- Award ID(s):
- 2049935
- PAR ID:
- 10481520
- Publisher / Repository:
- Springer Nature
- Date Published:
- Journal Name:
- Psychonomic Bulletin & Review
- Volume:
- 30
- Issue:
- 2
- ISSN:
- 1069-9384
- Page Range / eLocation ID:
- 731 to 738
- Subject(s) / Keyword(s):
- Biases Cognitive reference points Heuristics Judgment Perceptions of randomness Representativeness
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Limited language experience in childhood is common among deaf individuals, which prior research has shown to lead to low levels of language processing. Although basic structures such as word order have been found to be resilient to conditions of sparse language input in early life, whether they are robust to conditions of extreme language delay is unknown. The sentence comprehension strategies of post‐childhood, first‐language (L1) learners of American Sign Language (ASL) with at least 9 years of language experience were investigated, in comparison to two control groups of learners with full access to language from birth (deaf native signers and hearing L2 learners who were native English speakers). The results of a sentence‐to‐picture matching experiment show that event knowledge overrides word order for post‐childhood L1 learners, regardless of the animacy of the subject, while both deaf native signers and hearing L2 signers consistently rely on word order to comprehend sentences. Language inaccessibility throughout early childhood impedes the acquisition of even basic word order. Similar to the strategies used by very young children prior to the development of basic sentence structure, post‐childhood L1 learners rely more on context and event knowledge to comprehend sentences. Language experience during childhood is critical to the development of basic sentence structure.more » « less
-
Combining uncertainty information with AI recommendations supports calibration with domain knowledgeThe use of Artificial Intelligence (AI) decision support is increasing in high-stakes contexts, such as healthcare, defense, and finance. Uncertainty information may help users better leverage AI predictions, especially when combined with their domain knowledge. We conducted a human-subject experiment with an online sample to examine the effects of presenting uncertainty information with AI recommendations. The experimental stimuli and task, which included identifying plant and animal images, are from an existing image recognition deep learning model, a popular approach to AI. The uncertainty information was predicted probabilities for whether each label was the true label. This information was presented numerically and visually. In the study, we tested the effect of AI recommendations in a within-subject comparison and uncertainty information in a between-subject comparison. The results suggest that AI recommendations increased both participants’ accuracy and confidence. Further, providing uncertainty information significantly increased accuracy but not confidence, suggesting that it may be effective for reducing overconfidence. In this task, participants tended to have higher domain knowledge for animals than plants based on a self-reported measure of domain knowledge. Participants with more domain knowledge were appropriately less confident when uncertainty information was provided. This suggests that people use AI and uncertainty information differently, such as an expert versus second opinion, depending on their level of domain knowledge. These results suggest that if presented appropriately, uncertainty information can potentially decrease overconfidence that is induced by using AI recommendations.more » « less
-
Abstract Objective To assess whether exposure to high temperatures in pregnancy is associated with increased risk for preterm birth, low birth weight, and stillbirth. Design Systematic review and random effects meta-analysis. Data sources Medline and Web of Science searched up to September 2018, updated in August 2019. Eligibility criteria for selecting studies Clinical studies on associations between high environmental temperatures, and preterm birth, birth weight, and stillbirths. Results 14 880 records and 175 full text articles were screened. 70 studies were included, set in 27 countries, seven of which were countries with low or middle income. In 40 of 47 studies, preterm births were more common at higher than lower temperatures. Exposures were classified as heatwaves, 1°C increments, and temperature threshold cutoff points. In random effects meta-analysis, odds of a preterm birth rose 1.05-fold (95% confidence interval 1.03 to 1.07) per 1°C increase in temperature and 1.16-fold (1.10 to 1.23) during heatwaves. Higher temperature was associated with reduced birth weight in 18 of 28 studies, with considerable statistical heterogeneity. Eight studies on stillbirths all showed associations between temperature and stillbirth, with stillbirths increasing 1.05-fold (1.01 to 1.08) per 1°C rise in temperature. Associations between temperature and outcomes were largest among women in lower socioeconomic groups and at age extremes. The multiple temperature metrics and lag analyses limited comparison between studies and settings. Conclusions Although summary effect sizes are relatively small, heat exposures are common and the outcomes are important determinants of population health. Linkages between socioeconomic status and study outcomes suggest that risks might be largest in low and middle income countries. Temperature rises with global warming could have major implications for child health. Systematic review registration PROSPERO CRD 42019140136 and CRD 42018118113.more » « less
-
Despite the progress made toward generating and utilizing population-scale family trees to study historical population dynamics, little is known about their representativeness for the entire population. In this article, we confront the inherent complexities and biases in historical data collection and shed light on the extensive areas of history that remain unknown, unrecorded, or inaccurately portrayed. Although we do not provide definitive solutions for these data gaps, we aim to initiate a dialogue on these critical issues, contributing to the discourse on ethical data collection and representation in historical research. We first report on the preliminary results of a record linkage experiment between family tree records and a historical census, emphasizing the need for methods that integrate historical data from multiple sources to systematically evaluate representativeness. The experiment reveals significant underrepresentation of certain groups in the United States, notably Native American, Black, and Mexican persons, as well as those from eastern Europe, southern Europe, and Ireland. These findings underscore the ethical responsibilities that should guide historical research, including the need to address underrepresentation and improve methodologies to better reflect the diversity of population dynamics and migration patterns. To complement these efforts, we advocate for the use of interactive story maps to amplify the qualitative narratives of underrepresented populations and integrate them into the broader historical narrative. Our endeavor to map migration and demographic changes is not just about tracing the past; it’s about shaping a more equitable and comprehensive understanding of history that honors the diversity of all its participants.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3758/s13423-022-02188-9
