An official website of the United States government
Transformer language models have made tremendous strides in natural language understanding tasks. However, the complexity of natural language makes it challenging to ascertain how accurately these models are tracking the world state underlying the text. Motivated by this issue, we consider the task of language modeling for the game of chess. Unlike natural language, chess notations describe a simple, constrained, and deterministic domain. Moreover, we observe that the appropriate choice of chess notation allows for directly probing the world state, without requiring any additional probing-related machinery. We find that: (a) With enough training data, transformer language models can learn to track pieces and predict legal moves with high accuracy when trained solely on move sequences. (b) For small training sets providing access to board state information during training can yield significant improvements. (c) The success of transformer language models is dependent on access to the entire game history i.e. “full attention”. Approximating this full attention results in a significant performance drop. We propose this testbed as a benchmark for future work on the development and analysis of transformer language models.
more »
« less
Cultural transmission of move choice in chess
The study of cultural evolution benefits from detailed analysis of cultural transmission in specific human domains. Chess provides a platform for understanding the transmission of knowledge due to its active community of players, precise behaviours and long-term records of high-quality data. In this paper, we perform an analysis of chess in the context of cultural evolution, describing multiple cultural factors that affect move choice. We then build a population-level statistical model of move choice in chess, based on the Dirichlet-multinomial likelihood, to analyse cultural transmission over decades of recorded games played by leading players. For moves made in specific positions, we evaluate the relative effects of frequency-dependent bias, success bias and prestige bias on the dynamics of move frequencies. We observe that negative frequency-dependent bias plays a role in the dynamics of certain moves, and that other moves are compatible with transmission under prestige bias or success bias. These apparent biases may reflect recent changes, namely the introduction of computer chess engines and online tournament broadcasts. Our analysis of chess provides insights into broader questions concerning how social learning biases affect cultural evolution.
more »
« less
- Award ID(s):
- 2116322
- PAR ID:
- 10522033
- Publisher / Repository:
- The Royal Society Publishing
- Date Published:
- Journal Name:
- Proceedings of the Royal Society B: Biological Sciences
- Volume:
- 290
- Issue:
- 2011
- ISSN:
- 0962-8452
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Extensive work in the social sciences suggests that vaccination decisions are subject to incentives, biases, and social learning processes, including prestige bias transmission. High status figures, like doctors and public health officials, can be effective messengers for vaccination information and uptake under certain conditions. In communities where there is significant medical mistrust and less interaction with markets and formal medical systems, prestige bias social learning may operate through different channels. Here, we examine the role of prestige bias on vaccine decisions in two ethnic groups (Himba and Herero) with varying levels of market integration and experiences with formal healthcare systems. Participants completed a ranking task, comparing the influence of four prestigious individuals on vaccine decisions and a survey on medical mistrust. Using Plackett-Luce models, we compare the influence of location, ethnic affiliation, and other covariates on rankings. A multi-level model compared the influence of those within and outside one's ethnic group, as well as specialist (doctor/healer) and generalist (chief/governor) prestige figures. Results indicate changes in the rank of prestigious individuals across the rural-urban gradient. Our results demonstrate significant variability in prestige-biased social learning about vaccine decision making. Medical mistrust did not impact rankings. Contrary to previous work, we find that whether a prestigious individual is locally prominent is more important than their expertise in the relevant domain (health and healing). These findings emphasize the need for more context-specific studies of prestige bias, which can improve our understanding of healthcare decision-making and guide public health messaging across diverse contexts.more » « less
-
How do cultural biases, trust in government, and perceptions of risk and protective actions influence compliance with regulation of COVID-19? Analyzing Chinese (n = 646) and American public opinion samples (n = 1,325) from spring 2020, we use Grid–Group Cultural Theory and the Protective Action Decision Model to specify, respectively, cultural influences on public risk perceptions and decision-making regarding protective actions. We find that cultural biases mostly affect protective actions indirectly through public perceptions. Regardless of country, hierarchical cultural biases increase protective behaviors via positive perceptions of protective actions. However, other indirect effects of cultural bias via public perceptions vary across both protective actions and countries. Moreover, trust in government only mediates the effect of cultural bias in China and risk perception only mediates the effect of cultural bias in the United States. Our findings suggest that regulators in both countries should craft regulations that are congenial to culturally diverse populations.more » « less
-
Cumulative cultural evolution (CCE) occurs among humans who may be presented with many similar options from which to choose, as well as many social influences and diverse environments. It is unknown what general principles underlie the wide range of CCE dynamics and whether they can all be explained by the same unified paradigm. Here, we present a scalable evolutionary model of discrete choice with social learning, based on a few behavioural science assumptions. This paradigm connects the degree of transparency in social learning to the human tendency to imitate others. Computer simulations and quantitative analysis show the interaction of three primary factors—information transparency, popularity bias and population size—drives the pace of CCE. The model predicts a stable rate of evolutionary change for modest degrees of popularity bias. As popularity bias grows, the transition from gradual to punctuated change occurs, with maladaptive subpopulations arising on their own. When the popularity bias gets too severe, CCE stops. This provides a consistent framework for explaining the rich and complex adaptive dynamics taking place in the real world, such as modern digital media.more » « less
-
The Affect Heuristic-Cultural Cognition Theory (AH-CCT) model and the Solution Aversion-based (SA) model both suggest affect, meaning feelings or discrete emotions about a target, mediates associations between ‘culture,’ such as political ideology or cultural biases, and risk responses, such as risk perceptions, protective behaviours, and supportive attitudes towards protective policy. However, the models differ respectively by defining negative affect as directed towards the hazard (‘hazard affect’) or a specific behaviour or policy response (‘solution aversion,’ negative affect about a proposed risk reduction method). We compare these models with longitudinal mediation analysis of U.S. COVID-19 survey data (n = 866 in smallest-sample wave). Solution aversion accounted for more associations of culture with risk perceptions, such as personal risk, collective risk, and risk severity; behaviour and behavioural intentions, regarding mask wearing, avoiding large public gatherings, and vaccination; and support for risk mitigation policies, regarding mask mandates, public gathering bans, and vaccination mandates. Statistically significant direct effects were rare and were mainly for egalitarian cultural bias; indirect effects occurred for egalitarians, political conservatives, and individualists. Implications for further research on risk responses are discussed relative to limited previous work on these affect-mediation models.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1098/rspb.2023.1634
