An official website of the United States government
Artificial Intelligence (AI) is a transformative force in communication and messaging strategy, with potential to disrupt traditional approaches. Large language models (LLMs), a form of AI, are capable of generating high-quality, humanlike text. We investigate the persuasive quality of AI-generated messages to understand how AI could impact public health messaging. Specifically, through a series of studies designed to characterize and evaluate generative AI in developing public health messages, we analyze COVID-19 pro-vaccination messages generated by GPT-3, a state-of-the-art instantiation of a large language model. Study 1 is a systematic evaluation of GPT-3's ability to generate pro-vaccination messages. Study 2 then observed peoples' perceptions of curated GPT-3-generated messages compared to human-authored messages released by the CDC (Centers for Disease Control and Prevention), finding that GPT-3 messages were perceived as more effective, stronger arguments, and evoked more positive attitudes than CDC messages. Finally, Study 3 assessed the role of source labels on perceived quality, finding that while participants preferred AI-generated messages, they expressed dispreference for messages that were labeled as AI-generated. The results suggest that, with human supervision, AI can be used to create effective public health messages, but that individuals prefer their public health messages to come from human institutions rather than AI sources. We propose best practices for assessing generative outputs of large language models in future social science research and ways health professionals can use AI systems to augment public health messaging.
more »
« less
Generative agent‐based modeling: an introduction and tutorial
Abstract We discuss the emerging new opportunity for building feedback‐rich computational models of social systems using generative artificial intelligence. Referred to as generative agent‐based models (GABMs), such individual‐level models utilize large language models to represent human decision‐making in social settings. We provide a GABM case in which human behavior can be incorporated into simulation models by coupling a mechanistic model of human interactions with a pre‐trained large language model. This is achieved by introducing a simple GABM of social norm diffusion in an organization. For educational purposes, the model is intentionally kept simple. We examine a wide range of scenarios and the sensitivity of the results to several changes in the prompt. We hope the article and the model serve as a guide for building useful dynamic models of various social systems that include realistic human reasoning and decision‐making. © 2024 System Dynamics Society.
more »
« less
- Award ID(s):
- 2229819
- PAR ID:
- 10486128
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- System Dynamics Review
- Volume:
- 40
- Issue:
- 1
- ISSN:
- 0883-7066
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract The rise of Large Language Models (LLMs) and generative visual analytics systems has transformed data‐driven insights, yet significant challenges persist in accurately interpreting users analytical and interaction intents. While language inputs offer flexibility, they often lack precision, making the expression of complex intents inefficient, error‐prone, and time‐intensive. To address these limitations, we investigate the design space of multimodal interactions for generative visual analytics through a literature review and pilot brainstorming sessions. Building on these insights, we introduce a highly extensible workflow that integrates multiple LLM agents for intent inference and visualization generation. We develop InterChat, a generative visual analytics system that combines direct manipulation of visual elements with natural language inputs. This integration enables precise intent communication and supports progressive, visually driven exploratory data analyses. By employing effective prompt engineering, and contextual interaction linking, alongside intuitive visualization and interaction designs, InterChat bridges the gap between user interactions and LLM‐driven visualizations, enhancing both interpretability and usability. Extensive evaluations, including two usage scenarios, a user study, and expert feedback, demonstrate the effectiveness of InterChat. Results show significant improvements in the accuracy and efficiency of handling complex visual analytics tasks, highlighting the potential of multimodal interactions to redefine user engagement and analytical depth in generative visual analytics.more » « less
-
While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional information. We apply our approach, named ReAct, to a diverse set of language and decision making tasks and demonstrate its effectiveness over state-of-the-art baselines, as well as improved human interpretability and trustworthiness over methods without reasoning or acting components. Concretely, on question answering (HotpotQA) and fact verification (Fever), ReAct overcomes issues of hallucination and error propagation prevalent in chain-of-thought reasoning by interacting with a simple Wikipedia API, and generates human-like task-solving trajectories that are more interpretable than baselines without reasoning traces. On two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples.more » « less
-
null (Ed.)In the past decade, there has been a surge of interest in using games derived from experimental economics to test decision-making behaviour across species. In most cases, researchers are using the games as a tool, for instance, to understand what factors influence decision-making, how decision-making differs across species or contexts, or to ask broader questions about species’ propensities to cooperate or compete. These games have been quite successful in this regard. To what degree, however, do these games tap into species' economic decision-making? For the purpose of understanding the evolution of economic systems in humans, this is the key question. To study this, we can break economic decision-making down into smaller components, each of which is a potential step in the evolution of human economic behaviour. We can then use data from economic games, which are simplified, highly structured models of decision-making and therefore ideal for the comparative approach, to directly compare these components across species and contexts, as well as in relation to more naturalistic behaviours, to better understand the evolution of economic behaviour and the social and ecological contexts that influenced it. The comparative approach has successfully informed us about the evolution of other complex traits, such as language and morality, and should help us more deeply understand why and how human economic systems evolved. This article is part of the theme issue ‘Existence and prevalence of economic behaviours among non-human primates’.more » « less
-
Birchall, S Jeff (Ed.)This paper explores the concept of co-stewardship in the Arctic through the lens of the Study of Environmental Arctic Change’s Human Wellbeing (HWB) team. Rooted in Indigenous knowledge and collaborative science, our work prioritizes equity in decision-making, recognizing multiple knowledge systems as equally valuable. Through intentional team-building, trust, and reciprocity, we examine successes, challenges, and opportunities in co-stewardship. Key successes include fostering meaningful relationships, integrating Indigenous perspectives into scientific and policy discussions, and uplifting innovative knowledge-sharing tools such as oral histories and visual storytelling. However, structural challenges persist, including colonial policy frameworks, inadequate funding models, and a lack of institutional mechanisms to support Indigenous leadership in co-stewardship initiatives. We propose policy shifts, long-term funding commitments, and greater Indigenous representation in decision-making as steps toward meaningful change. This work underscores the importance of Indigenous-led stewardship in addressing Arctic environmental and social challenges, offering a model for collaborative governance rooted in respect and reciprocity.more » « less
