More Like this

1. Transparency Enhances Positive Perceptions of Social Artificial Intelligence

   https://doi.org/10.1155/2023/5550418  

   Xu, Ying; Bradford, Nora; Garg, Radhika (September 2023, Human Behavior and Emerging Technologies)

   Chakraborty, Pinaki (Ed.)

   Social chatbots are aimed at building emotional bonds with users, and thus it is particularly important to design these technologies so as to elicit positive perceptions from users. In the current study, we investigate the impacts that transparent explanations of chatbots’ mechanisms have on users’ perceptions of the chatbots. A total of 914 participants were recruited from Amazon Mechanical Turk. They were randomly assigned to observe conversations between a hypothetical chatbot and a user in one of the two-by-two experimental conditions: whether the participants received an explanation about how the chatbot was trained and whether the chatbot was framed as an intelligent entity or a machine. A fifth group, who believed they were observing interactions between two humans, served as a control. Analyses of participants’ responses to the postobservation survey indicated that transparency positively affected perceptions of social chatbots by leading users to (1) find the chatbot less creepy, (2) feel greater affinity to the chatbot, and (3) perceive the chatbot as more socially intelligent, though these effects were small. Moreover, transparency appeared to have a larger effect on increasing the perceived social intelligence among participants with lower prior AI knowledge. These findings have implications for the design of future social chatbots and support the addition of transparency and explanation for chatbot users. 

   more » « less
2. Chatbots Language Design: The Influence of Language Variation on User Experience with Tourist Assistant Chatbots

   https://doi.org/10.1145/3487193  

   Chaves, Ana Paula; Egbert, Jesse; Hocking, Toby; Doerry, Eck; Gerosa, Marco Aurelio (April 2022, ACM Transactions on Computer-Human Interaction)

   Chatbots are often designed to mimic social roles attributed to humans. However, little is known about the impact of using language that fails to conform to the associated social role. Our research draws on sociolinguistic to investigate how a chatbot’s language choices can adhere to the expected social role the agent performs within a context. We seek to understand whether chatbots design should account for linguistic register. This research analyzes how register differences play a role in shaping the user’s perception of the human-chatbot interaction. We produced parallel corpora of conversations in the tourism domain with similar content and varying register characteristics and evaluated users’ preferences of chatbot’s linguistic choices in terms of appropriateness, credibility, and user experience. Our results show that register characteristics are strong predictors of user’s preferences, which points to the needs of designing chatbots with register-appropriate language to improve acceptance and users’ perceptions of chatbot interactions. 

   more » « less
3. Putting ChatGPT’s Medical Advice to the (Turing) Test: Survey Study

   https://doi.org/10.2196/46939  

   Nov, Oded; Singh, Nina; Mann, Devin (January 2023, JMIR Medical Education)

   Background Chatbots are being piloted to draft responses to patient questions, but patients’ ability to distinguish between provider and chatbot responses and patients’ trust in chatbots’ functions are not well established. Objective This study aimed to assess the feasibility of using ChatGPT (Chat Generative Pre-trained Transformer) or a similar artificial intelligence–based chatbot for patient-provider communication. Methods A survey study was conducted in January 2023. Ten representative, nonadministrative patient-provider interactions were extracted from the electronic health record. Patients’ questions were entered into ChatGPT with a request for the chatbot to respond using approximately the same word count as the human provider’s response. In the survey, each patient question was followed by a provider- or ChatGPT-generated response. Participants were informed that 5 responses were provider generated and 5 were chatbot generated. Participants were asked—and incentivized financially—to correctly identify the response source. Participants were also asked about their trust in chatbots’ functions in patient-provider communication, using a Likert scale from 1-5. Results A US-representative sample of 430 study participants aged 18 and older were recruited on Prolific, a crowdsourcing platform for academic studies. In all, 426 participants filled out the full survey. After removing participants who spent less than 3 minutes on the survey, 392 respondents remained. Overall, 53.3% (209/392) of respondents analyzed were women, and the average age was 47.1 (range 18-91) years. The correct classification of responses ranged between 49% (192/392) to 85.7% (336/392) for different questions. On average, chatbot responses were identified correctly in 65.5% (1284/1960) of the cases, and human provider responses were identified correctly in 65.1% (1276/1960) of the cases. On average, responses toward patients’ trust in chatbots’ functions were weakly positive (mean Likert score 3.4 out of 5), with lower trust as the health-related complexity of the task in the questions increased. Conclusions ChatGPT responses to patient questions were weakly distinguishable from provider responses. Laypeople appear to trust the use of chatbots to answer lower-risk health questions. It is important to continue studying patient-chatbot interaction as chatbots move from administrative to more clinical roles in health care. 

   more » « less
4. Social Media Co-pilot: Designing a chatbot with teens and educators to combat cyberbullying

   https://doi.org/10.1016/j.ijcci.2024.100680  

   Zou, Wenting; Yang, Qian; DiFranzo, Dominic; Chen, Melissa; Hui, Winice; Bazarova, Natalie N (August 2024, International Journal of Child-Computer Interaction)

   Teens often encounter cyberbullying on social media. One promising way to reduce cyberbullying is through empowering teens to stand up for their peers and cultivating prosocial norms online. While there is no shortage of bystander interventions that have shown potential, little research has explored designing chatbots with users to provide a contextualized and embedded “learning at the moment” experience for bystanders. This study involved teens and educators in two design sessions: an in-depth interview to identify the barriers that prevent upstanding behaviors, and interaction with the “social media co-pilot'' chatbot prototype to identify design guidelines to empower teens to overcome these barriers. Qualitative analysis on the conversations from the two design sessions revealed three factors that curb teens' upstanding behaviors: a) inadequate knowledge about social norms, appropriate language, and consequences, b) inhibitive emotions such as fear of retaliation and confrontation; c) lack of empathy toward their peers. Key parameters were also identified to shape chatbot responses to encourage upstanding behaviors, such as a) adopting voices representing multiple roles, b) empathetic, friendly and encouraging tone, c) reflective, specific and relatable language and d) appropriate length. These insights inform the design of personalized and scalable education programs and moderation tools to combat cyberbullying. 

   more » « less
5. CommunityBots: Creating and Evaluating A Multi-Agent Chatbot Platform for Public Input Elicitation

   https://doi.org/10.1145/3579469  

   Jiang, Zhiqiu; Rashik, Mashrur; Panchal, Kunjal; Jasim, Mahmood; Sarvghad, Ali; Riahi, Pari; DeWitt, Erica; Thurber, Fey; Mahyar, Narges (April 2023, Proceedings of the ACM on Human-Computer Interaction)

   In recent years, the popularity of AI-enabled conversational agents or chatbots has risen as an alternative to traditional online surveys to elicit information from people. However, there is a gap in using single-agent chatbots to converse and gather multi-faceted information across a wide variety of topics. Prior works suggest that single-agent chatbots struggle to understand user intentions and interpret human language during a multi-faceted conversation. In this work, we investigated how multi-agent chatbot systems can be utilized to conduct a multi-faceted conversation across multiple domains. To that end, we conducted a Wizard of Oz study to investigate the design of a multi-agent chatbot for gathering public input across multiple high-level domains and their associated topics. Next, we designed, developed, and evaluated CommunityBots - a multi-agent chatbot platform where each chatbot handles a different domain individually. To manage conversation across multiple topics and chatbots, we proposed a novel Conversation and Topic Management (CTM) mechanism that handles topic-switching and chatbot-switching based on user responses and intentions. We conducted a between-subject study comparing CommunityBots to a single-agent chatbot baseline with 96 crowd workers. The results from our evaluation demonstrate that CommunityBots participants were significantly more engaged, provided higher quality responses, and experienced fewer conversation interruptions while conversing with multiple different chatbots in the same session. We also found that the visual cues integrated with the interface helped the participants better understand the functionalities of the CTM mechanism, which enabled them to perceive changes in textual conversation, leading to better user satisfaction. Based on the empirical insights from our study, we discuss future research avenues for multi-agent chatbot design and its application for rich information elicitation. 

   more » « less