Conversational systems typically focus on functional tasks such as scheduling appointments or creating todo lists. Instead we design and evaluate SlugBot (SB), one of 8 semifinalists in the 2018 AlexaPrize, whose goal is to support casual open-domain social inter-action. This novel application requires both broad topic coverage and engaging interactive skills. We developed a new technical approach to meet this demanding situation by crowd-sourcing novel content and introducing playful conversational strategies based on storytelling and games. We collected over 10,000 conversations during August 2018 as part of the Alexa Prize competition. We also conducted an in-lab follow-up qualitative evaluation. Over-all users found SB moderately engaging; conversations averaged 3.6 minutes and involved 26 user turns. However, users reacted very differently to different conversation subtypes. Storytelling and games were evaluated positively; these were seen as entertaining with predictable interactive structure. They also led users to impute personality and intelligence to SB. In contrast, search and general Chit-Chat induced coverage problems; here users found it hard to infer what topics SB could understand, with these conversations seen as being too system-driven. Theoretical and design implications suggest a move away from conversational systems that simply provide factual information. Future systems should be designed to have their own opinions with personal stories to share, and SB provides an example of how we might achieve this.
more »
« less
Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversations
We present Chirpy Cardinal, an open-domain dialogue agent, as a research platform
for the 2019 Alexa Prize competition. Building an open-domain socialbot
that talks to real people is challenging – such a system must meet multiple user
expectations such as broad world knowledge, conversational style, and emotional
connection. Our socialbot engages users on their terms – prioritizing their interests,
feelings and autonomy. As a result, our socialbot provides a responsive, personalized
user experience, capable of talking knowledgeably about a wide variety of
topics, as well as chatting empathetically about ordinary life. Neural generation
plays a key role in achieving these goals, providing the backbone for our conversational
and emotional tone. At the end of the competition, Chirpy Cardinal
progressed to the finals with an average rating of 3.6/5.0, a median conversation
duration of 2 minutes 16 seconds, and a 90th percentile duration of over 12 minutes.
more »
« less
- Award ID(s):
- 1900638
- NSF-PAR ID:
- 10318326
- Date Published:
- Journal Name:
- 3rd Proceedings of Alexa Prize (Alexa Prize 2019)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We present Chirpy Cardinal, an open-domain social chatbot. Aiming to be both informative and conversational, our bot chats with users in an authentic, emotionally intelligent way. By integrating controlled neural generation with scaffolded, hand-written dialogue, we let both the user and bot take turns driving the conversation, producing an engaging and socially fluent experience. Deployed in the fourth iteration of the Alexa Prize Socialbot Grand Challenge, Chirpy Cardinal handled thousands of conversations per day, placing second out of nine bots with an average user rating of 3.58/5.more » « less
-
Agency is essential to play. As we design conversational agents for early childhood, how might we increase the child-centeredness of our approaches? Giving children agency and control in choosing their agent representations might contribute to the overall playfulness of our designs. In this study with 33 children ages 4–5 years old, we engaged children in a creative storytelling interaction with conversational agents in stuffed animal embodiments. Young children conversed with the stuffed animal agents to tell stories about their creative play, engaging in question and answer conversation from 2 minutes to 24 minutes. We then interviewed the children about their perceptions of the agent’s voice, and their ideas for agent voices, dialogues, and interactions. From babies to robot daddies, we discover three themes from children’s suggestions: Family Voices, Robot Voices, and Character Voices. Additionally, children desire agents who (1) scaffold creative play in addition to storytelling, (2) foster personal, social, and emotional connections, and (3) support children’s agency and control. Across these themes, we recommend design strategies to support the overall playful child-centeredness of conversational agent design.more » « less
-
We study two approaches for predicting an appropriate pose for a robot to take part in group formations typical of social human conversations subject to the physical layout of the surrounding environment. One method is model-based and explicitly encodes key geometric aspects of conversational formations. The other method is data-driven. It implicitly models key properties of spatial arrangements using graph neural networks and an adversarial training regimen. We evaluate the proposed approaches through quantitative metrics designed for this problem domain and via a human experiment. Our results suggest that the proposed methods are effective at reasoning about the environment layout and conversational group formations. They can also be used repeatedly to simulate conversational spatial arrangements despite being designed to output a single pose at a time. However, the methods showed different strengths. For example, the geometric approach was more successful at avoiding poses generated in nonfree areas of the environment, but the data-driven method was better at capturing the variability of conversational spatial formations. We discuss ways to address open challenges for the pose generation problem and other interesting avenues for future work.more » « less
-
Users often fail to formulate their complex information needs in a single query. As a consequence, they need to scan multiple result pages and/or reformulate their queries, which is a frustrating experience. Alternatively, systems can improve user satisfaction by proactively asking questions from the users to clarify their information needs. Asking clarifying questions is especially important in information-seeking conversational systems, since they can only return a limited number (often only one) of results. In this paper, we formulate the task of asking clarifying questions in open-domain information retrieval. We propose an offline evaluation methodology for the task. In this research, we create a dataset, called Qulac, through crowdsourcing. Our dataset is based on the TREC Web Track 2009-2012 data and consists of over 10K question-answer pairs for 198 TREC topics with 762 facets. Our experiments on an oracle model demonstrate that asking only one good question leads to over 100% retrieval performance improvement, which clearly demonstrates the potential impact of the task. We further propose a neural model for selecting clarifying question based on the original query and the previous question-answer interactions. Our model significantly outperforms competitive baselines. To foster research in this area, we have made Qulac publicly available.more » « less