Given the more widespread nature of natural language interfaces, it is increasingly important to understand who are accessing those interfaces, and how those interfaces are being used. In this paper, we explore spellchecking in the context of web search with children as the target audience. In particular, via a literature review we show that, while widely used, popular search tools are ill-designed for children. We then use spellcheckers as a case study to highlight the need for an interdisciplinary approach that brings together natural language processing, education, human-computer interaction to address a known information retrieval problem: query misspelling. We conclude that it is imperative that those for whom the interfaces are designed have a voice in the design process.
more »
« less
Children and search tools: Evaluation remains unclear
As children search the internet for materials, they o en turn to search engines that, unfortunately, o er children li le support as they formulate queries to initiate the search process or examine resources for relevance. While some solutions have been proposed to address this, inherent to this issue is the need to evaluate the e ectiveness of these solutions. We posit that the evaluation of the diverse aspects involved in the search process – from query suggestion generation to resource retrieval – requires a complex, multi-faceted approach that draws on evaluation methods utilized in human-computer interaction, information retrieval, natural language processing, education, and psychology.
more »
« less
- Award ID(s):
- 1763649
- NSF-PAR ID:
- 10099389
- Date Published:
- Journal Name:
- KidRec Workshop co-located with ACM IDC 2019
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
To find resources on the internet, children frequently interact with search engines (SE). In doing so, they are generally unaware that certain characteristics of search queries and resources are used to prioritize results. The ability to successfully craft a search query and explore retrieved results in the quest for relevant ones is more or less assumed for adults but can be uniquely challenging for children. While many characteristics are factored into the design of filtering and ranking algorithms employed by SE, a lack of transparency in the displayed information regarding those characteristics leveraged during this prioritization process of resources inhibits children from having direct access to information about displayed results that may provide valuable insight and aid in the search process and development of search literacy. We posit that simple visual cues can be provided to reveal those characteristics, thus increasing the overall transparency of the information displayed and encouraging children to make more effective use of the material presented to them.more » « less
-
People often have difficulty in expressing their information needs. Many times this results from a lack of clarity about the task at hand, or the way an information or search system works. In addition, people may not know what they do not know. The former is addressed by search systems by providing recommendations, whereas there are no good solutions for the latter problem. Even when a search system makes recommendations, they are limited to suggesting objects such as queries and documents only. They do not consider providing suggestions for strategies, people, or processes. This Perspective Paper addresses it by showing how to investigate the nature of the work a person is doing, predicting the potential problems they may encounter, and providing help to overcome those problems. This help could be an object such as a document or a query, a strategy, or a person. This whole process is referred to as Information Fostering. Beyond crafting a general-purpose recommender system, Information Fostering is the idea of providing proactive suggestions and help to information seekers. This could allow them avoid potential problems and capture promising opportunities from a search process before it is too late. The current paper presents this new perspective by outlining desired characteristics of an Information Fostering system, envisioning application scenarios, and proposing a set of potential methods for moving forward. Beyond these details, the primary purpose of this paper is to offer a new viewpoint that looks at the other side of the information seeking coin, by bringing together ideas from human-computer interaction, information retrieval, recommender systems, and education.more » « less
-
more » « less
Abstract Evaluation metrics such as precision, recall and normalized discounted cumulative gain have been widely applied in
ad hoc retrieval experiments. They have facilitated the assessment of system performance in various topics over the past decade. However, the effectiveness of such metrics in capturing users’ in-situ search experience, especially in complex search tasks that trigger interactive search sessions, is limited. To address this challenge, it is necessary to adaptively adjust the evaluation strategies of search systems to better respond to users’ changing information needs and evaluation criteria. In this work, we adopt a taxonomy of search task states that a user goes through in different scenarios and moments of search sessions, and perform a meta-evaluation of existing metrics to better understand their effectiveness in measuring user satisfaction. We then built models for predicting task states behind queries based on in-session signals. Furthermore, we constructed and meta-evaluated new state-aware evaluation metrics. Our analysis and experimental evaluation are performed on two datasets collected from a field study and a laboratory study, respectively. Results demonstrate that the effectiveness of individual evaluation metrics varies across task states. Meanwhile, task states can be detected from in-session signals. Our new state-aware evaluation metrics could better reflect in-situ user satisfaction than an extensive list of the widely used measures we analyzed in this work in certain states. Findings of our research can inspire the design and meta-evaluation of user-centered adaptive evaluation metrics, and also shed light on the development of state-aware interactive search systems. -
Children use popular web search tools, which are generally designed for adult users. Because children have different developmental needs than adults, these tools may not always adequately support their search for information. Moreover, even though search tools offer support to help in query formulation, these too are aimed at adults and may hinder children rather than help them. This calls for the examination of existing technologies in this area, to better understand what remains to be done when it comes to facilitating query-formulation tasks for young users. In this paper, we investigate interaction elements of query formulation--including query suggestion algorithms--for children. The primary goals of our research efforts are to: (i) examine existing plug-ins and interfaces that explicitly aid children's query formulation; (ii) investigate children's interactions with suggestions offered by a general-purpose query suggestion strategy vs. a counterpart designed with children in mind; and (iii) identify, via participatory design sessions, their preferences when it comes to tools / strategies that can help children find information and guide them through the query formulation process. Our analysis shows that existing tools do not meet children's needs and expectations; the outcomes of our work can guide researchers and developers as they implement query formulation strategies for children.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
