Search for: All records

Search systems are often used to support learning-oriented goals. This trend has given rise to the “searchas- learning” movement, which proposes that search systems should be designed to support learning. To this end, an important research question is: How does a searcher’s type of learning objective (LO) influence their trajectory (or pathway) toward that objective? We report on a lab study (N = 36) in which participants gathered information to meet a specific type of LO. To characterize LOs and pathways, we leveraged Anderson and Krathwohl’s (A&K’s) taxonomy [3]. A&K’s taxonomy situates LOs at the intersection of two orthogonal dimensions: (1) cognitive process (CP) (remember, understand, apply, analyze, evaluate, and create) and (2) knowledge type (factual, conceptual, procedural, and metacognitive knowledge). Participants completed learning-oriented search tasks that varied along three CPs (apply, evaluate, and create) and three knowledge types (factual, conceptual, and procedural knowledge). A pathway is defined as a sequence of learning instances (e.g., subgoals) that were also each classified into cells from A&K’s taxonomy. Our study used a think-aloud protocol, and pathways were generated through a qualitative analysis of participants’ thinkaloud comments and recorded screen activities. We investigate three research questions. First, in RQ1, we study the impact of the LO on pathway characteristics (e.g., pathway length). Second, in RQ2, we study the impact of the LO on the types of A&K cells traversed along the pathway. Third, in RQ3, we study common and uncommon transitions between A&K cells along pathways conditioned on the knowledge type of the objective. We discuss implications of our results for designing search systems to support learning. 

People often search for information in order to learn something new. In recent years, the “search-as-learning” movement has argued that search systems should be better designed to support learning. Current search systems (especially Web search engines) are largely designed and optimized to fulfill simple look-up tasks (e.g., navigational or fact-finding search tasks). However, they provide less support for searchers working on complex tasks that involve learning. Search-as-learning studies have investigated a wide range of research questions. For example, studies have aimed to better understand how characteristics of the individual searcher, the type of search task, and interactive features provided by the system can influence learning outcomes. Learning assessment is a key component in search-as-learning studies. Assessment materials are used to both gauge prior knowledge and measure learning during or after one or more search sessions. In this paper, we provide a systematic review of different types of assessments used in search-as-learning studies to date. The paper makes the following three contributions. First, we review different types of assessments used and discuss their potential benefits and drawbacks. Second, we review assessments used outside of search-as-learning, which may provide insights and opportunities for future research. Third, we provide recommendations for future research. Importantly, we argue that future studies should clearly define learning objectives and develop assessment materials that reliably capture the intended type of learning. For example, assessment materials should test a participant’s ability to engage with specific cognitive processes, which may range from simple (e.g., memorization) to more complex (e.g., critical and creative thinking). Additionally, we argue that future studies should consider two dimensions that are understudied in search-as-learning: long-term retention (i.e., being able to use what was learned in the long term) and transfer of learning (i.e., being able to use what was learned in a novel context). 

In recent years, the “search as learning” community has argued that search systems should be designed to support learning. We report on a lab study in which we manipulated the learning objectives associated with search tasks assigned to participants. We manipulated learning objectives by leveraging Anderson and Krathwohl’s taxonomy of learning (A&K’s taxonomy) [2], which situates learning objectives at the intersection of two orthogonal dimensions: the cognitive process and the knowledge type dimension. Participants in our study completed tasks with learning objectives that varied across three cognitive processes (apply, evaluate, and create) and three knowledge types (factual, conceptual, and procedural knowledge). We focus on the effects of the task’s cognitive process and knowledge type on participants’ pre-/post-task perceptions and search behaviors. Our results found that the three knowledge types considered in our study had a greater effect than the three cognitive processes. Specifically, conceptual knowledge tasks were perceived to be more difficult and required more search activity. We discuss implications for designing search systems that support learning. 

In this paper, we present results from an exploratory study to investigate users’ behaviors and preferences for three different styles of search results presentation in a virtual reality (VR) head-mounted display (HMD). Prior work in 2D displays has suggested possible benefits of presenting information in ways that exploit users’ spatial cognition abilities. We designed a VR system that displays search results in three different spatial arrangements: a list of 8 results, a 4x5 grid, and a 2x10 arc. These spatial display conditions were designed to differ in terms of the number of results displayed per page (8 vs 20) and the amount of head movement required to scan the results (list < grid < arc). Thirty-six participants completed 6 search trials in each display condition (18 total). For each trial, the participant was presented with a display of search results and asked to find a given target result or to indicate that the target was not present. We collected data about users’ behaviors with and perceptions about the three display conditions using interaction data, questionnaires, and interviews. We explore the effects of display condition and target presence on behavioral measures (e.g., completion time, head movement, paging events, accuracy) and on users’ perceptions (e.g., workload, ease of use, comfort, confidence, difficulty, and lostness). Our results suggest that there was no difference in accuracy among the display conditions, but that users completed tasks more quickly using the arc. However, users also expressed lower preferences for the arc, instead preferring the list and grid displays. Our findings extend prior research on visual search into to the area of 3-dimensional result displays for interactive information retrieval in VR HMD environments. 

An important question in interactive information retrieval (IIR) is: How do task characteristics influence users’ needs? In this paper, we investigate the effects of cognitive task complexity on the types of information considered useful for a task. We characterize information types from two perspectives. From one perspective, we classify task-related information items based on inherent characteristics (referred to as info-types): factual statements, concepts/definitions, opinionated statements, and insights—tips/advice related to the task domain. From a second perspective, we used Byström and Järvelin’s framework [5] to define information types based on how the information might be used to complete the task (referred to as functional roles): (1) to help the task doer understand the task requirements (problem information); (2) to help the task doer strategize on how to approach the task (problem-solving information); and (3) to help the task doer learn about the task domain (domain information). Our results suggest that: (1) cognitive task complexity influences the functional roles of information items deemed useful for the task (RQ1); (2) certain info-types are more (or less) likely to play certain functional roles (RQ2); and task complexity influences the variety of functional roles played by info-types (RQ3). 

There is a growing body of research in the Search as Learning community that recognizes the need for users to learn during search, but modern search systems have yet to adapt to support this need. Our research proposes three research goals toward addressing the support of user learning during search. Research goal 1 (RG1) introduces a more precise and reliable metric of assessing user learning. Anderson & Krathwohl’s 2-dimensional taxonomy is used as a framework to develop learning objectives and assessment questions to measure user learning during search. Additionally, Anderson & Krathwohl’s taxonomy is used as a coding scheme to outline the pathways users traverse along the way to a particular learning objective. Research goal 2 (RG2) investigates the prediction of learning objectives using behavioral measures. Finally, research goal 3 (RG3) proposes a search system that presents information relevant to the user based on their current learning sub-goal and scaffolds information based on the pathways they are likely to traverse given a particular learning objective. 

Researchers in interactive information retrieval (IIR) have studied and refined 2D presentations of search results for years. Recent advances are bringing augmented reality (AR) and virtual reality (VR) to real-world systems, though the IIR community has done relatively little work to explore and understand aspects of 3D presentations of search results, effects of immersive environments, and the impacts of spatial cognition and different spatial arrangements of results displays in 3D. In the research proposed here, I outline my plan to use immerse environments to investigate how users’ spatial cognition may influence the information retrieval process. Specifically, this work will observe how spatial arrangements of search results affect users’ ability to find information in the postquery, visual search phase of the IIR process across quantitative and qualitative measures. 

Search tasks play an important role in the study and development of interactive information retrieval (IIR) systems. Prior work has examined how search tasks vary along dimensions such as the task’s main activity, end goal, structure, and complexity. Recently, researchers have been exploring task complexity from the perspective of cognitive complexity—related to the types (and variety) of mental activities required by the task. Anderson & Krathwohl’s two-dimensional taxonomy of learning has been a commonly used framework for investigating tasks from the perspective of cognitive complexity [1]. A&K’s 2D taxonomy involves a cognitive process dimension and an orthogonal knowledge dimension. Prior IIR research has successfully leveraged the cognitive process dimension of this 2D taxonomy to develop search tasks and investigate their effects on searchers’ needs, perceptions, and behaviors. However, the knowledge dimension of the taxonomy has been largely ignored. In this conceptual paper, we argue that future IIR research should consider both dimensions of A&K’s taxonomy. Specifically, we discuss related work, present details on both dimensions of A&K’s taxonomy, and explain how to use the taxonomy to develop search tasks and learning assessment materials. Additionally, we discuss how considering both dimensions of A&K’s taxonomy has important implications for future IIR research. 

A search trail is an interactive visualization of how a previous searcher approached a related task. Using search trails to assist users requires understanding aspects of the task, user, and trails. In this paper, we examine two questions. First, what are task characteristics that influence a user's ability to gain benefits from others' trails? Second, what is the impact of a "mismatch" between a current user's task and previous user's task which originated the trail? We report on a study that investigated the influence of two factors on participants' perceptions and behaviors while using search trails to complete tasks. Our first factor, task scope, focused on the scope of the task assigned to the participant (broad to narrow). Our manipulation of this factor involved varying the number of constraints associated with tasks. Our second factor, trail scope, focused on the scope of the task that originated the search trails given to participants. We investigated how task scope and trail scope affected participants' (RQ1) pre-task perceptions, (RQ2) post-task perceptions, and (RQ3) search behaviors. We discuss implications of our results for systems that use search trails to provide assistance.