More Like this

1. Exploring latent states of problem‐solving competence using hidden Markov model on process data

   https://doi.org/10.1111/jcal.12559  

   Xiao, Yue; He, Qiwei; Veldkamp, Bernard; Liu, Hongyun (May 2021, Journal of Computer Assisted Learning)

   Abstract The response process of problem‐solving items contains rich information about respondents' behaviours and cognitive process in the digital tasks, while the information extraction is a big challenge. The aim of the study is to use a data‐driven approach to explore the latent states and state transitions underlying problem‐solving process to reflect test‐takers' behavioural patterns, and to investigate how these states and state transitions could be associated with test‐takers' performance. We employed the Hidden Markov Modelling approach to identify test takers' hidden states during the problem‐solving process and compared the frequency of states and/or state transitions between different performance groups. We conducted comparable studies in two problem‐solving items with a focus on the US sample that was collected in PIAAC 2012, and examined the correlation between those frequencies from two items. Latent states and transitions between them underlying the problem‐solving process were identified and found significantly different by performance groups. The groups with correct responses in both items were found more engaged in tasks and more often to use efficient tools to solve problems, while the group with incorrect responses was found more likely to use shorter action sequences and exhibit hesitative behaviours. Consistent behavioural patterns were identified across items. This study demonstrates the value of data‐driven based HMM approach to better understand respondents' behavioural patterns and cognitive transmissions underneath the observable action sequences in complex problem‐solving tasks. 

   more » « less
2. Phenotyping Cognitive Impairment using Graphomotor and Latency Features in Digital Clock Drawing Test*

   https://doi.org/10.1109/EMBC44109.2020.9176469  

   Davoudi, Anis; Dion, Catherine; Amini, Shawna; Libon, David J.; Tighe, Patrick J.; Price, Catherine C.; Rashidi, Parisa (July 2020, 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC))

   null (Ed.)

   The Clock Drawing Test, where the participant is asked to draw a clock from memory and copy a model clock, is widely used for screening of cognitive impairment. The digital version of the clock test, the digital clock drawing test (dCDT), employs accelerometer and pressure sensors of a digital pen to capture time and pressure information from a participant's performance in a granular digital format. While visual features of the clock drawing test have previously been studied, little is known about the relationship between demographic and cognitive impairment characteristics with dCDT latency and graphomotor features. Here, we examine dCDT feature clusters with respect to sociodemographic and cognitive impairment outcomes. Our results show that the clusters are not significantly different in terms of age and gender, but did significantly differ in terms of education, Mini-Mental State Exam scores, and cognitive impairment diagnoses.This study shows that features extracted from digital clock drawings can provide important information regarding cognitive reserve and cognitive impairments. 

   more » « less
3. Combining Clickstream Analyses and Graph-Modeled Data Clustering for Identifying Common Response Processes

   https://doi.org/10.1007/s11336-020-09743-0  

   Ulitzsch, Esther; He, Qiwei; Ulitzsch, Vincent; Molter, Hendrik; Nichterlein, André; Niedermeier, Rolf; Pohl, Steffi (March 2021, Psychometrika)

   Abstract Complex interactive test items are becoming more widely used in assessments. Being computer-administered, assessments using interactive items allow logging time-stamped action sequences. These sequences pose a rich source of information that may facilitate investigating how examinees approach an item and arrive at their given response. There is a rich body of research leveraging action sequence data for investigating examinees’ behavior. However, the associated timing data have been considered mainly on the item-level, if at all. Considering timing data on the action-level in addition to action sequences, however, has vast potential to support a more fine-grained assessment of examinees’ behavior. We provide an approach that jointly considers action sequences and action-level times for identifying common response processes. In doing so, we integrate tools from clickstream analyses and graph-modeled data clustering with psychometrics. In our approach, we (a) provide similarity measures that are based on both actions and the associated action-level timing data and (b) subsequently employ cluster edge deletion for identifying homogeneous, interpretable, well-separated groups of action patterns, each describing a common response process. Guidelines on how to apply the approach are provided. The approach and its utility are illustrated on a complex problem-solving item from PIAAC 2012. 

   more » « less
4. An exploratory analysis of the latent structure of process data via action sequence autoencoders

   https://doi.org/10.1111/bmsp.12203  

   Tang, Xueying; Wang, Zhi; Liu, Jingchen; Ying, Zhiliang (May 2020, British Journal of Mathematical and Statistical Psychology)

   Computer simulations have become a popular tool for assessing complex skills such as problem‐solving. Log files of computer‐based items record the human–computer interactive processes for each respondent in full. The response processes are very diverse, noisy, and of non‐standard formats. Few generic methods have been developed to exploit the information contained in process data. In this paper we propose a method to extract latent variables from process data. The method utilizes a sequence‐to‐sequence autoencoder to compress response processes into standard numerical vectors. It does not require prior knowledge of the specific items and human–computer interaction patterns. The proposed method is applied to both simulated and real process data to demonstrate that the resulting latent variables extract useful information from the response processes. 

   more » « less
5. A Qualitative Analysis of the Effects of Task Complexity on the Functional Role of Information

   https://doi.org/10.1145/3343413.3377992  

   Choi, Bogeum; Arguello, Jaime (March 2020, Proceedings of the 2020 Conference on Human Information Interaction and Retrieval)

   An important question in interactive information retrieval (IIR) is: How do task characteristics influence users’ needs? In this paper, we investigate the effects of cognitive task complexity on the types of information considered useful for a task. We characterize information types from two perspectives. From one perspective, we classify task-related information items based on inherent characteristics (referred to as info-types): factual statements, concepts/definitions, opinionated statements, and insights—tips/advice related to the task domain. From a second perspective, we used Byström and Järvelin’s framework [5] to define information types based on how the information might be used to complete the task (referred to as functional roles): (1) to help the task doer understand the task requirements (problem information); (2) to help the task doer strategize on how to approach the task (problem-solving information); and (3) to help the task doer learn about the task domain (domain information). Our results suggest that: (1) cognitive task complexity influences the functional roles of information items deemed useful for the task (RQ1); (2) certain info-types are more (or less) likely to play certain functional roles (RQ2); and task complexity influences the variety of functional roles played by info-types (RQ3). 

   more » « less