An official website of the United States government
Abstract Converging evidence has demonstrated that humans exhibit two distinct strategies when learning in complex environments. One is model-free learning, i.e., simple reinforcement of rewarded actions, and the other is model-based learning, which considers the structure of the environment. Recent work has argued that people exhibit little model-based behavior unless it leads to higher rewards. Here we use mouse tracking to study model-based learning in stochastic and deterministic (pattern-based) environments of varying difficulty. In both tasks participants’ mouse movements reveal that they learned the structures of their environments, despite the fact that standard behavior-based estimates suggested no such learning in the stochastic task. Thus, we argue that mouse tracking can reveal whether subjects have structure knowledge, which is necessary but not sufficient for model-based choice.
more »
« less
Mouse tracking performance: A new approach to analyzing continuous mouse tracking data
Abstract Mouse tracking is an important source of data in cognitive science. Most contemporary mouse tracking studies use binary-choice tasks and analyze the curvature or velocity of an individual mouse movement during an experimental trial as participants select from one of the two options. However, there are many types of mouse tracking data available beyond what is produced in a binary-choice task, including naturalistic data from web users. In order to utilize these data, cognitive scientists need tools that are robust to the lack of trial-by-trial structure in most normal computer tasks. We use singular value decomposition (SVD) and detrended fluctuation analysis (DFA) to analyze whole time series of unstructured mouse movement data. We also introduce a new technique for describing two-dimensional mouse traces as complex-valued time series, which allows SVD and DFA to be applied in a straightforward way without losing important spatial information. We find that there is useful information at the level of whole time series, and we use this information to predict performance in an online task. We also discuss how the implications of these results can advance the use of mouse tracking research in cognitive science.
more »
« less
- Award ID(s):
- 1840265
- PAR ID:
- 10505334
- Publisher / Repository:
- Springer
- Date Published:
- Journal Name:
- Behavior Research Methods
- ISSN:
- 1554-3528
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Human decision making behavior is observed with choice-response time data during psychological experiments. Drift-diffusion models of this data consist of a Wiener first-passage time (WFPT) distribution and are described by cognitive parameters: drift rate, boundary separation, and starting point. These estimated parameters are of interest to neuroscientists as they can be mapped to features of cognitive processes of decision making (such as speed, caution, and bias) and related to brain activity. The observed patterns of RT also reflect the variability of cognitive processes from trial to trial mediated by neural dynamics. We adapted a SincNet-based shallow neural network architecture to fit the Drift-Diffusion model using EEG signals on every experimental trial. The model consists of a SincNet layer, a depthwise spatial convolution layer, and two separate fully connected layers that predict drift rate and boundary for each trial in-parallel. The SincNet layer parametrized the kernels in order to directly learn the low and high cutoff frequencies of bandpass filters that are applied to the EEG data to predict drift and boundary parameters. During training, model parameters were updated by minimizing the negative log likelihood function of WFPT distribution given trial RT. We developed separate decision SincNet models for each participant performing a two-alternative forced-choice task by discriminating whether a Gabor patch presented with noise is high or low spatial frequency. Our results showed that single-trial estimates of drift and boundary performed better at predicting RTs than the median estimates in both training and test data sets, suggesting that our model can successfully use EEG features to estimate meaningful single-trial Diffusion model parameters. Furthermore the shallow SincNet architecture identified time windows of information processing related to evidence accumulation and caution and the EEG frequency bands that reflect these processes within each participant.more » « less
-
Human decision making behavior is observed with choice-response time data during psychological experiments. Drift-diffusion models of this data consist of a Wiener first-passage time (WFPT) distribution and are described by cognitive parameters: drift rate, boundary separation, and starting point. These estimated parameters are of interest to neuroscientists as they can be mapped to features of cognitive processes of decision making (such as speed, caution, and bias) and related to brain activity. The observed patterns of RT also reflect the variability of cognitive processes from trial to trial mediated by neural dynamics. We adapted a SincNet-based shallow neural network architecture to fit the Drift-Diffusion model using EEG signals on every experimental trial. The model consists of a SincNet layer, a depthwise spatial convolution layer, and two separate FC layers that predict drift rate and boundary for each trial in-parallel. The SincNet layer parametrized the kernels in order to directly learn the low and high cutoff frequencies of bandpass filters that are applied to the EEG data to predict drift and boundary parameters. During training, model parameters were updated by minimizing the negative log likelihood function of WFPT distribution given trial RT. We developed separate decision SincNet models for each participant performing a two-alternative forced-choice task. Our results showed that single-trial estimates of drift and boundary performed better at predicting RTs than the median estimates in both training and test data sets, suggesting that our model can successfully use EEG features to estimate meaningful single-trial Diffusion model parameters. Furthermore, the shallow SincNet architecture identified time windows of information processing related to evidence accumulation and caution and the EEG frequency bands that reflect these processes within each participant.more » « less
-
Abstract Retrospective judgments require decision-makers to gather information over time and integrate that information into a summary statistic like the average. Many retrospective judgments require putting equal weight on early and late information, in contrast to prospective judgments that involve predicting the future and so rely more on late information. We investigate how people weight information over time when continuously reporting the average stimulus strength in a sequence of displays. We investigate the consistency of these temporal profiles across perceptual and value-based tasks using both behavior and functional magnetic resonance imaging (fMRI) data. We found that people display remarkably consistent temporal weighting functions across choice domains, with a generally strong recency bias and modest primacy bias. The fMRI data revealed evidence-tracking activity in the cuneus in both tasks and in the left dorsolateral prefrontal cortex in the value-based task. Finally, a network of cognitive control regions is more active for people who exhibit a stronger primacy vs. recency bias. Together, our behavioral findings indicate that people consistently overweight recency when evaluating past information, and the neural data suggest that overcoming this tendency may require cognitive control.more » « less
-
null (Ed.)Recent research in empirical software engineering is applying techniques from neurocognitive science and breaking new grounds in the ways that researchers can model and analyze the cognitive processes of developers as they interact with software artifacts. However, given the novelty of this line of research, only one tool exists to help researchers represent and analyze this kind of multi-modal biometric data. While this tool does help with visualizing temporal eyetracking and physiological data, it does not allow for the mapping of physiological data to source code elements, instead projecting information over images of code. One drawback of this is that researchers are still unable to meaningfully combine and map physiological and eye tracking data to source code artifacts. The use of images also bars the support of long or multiple code files, which prevents researchers from analyzing data from experiments conducted in realistic settings. To address these drawbacks, we propose VITALSE, a tool for the interactive visualization of combined multi-modal biometric data for software engineering tasks. VITALSE provides interactive and customizable temporal heatmaps created with synchronized eyetracking and biometric data. The tool supports analysis on multiple files, user defined annotations for points of interest over source code elements, and high level customizable metric summaries for the provided dataset. VITALSE, a video demonstration, and sample data to demonstrate its capabilities can be found at http://www.vitalse.app.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3758/s13428-023-02210-5
