An official website of the United States government
In citizen science, participants’ productivity is imperative to project success. We investigate the feasibility of a collaborative approach to citizen science, within which productivity is enhanced by capitalizing on the diversity of individual attributes among participants. Specifically, we explore the possibility of enhancing productivity by integrating multiple individual attributes to inform the choice of which task should be assigned to which individual. To that end, we collect data in an online citizen science project composed of two task types: (i) filtering images of interest from an image repository in a limited time, and (ii) allocating tags on the object in the filtered images over unlimited time. The first task is assigned to those who have more experience in playing action video games, and the second task to those who have higher intrinsic motivation to participate. While each attribute has weak predictive power on the task performance, we demonstrate a greater increase in productivity when assigning participants to the task based on a combination of these attributes. We acknowledge that such an increase is modest compared to the case where participants are randomly assigned to the tasks, which could offset the effort of implementing our attribute-based task assignment scheme. This study constitutes a first step toward understanding and capitalizing on individual differences in attributes toward enhancing productivity in collaborative citizen science.
more »
« less
A joint analysis of dropout and learning functions in human decision-making with massive online data
The introduction of large-scale data sets in psychology allows for more robust accounts of various cognitive mechanisms, one of which is human learning. However, these data sets provide participants with complete autonomy over their own participation in the task, and therefore require precisely studying the factors influencing dropout alongside learning. In this work, we present such a data set where 1,234,844 participants play 10,874,547 games of a challenging variant of tic-tac-toe. We establish that there is a correlation between task performance and total experience, and independently analyze participants’ dropout behavior and learning trajectories. We find evidence for stopping patterns as a function of playing strength and investigate the processes underlying playing strength increases with experience using a set of metrics derived from a planning model. Finally, we develop a joint model to account for both dropout and learning functions which replicates our empirical findings.
more »
« less
- Award ID(s):
- 2008331
- PAR ID:
- 10377455
- Date Published:
- Journal Name:
- Proceedings of the 44th Annual Conference of the Cognitive Science Society
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract We consider user retention analytics for online freemium role-playing games (RPGs). RPGs constitute a very popular genre of computer-based games that, along with a player’s gaming actions, focus on the development of the player’s in-game virtual character through a persistent exploration of the gaming environment. Most RPGs follow the freemium business model in which the gamers can play for free but they are charged for premium add-on amenities. As with other freemium products, RPGs suffer from the curse of high dropout rates. This makes retention analysis extremely important for successful operation and survival of their gaming portals. Here, we develop a disciplined statistical framework for retention analysis by modelling multiple in-game player characteristics along with the dropout probabilities. We capture players’ motivations through engagement times, collaboration and achievement score at each level of the game, and jointly model them using a generalized linear mixed model (glmm) framework that further includes a time-to-event variable corresponding to churn. We capture the interdependencies in a player’s level-wise engagement, collaboration, achievement with dropout through a shared parameter model. We illustrate interesting changes in player behaviours as the gaming level progresses. The parameters in our joint model were estimated by a Hamiltonian Monte Carlo algorithm which incorporated a divide-and-recombine approach for increased scalability in glmm estimation that was needed to accommodate our large longitudinal gaming data-set. By incorporating the level-wise changes in a player’s motivations and using them for dropout rate prediction, our method greatly improves on state-of-the-art retention models. Based on data from a popular action based RPG, we demonstrate the competitive optimality of our proposed joint modelling approach by exhibiting its improved predictive performance over competitors. In particular, we outperform aggregate statistics based methods that ignore level-wise progressions as well as progression tracking non-joint model such as the Cox proportional hazards model. We also display improved predictions of popular marketing retention statistics and discuss how they can be used in managerial decision making.more » « less
-
Abstract This article describes a novel method for quantifying fixation disparity and evaluates its role in visuospatial cognition during an authentic learning task, specifically, the determination of molecule chirality in organic chemistry involving mental rotation and pattern comparison. The first study examined the influence of molecular model dimensionality (2D vs. 3D) on chirality determination performance and visual attention of 55 participants. The second study explored how the sustained playing of the tile‐matching game Mahjong, a pattern comparison game, can affect visual attention and visuospatial performance during the chirality determination task of 59 participants. Fixation disparity was one of the eye tracking variables explored. Both studies revealed that (1) individuals with higher fixation disparity underperformed on the chirality task, which involves mental rotation and pattern comparison, and (2) fixation disparity improved over time in participants who played Mahjong. This work provides important implications for using fixation disparity as a possible biomarker of visuospatial performance.more » « less
-
ABSTRACT Several studies suggest that children's learning and engagement with the content of play activities is affected by the ways parents and children interact. In particular, when parents are overly directive and set more goals during play with their children, their children tend to play less or are less engaged by subsequent challenges with the activity on their own. A concern, however, is that this directed interaction style is only compared with other styles of parent–child interaction, not with a baseline measure of engagement or learning. The present study incorporates such a baseline measure, comparing it with previously‐collected data on children's engagement and learning in a set of circuit‐building challenges. Regarding engagement, children were less engaged by the challenges when their parents were more directed during a free play setting (tested in Sobel et al. 2021) than when children had no prior experience playing with the circuit components. Regarding learning, children were better able to complete the circuit challenges and provided more causal explanations for how the completed challenges worked when they had experience playing with the circuit blocks with their parent. Overall, these data suggest that parent–child interaction during a STEM activity relates to both children's engagement and performance on challenges related to that activity.more » « less
-
This article introduces a model-based approach for training feedback controllers for an autonomous agent operating in a highly non-linear (albeit deterministic) environment. We desire the trained policy to ensure that the agent satisfies specific task objectives and safety constraints, both expressed in Discrete-Time Signal Temporal Logic (DT-STL). One advantage for reformulation of a task via formal frameworks, like DT-STL, is that it permits quantitative satisfaction semantics. In other words, given a trajectory and a DT-STL formula, we can compute therobustness, which can be interpreted as an approximate signed distance between the trajectory and the set of trajectories satisfying the formula. We utilize feedback control, and we assume a feed forward neural network for learning the feedback controller. We show how this learning problem is similar to training recurrent neural networks (RNNs), where the number of recurrent units is proportional to the temporal horizon of the agent’s task objectives. This poses a challenge: RNNs are susceptible to vanishing and exploding gradients, and naïve gradient descent-based strategies to solve long-horizon task objectives thus suffer from the same problems. To address this challenge, we introduce a novel gradient approximation algorithm based on the idea of dropout or gradient sampling. One of the main contributions is the notion ofcontroller network dropout, where we approximate the NN controller in several timesteps in the task horizon by the control input obtained using the controller in a previous training step. We show that our control synthesis methodology can be quite helpful for stochastic gradient descent to converge with less numerical issues, enabling scalable back-propagation over longer time horizons and trajectories over higher-dimensional state spaces. We demonstrate the efficacy of our approach on various motion planning applications requiring complex spatio-temporal and sequential tasks ranging over thousands of timesteps.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
