skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Computerized Block Games for Automated Cognitive Assessment: Development and Evaluation Study
Background Cognitive assessment using tangible objects can measure fine motor and hand-eye coordination skills along with other cognitive domains. Administering such tests is often expensive, labor-intensive, and error prone owing to manual recording and potential subjectivity. Automating the administration and scoring processes can address these difficulties while reducing time and cost. e-Cube is a new vision-based, computerized cognitive assessment tool that integrates computational measures of play complexity and item generators to enable automated and adaptive testing. The e-Cube games use a set of cubes, and the system tracks the movements and locations of these cubes as manipulated by the player. Objective The primary objectives of the study were to validate the play complexity measures that form the basis of developing the adaptive assessment system and evaluate the preliminary utility and usability of the e-Cube system as an automated cognitive assessment tool. Methods This study used 6 e-Cube games, namely, Assembly, Shape-Matching, Sequence-Memory, Spatial-Memory, Path-Tracking, and Maze, each targeting different cognitive domains. In total, 2 versions of the games, the fixed version with predetermined sets of items and the adaptive version using the autonomous item generators, were prepared for comparative evaluation. Enrolled participants (N=80; aged 18-60 years) were divided into 2 groups: 48% (38/80) of the participants in the fixed group and 52% (42/80) in the adaptive group. Each was administered the 6 e-Cube games; 3 subtests of the Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV; Block Design, Digit Span, and Matrix Reasoning); and the System Usability Scale (SUS). Statistical analyses at the 95% significance level were applied. Results The play complexity values were correlated with the performance indicators (ie, correctness and completion time). The adaptive e-Cube games were correlated with the WAIS-IV subtests (r=0.49, 95% CI 0.21-0.70; P<.001 for Assembly and Block Design; r=0.34, 95% CI 0.03-0.59; P=.03 for Shape-Matching and Matrix Reasoning; r=0.51, 95% CI 0.24-0.72; P<.001 for Spatial-Memory and Digit Span; r=0.45, 95% CI 0.16-0.67; P=.003 for Path-Tracking and Block Design; and r=0.45, 95% CI 0.16-0.67; P=.003 for Path-Tracking and Matrix Reasoning). The fixed version showed weaker correlations with the WAIS-IV subtests. The e-Cube system showed a low false detection rate (6/5990, 0.1%) and was determined to be usable, with an average SUS score of 86.01 (SD 8.75). Conclusions The correlations between the play complexity values and performance indicators supported the validity of the play complexity measures. Correlations between the adaptive e-Cube games and the WAIS-IV subtests demonstrated the potential utility of the e-Cube games for cognitive assessment, but a further validation study is needed to confirm this. The low false detection rate and high SUS scores indicated that e-Cube is technically reliable and usable.  more » « less
Award ID(s):
2002721
PAR ID:
10441128
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
JMIR Serious Games
Volume:
11
ISSN:
2291-9279
Page Range / eLocation ID:
e40931
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Although electronic cigarette (e-cigarette) aerosol contains similar toxicants to combustible cigarettes, few studies have examined their influence on fecundability. We assessed the association between e-cigarette use and fecundability, overall and according to combustible cigarette smoking history, in a cohort of 4,586 North American women (aged 21–45 years) enrolled during 2017–2020 in Pregnancy Study Online, a Web-based prospective preconception study. Women reported current and former e-cigarette use on baseline and follow-up questionnaires, and they completed bimonthly follow-up questionnaires until self-reported pregnancy or censoring. Fecundability ratios and 95% confidence intervals were calculated using proportional probabilities models, controlling for potential confounders. Overall, 17% of women had ever used e-cigarettes and 4% were current users. Compared with never use of e-cigarettes, current e-cigarette use was associated with slightly lower fecundability (fecundability ratio = 0.84, 95% confidence interval (CI): 0.67, 1.06). Compared with current nonusers of e-cigarettes and combustible cigarettes, fecundability ratios were 0.83 (95% CI: 0.54, 1.29) for current dual users of e-cigarettes and combustible cigarettes, 0.91 (95% CI: 0.70, 1.18) for current e-cigarette users who were nonsmokers of combustible cigarettes, and 1.01 (95% CI: 0.85, 1.20) for nonusers of e-cigarettes who were current smokers of combustible cigarettes. Current e-cigarette use was associated with slightly reduced fecundability, but estimates of its independent and joint associations with combustible cigarette smoking were inconsistent and imprecise. 
    more » « less
  2. Objective Math and reading are related at the disability level and along the continuum of skill (Cirino, 2022). Cognitive correlates of math and reading in children are well-known separately, with a recent focus on the reason for their overlap. However, less is known about these issues in community college (CC) students despite more than half of post-secondary education occurring at this level. Here we assess cognitive predictors of math and reading (language, working memory, processing speed, nonverbal reasoning, attention) in CC students, outcome overlap, and the extent that predictors account for overlap. We expect all predictors to relate to achievement, with language and working memory as the strongest predictors, and accounting for the most overlap. We also expect more overlap and stronger prediction for complex outcomes (reading comprehension and math applications) relative to foundational skills (word reading and computations). Participants and Methods Participants were 94 CC students enrolled in their first math class. Approximately half the students were taking developmental coursework. Participants were administered four KTEA-3 measures: Letter-Word Reading, Reading Comprehension, Math Computation, and Math Concepts and Application. Language consisted of Vocabulary (K-BIT-2), and Elision and Rapid Naming subtests of the CTOPP-2. Working memory was assessed with two complex span measures (Symmetry Span and Reading Span). Processing speed was measured with the WAIS-IV, and nonverbal reasoning with the K-BIT-2. Attention was assessed via a researcher-designed continuous performance task and a self-rating scale. Multiple regression assessed cognitive prediction for each achievement measure; and partial correlation evaluated overlap. Results For computations, all predictors accounted for R2=53% variance; nonverbal reasoning and elision were unique predictors (p<.05). For math applications, R2=58%, with unique prediction for nonverbal reasoning, vocabulary, elision, and symmetry span. For word reading, R2=50%, with unique prediction for vocabulary, elision, and reading span. Finally, for reading comprehension, R2=47%, with unique prediction for vocabulary and nonverbal reasoning. Regarding overlap, computations and word reading correlated r=.50, and math applications and reading comprehension r=.57, which is higher than a recent meta-analysis (Unal et al., 2023). Language was the strongest contributor of overlap; these variables reduced the correlation for foundational achievement by 50%, and for complex achievement, by 32%. Other domains accounted for little overlap, despite significant zero-order correlations. Substantive results were generally similar when covariates were considered. Conclusions Individual prediction was dominated by language, nonverbal reasoning, and working memory variables. Math and reading performances were strongly related, and language was the strongest predictor of this overlap, which is only partially consistent with extant literature but adds context and generalization for CC students. Attention and processing speed were only weakly related to performance, which may reflect the overlearned nature of these skills at this level. Future work might need to include more malleable factors (e.g., motivation), as well as broader views of achievement (e.g., course grades). 
    more » « less
  3. null (Ed.)
    In this paper, we present the Visuospatial Reasoning Environment for Experimentation (VREE). VREE provides a simulated environment where intelligent agents interact with virtual objects while solving different visuospatial reasoning tasks. This paper shows how VREE is valuable for studying the sufficiency of visual imagery approaches for a large number of visuospatial reasoning tasks as well as how diverse strategies can be represented and studied within a single task. We present results from computational experiments using VREE on the block design task and on numerous subtests from the Leiter-R test battery on nonverbal intelligence. 
    more » « less
  4. ImportanceAssessing nontechnical skills in operating rooms (ORs) is crucial for enhancing surgical performance and patient safety. However, automated and real-time evaluation of these skills remains challenging. ObjectiveTo explore the feasibility of using motion features extracted from surgical video recordings to automatically assess nontechnical skills during cardiac surgical procedures. Design, Setting, and ParticipantsThis cross-sectional study used video recordings of cardiac surgical procedures at a tertiary academic US hospital collected from January 2021 through May 2022. The OpenPose library was used to analyze videos to extract body pose estimations of team members and compute various team motion features. The Non-Technical Skills for Surgeons (NOTSS) assessment tool was employed for rating the OR team’s nontechnical skills by 3 expert raters. Main Outcomes and MeasuresNOTSS overall score, with motion features extracted from surgical videos as measures. ResultsA total of 30 complete cardiac surgery procedures were included: 26 (86.6%) were on-pump coronary artery bypass graft procedures and 4 (13.4%) were aortic valve replacement or repair procedures. All patients were male, and the mean (SD) age was 72 (6.3) years. All surgical teams were composed of 4 key roles (attending surgeon, attending anesthesiologist, primary perfusionist, and scrub nurse) with additional supporting roles. NOTSS scores correlated significantly with trajectory (r = 0.51,P = .005), acceleration (r = 0.48,P = .008), and entropy (r = −0.52,P = .004) of team displacement. Multiple linear regression, adjusted for patient factors, showed average team trajectory (adjustedR2 = 0.335; coefficient, 10.51 [95% CI, 8.81-12.21];P = .004) and team displacement entropy (adjustedR2 = 0.304; coefficient, −12.64 [95% CI, −20.54 to −4.74];P = .003) were associated with NOTSS scores. Conclusions and RelevanceThis study suggests a significant link between OR team movements and nontechnical skills ratings by NOTSS during cardiac surgical procedures, suggesting automated surgical video analysis could enhance nontechnical skills assessment. Further investigation across different hospitals and specialties is necessary to validate these findings. 
    more » « less
  5. Magnetic modular cubes are cube-shaped bodies with embedded permanent magnets. The cubes are uniformly controlled by a global time-varying magnetic field.A 2D physics simulator is used to simulate global control and the resulting continuous movement of magnetic modular cube structures. We develop local plans, closed-loop control algorithms for planning the connection of two structures at desired faces. The global planner generates a building instruction graph for a target structure that we traverse in a depth-first-search approach by repeatedly applying local plans.We analyze how structure size and shape affect planning time. The planner solves 80% of the randomly created instances with up to 12 cubes in an average time of about 200 seconds. 
    more » « less