Search for: All records

ABSTRACT Automated scoring is a current hot topic in creativity research. However, most research has focused on the English language and popular verbal creative thinking tasks, such as the alternate uses task. Therefore, in this study, we present a large language model approach for automated scoring of a scientific creative thinking task that assesses divergent ideation in experimental tasks in the German language. Participants are required to generate alternative explanations for an empirical observation. This work analyzed a total of 13,423 unique responses. To predict human ratings of originality, we used XLM‐RoBERTa (Cross‐lingual Language Model‐RoBERTa), a large, multilingual model. The prediction model was trained on 9,400 responses. Results showed a strong correlation between model predictions and human ratings in a held‐out test set (n = 2,682;r = 0.80; CI‐95% [0.79, 0.81]). These promising findings underscore the potential of large language models for automated scoring of scientific creative thinking in the German language. We encourage researchers to further investigate automated scoring of other domain‐specific creative thinking tasks. 

Abstract Complex cognitive processes, like creative thinking, rely on interactions among multiple neurocognitive processes to generate effective and innovative behaviors on demand, for which the brain’s connector hubs play a crucial role. However, the unique contribution of specific hub sets to creative thinking is unknown. Employing three functional magnetic resonance imaging datasets (total N = 1,911), we demonstrate that connector hub sets are organized in a hierarchical manner based on diversity, with “control-default hubs”—which combine regions from the frontoparietal control and default mode networks—positioned at the apex. Specifically, control-default hubs exhibit the most diverse resting-state connectivity profiles and play the most substantial role in facilitating interactions between regions with dissimilar neurocognitive functions, a phenomenon we refer to as “diverse functional interaction”. Critically, we found that the involvement of control-default hubs in facilitating diverse functional interaction robustly relates to creativity, explaining both task-induced functional connectivity changes and individual creative performance. Our findings suggest that control-default hubs drive diverse functional interaction in the brain, enabling complex cognition, including creative thinking. We thus uncover a biologically plausible explanation that further elucidates the widely reported contributions of certain frontoparietal control and default mode network regions in creativity studies. 

Abstract The visual modality is central to both reception and expression of human creativity. Creativity assessment paradigms, such as structured drawing tasks Barbot (2018), seek to characterize this key modality of creative ideation. However, visual creativity assessment paradigms often rely on cohorts of expert or naïve raters to gauge the level of creativity of the outputs. This comes at the cost of substantial human investment in both time and labor. To address these issues, recent work has leveraged the power of machine learning techniques to automatically extract creativity scores in the verbal domain (e.g., SemDis; Beaty & Johnson 2021). Yet, a comparably well-vetted solution for the assessment of visual creativity is missing. Here, we introduce AuDrA – an Automated Drawing Assessment platform to extract visual creativity scores from simple drawing productions. Using a collection of line drawings and human creativity ratings, we trained AuDrA and tested its generalizability to untrained drawing sets, raters, and tasks. Across four datasets, nearly 60 raters, and over 13,000 drawings, we found AuDrA scores to be highly correlated with human creativity ratings for new drawings on the same drawing task (r= .65 to .81; mean = .76). Importantly, correlations between AuDrA scores and human raters surpassed those between drawings’ elaboration (i.e., ink on the page) and human creativity raters, suggesting that AuDrA is sensitive to features of drawings beyond simple degree of complexity. We discuss future directions, limitations, and link the trained AuDrA model and a tutorial (https://osf.io/kqn9v/) to enable researchers to efficiently assess new drawings. 

Standard learning assessments like multiple-choice questions measure what students know but not how their knowledge is organized. Recent advances in cognitive network science provide quantitative tools for modeling the structure of semantic memory, revealing key learning mechanisms. In two studies, we examined the semantic memory networks of undergraduate students enrolled in an introductory psychology course. In Study 1, we administered a cumulative multiple-choice test of psychology knowledge, the Intro Psych Test, at the end of the course. To estimate semantic memory networks, we administered two verbal fluency tasks: domain-specific fluency (naming psychology concepts) and domain-general fluency (naming animals). Based on their performance on the Intro Psych Test, we categorized students into a high-knowledge or low-knowledge group, and compared their semantic memory networks. Study 1 (N = 213) found that the high-knowledge group had semantic memory networks that were more clustered, with shorter distances between concepts—across both the domain-specific (psychology) and domain-general (animal) categories—compared to the low-knowledge group. In Study 2 (N = 145), we replicated and extended these findings in a longitudinal study, collecting data near the start and end of the semester. In addition to replicating Study 1, we found the semantic memory networks of high-knowledge students became more interconnected over time, across both domain-general and domain-specific categories. These findings suggest that successful learners show a distinct semantic memory organization—characterized by high connectivity and short path distances between concepts—highlighting the utility of cognitive network science for studying variation in student learning. 

Crystallized intelligence (Gc)—knowledge acquired through education and experience—supports creativity. Yet whether Gc contributes to creativity beyond providing access to more knowledge, remains unclear. We explore the role of a “flexible” semantic memory network structure as a potential shared mechanism of Gc and creativity. Across two studies (N = 506 and N = 161) participants completed Gc tests of vocabulary knowledge and were divided into low, medium, and high Gc groups. They also completed two alternate uses tasks, to assess verbal creativity, and a semantic fluency task, to estimate semantic memory networks. Across both studies, the semantic memory network structure of the high Gc group was more flexible—less structured, more clustered, and more interconnected—than that of the low Gc group. The high Gc group also outperformed the low Gc group on the creativity tasks. Our results suggest that flexible access to semantic memory supports both verbal intelligence and creativity. 

Metaphor is crucial in human cognition and creativity, facilitating abstract thinking, analogical reasoning, and idea generation. Typically, human raters manually score the originality of responses to creative thinking tasks – a laborious and error-prone process. Previous research sought to remedy these risks by scoring creativity tasks automatically using semantic distance and large language models (LLMs). Here, we extend research on automatic creativity scoring to metaphor generation – the ability to creatively describe episodes and concepts using nonliteral language. Metaphor is arguably more abstract and naturalistic than prior targets of automated creativity assessment. We collected 4,589 responses from 1,546 participants to various metaphor prompts and corresponding human creativity ratings. We fine-tuned two open-source LLMs (RoBERTa and GPT-2) – effectively “teaching” them to score metaphors like humans – before testing their ability to accurately assess the creativity of new metaphors. Results showed both models reliably predicted new human creativity ratings (RoBERTa r = .72, GPT-2 r = .70), significantly more strongly than semantic distance (r = .42). Importantly, the fine-tuned models generalized accurately to metaphor prompts they had not been trained on (RoBERTa r = .68, GPT-2 r = .63). We provide open access to the fine-tuned models, allowing researchers to assess metaphor creativity in a reproducible and timely manner. 

Increasing evidence suggests that specific memory systems (e.g., semantic vs. episodic) may support specific creative thought processes. However, there are a number of inconsistencies in the literature regarding the strength, direction, and influence of different memory (semantic, episodic, working, and short-term) and creativity (divergent and convergent thinking) types, as well as the influence of external factors (age, stimuli modality) on this purported relationship. In this meta-analysis, we examined 525 correlations from 79 published studies and unpublished datasets, representing data from 12,846 individual participants. We found a small but significant (r = .19) correlation between memory and creative cognition. Among semantic, episodic, working, and short-term memory, all correlations were significant, but semantic memory – particularly verbal fluency, the ability to strategically retrieve information from long-term memory – was found to drive this relationship. Further, working memory capacity was found to be more strongly related to convergent than divergent creative thinking. We also found that within visual creativity, the relationship with visual memory was greater than that of verbal memory, but within verbal creativity, the relationship with verbal memory was greater than that of visual memory. Finally, the memory-creativity correlation was larger for children compared to young adults despite no impact of age on the overall effect size. These results yield three key conclusions: (1) semantic memory supports both verbal and nonverbal creative thinking, (2) working memory supports convergent creative thinking, and (3) the cognitive control of memory is central to performance on creative thinking tasks. 

Human ratings are ubiquitous in creativity research. Yet the process of rating responses to creativity tasks—typically several hundred or thousands of responses, per rater—is often time consuming and expensive. Planned missing data designs, where raters only rate a subset of the total number of responses, have been recently proposed as one possible solution to decrease overall rating time and monetary costs. However, researchers also need ratings that adhere to psychometric standards, such as a certain degree of reliability, and psychometric work with planned missing designs is currently lacking in the literature. In this work, we introduce how judge response theory and simulations can be used to fine-tune planning of missing data designs. We provide open code for the community and illustrate our proposed approach by a cost-effectiveness calculation based on a realistic example. We clearly show that fine tuning helps to save time (to perform the ratings) and monetary costs, while simultaneously targeting expected levels of reliability.