An official website of the United States government
ABSTRACT Automated scoring is a current hot topic in creativity research. However, most research has focused on the English language and popular verbal creative thinking tasks, such as the alternate uses task. Therefore, in this study, we present a large language model approach for automated scoring of a scientific creative thinking task that assesses divergent ideation in experimental tasks in the German language. Participants are required to generate alternative explanations for an empirical observation. This work analyzed a total of 13,423 unique responses. To predict human ratings of originality, we used XLM‐RoBERTa (Cross‐lingual Language Model‐RoBERTa), a large, multilingual model. The prediction model was trained on 9,400 responses. Results showed a strong correlation between model predictions and human ratings in a held‐out test set (n = 2,682;r = 0.80; CI‐95% [0.79, 0.81]). These promising findings underscore the potential of large language models for automated scoring of scientific creative thinking in the German language. We encourage researchers to further investigate automated scoring of other domain‐specific creative thinking tasks.
more »
« less
Don't throw the “bad” ideas away! Multidimensional top scoring increases reliability of divergent thinking tasks
Scoring divergent thinking tasks opens multiple avenues and possibilities – decisions researchers have to make. While some scholars postulate that scoring should focus on the best ideas provided, the measurement of the best responses (e.g., “top scoring”) comes along with challenges. More specifically, compared to the average quality across all responses, top scoring uses less information—the “bad” ideas are thrown away—which decreases reliability. To resolve this issue, this article introduces a multidimensional top-scoring approach analogous to linear growth modeling which retains information provided by all responses (best ideas and “bad” ideas). Across two studies, using both subjective human ratings and semantic distance originality scoring of responses to over a dozen divergent thinking tasks, we demonstrated that Maximum (the best idea) and Top2 Scoring (two best ideas) could surpass typically applied average scoring in measurement precision when the “bad” ideas’ originality is used as auxiliary information (i.e., additional information in the analysis). We thus recommend retaining all ideas when scoring divergent thinking tasks, and we discuss the potential this new approach holds for creativity research and practice.
more »
« less
- Award ID(s):
- 1920653
- PAR ID:
- 10525791
- Publisher / Repository:
- American Psychological Association
- Date Published:
- Journal Name:
- Psychology of Aesthetics, Creativity, and the Arts
- ISSN:
- 1931-3896
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract The current study addresses gaps in our understanding of the relationship between creative cognition, intelligence (IQ), and executive functioning (EF). Undergraduate students completed an IQ test, verbal and figural divergent thinking (DT) tests, and a self‐assessment of EF, across four study sessions. Participant data (N = 199) were analyzed using linear regression and PROCESS moderation models. Results demonstrated that EF interacts with IQ to predict figural and verbal DT in distinct ways, with different patterns emerging from different methods of scoring DT. Using traditional DT scoring,Gf(but notGc) significantly moderated the relationship between EF and scores on both verbal and figural DT tasks. Low EF was associated with diminished DT scores for those with lowGfscores, unrelated for those with relatively higherGf, and enhanced scores for those with the highestGf. Using originality ratio scores, low EF was associated with diminished originality in verbal DT responses for those with low IQ (bothGfandGc), unrelated for those with relatively higher IQ, and enhanced originality for those with the highestGc(but notGf) scores. Thus, there are several nuances in the way that EF interacts with IQ to predict DT.more » « less
-
Childhood is a pinnacle of both creativity and curiosity, and although these constructs theoretically overlap, few studies have probed whether they are directly related in childhood or driven by similar cognitive and emotional processes. Across two online Zoom sessions, 36 3- to 6 year-olds completed six tasks measuring diverse manifestations of curiosity and creativity, as well as tasks assessing vocabulary, self-esteem, and executive function. Caregivers also completed questionnaires regarding their children's curiosity. Only two significant, positive correlations were found between indices of creativity and curiosity: between originality of ideas (creativity) and breadth of exploration (curiosity), and between creativity on a production-based task and parent-reported breadth of exploration (curiosity). Further, the two constructs were predicted by different child characteristics. Age was the main predictor of creativity; originality of children's ideas in two divergent thinking tasks decreased with age, while fluency and holistic ratings of production-based tasks increased. Self-esteem, in turn, was the strongest predictor of curiosity, correlating positively with several subtypes of parent-reported curiosity. The results of this exploratory study suggest creativity and curiosity may not be as closely linked in childhood as some have proposed, and that pinpointing their relations will require careful attention to the individual components and expressions of each construct.more » « less
-
Semantic distance scoring provides an attractive alternative to other scoring approaches for responses in creative thinking tasks. In addition, evidence in support of semantic distance scoring has increased over the last few years. In one recent approach, it has been proposed to combine multiple semantic spaces to better balance the idiosyncratic influences of each space. Thereby, final semantic distance scores for each response are represented by a composite or factor score. However, semantic spaces are not necessarily equally weighted in mean scores, and the usage of factor scores requires high levels of factor determinacy (i.e., the correlation between estimates and true factor scores). Hence, in this work, we examined the weighting underlying mean scores, mean scores of standardized variables, factor loadings, weights that maximize reliability, and equally effective weights on common verbal creative thinking tasks. Both empirical and simulated factor determinacy, as well as Gilmer-Feldt’s composite reliability, were mostly good to excellent (i.e., > .80) across two task types (Alternate Uses and Creative Word Association), eight samples of data, and all weighting approaches. Person-level validity findings were further highly comparable across weighting approaches. Observed nuances and challenges of different weightings and the question of using composites vs. factor scores are thoroughly provided.more » « less
-
Transcranial direct current stimulation (tDCS) over the dorsolateral prefrontal cortex (DLPFC) has been shown to enhance divergent and convergent creative thinking. Yet, how stimulation impacts creative performance over time, and what cognitive mechanisms underlie any such enhancement, remain largely unanswered questions. In the present research, we aimed to (1) verify the impact of DLPFC tDCS on both convergent and divergent thinking, and further investigated (2) the temporal dynamics of divergent thinking, focusing on the serial order effect (i.e., the tendency for ideas to become more original and less frequent over time), and (3) any role that cognitive inhibition may play in mediating any effect of stimulation on creative thinking (considering the DLPFC’s involvement in driving inhibitory processes that are also relevant for creative thinking). In a within-subjects design, twenty-six participants received three types of cross-hemispheric tDCS stimulation over the DLPFC (left cathodal and right anodal, L-R+; left anodal and right cathodal, L+R-; and sham). Before stimulation, they completed a pre-flanker task measuring cognitive inhibition; during stimulation, they completed the Alternate Uses Task (AUT), Remote Associates Test (RAT), and post-flanker task. Results showed that, compared with the sham stimulation, originality of responses in the AUT was significantly enhanced in the L+R- condition, while no tDCS effect was observed for the RAT. Additionally, compared with the other stimulation conditions, we found a diminished serial order effect in the L+R- condition characterized by an accelerated production of more original ideas. Critically, the L+R- condition was accompanied by better performance on the flanker task. Our findings thus verify that L+R- tDCS over the DLPFC accelerates idea originality also providing tentative clues that inhibition may act as a cognitive mechanism underlying enhancements in divergent thinking resulting from frontal lobe neuromodulation.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1037/aca0000571
