An official website of the United States government
The purposes of this study included conducting a meta-analysis and reviewing the study reporting quality of math interventions implemented in informal learning environments (e.g., the home) by children’s caregivers. This meta-analysis included 25 preschool to third-grade math interventions with 83 effect sizes that yielded a statistically significant summary effect (g = 0.26, 95% CI [0.07, 0.45) on children’s math achievement. Significant moderators of the treatment effect included the intensity of caregiver training and type of outcome measure. There were larger average effects for interventions with caregiver training that included follow-up support and for outcomes that were comprehensive early numeracy measures. Studies met 58.0% of reporting quality indicators, and analyses revealed that quality of reporting has improved in recent years. The results of this study offer several recommendations for researchers and practitioners, particularly given the growing evidence base of math interventions conducted in informal learning environments.
more »
« less
This content will become publicly available on November 1, 2026
A meta-analysis of Richard Mayer's multimedia learning research: Searching for boundary conditions of design principles across multiple media types
Richard E. Mayer has made major contributions to Educational Psychology since the 1970s, including work on learning in mathematics, creativity, interest, measurement, problem solving, and especially multimedia learning, defined as learning from instructional material that includes information in both verbal and visual form. In a 2024 reflection, Mayer called for identifying boundary conditions—i.e., moderators of effects—of his multimedia design principles. In an effort to identify these, we meta-analyzed Mayer’s corpus of multimedia research. We searched Google Scholar, PsycINFO, and the Cambridge Handbook of Multimedia Learning 3rd Ed. for peer-reviewed articles on multimedia learning with Mayer as an author published 1990-2022 and located 92 articles reporting on 181 studies reporting on 591 separate effects. We coded for 9 moderators: multimedia design principle, multimedia type, age, academic domain, country/continent, treatment duration, dependent variable type, year, and authorship order. We analyzed the Hedge’s g effect sizes using a multilevel regression approach in the metafor package in R. The overall effect was g = 0.37, which was significantly moderated by all moderators, including a small decline in effect size per year. Mean effects by multimedia design principle were uneven, with the largest significant effects for removing seductive detail, modality principle, personalization, multimedia principle, sentence-level coherence, and self-explanation. Medium significant overall effects were found for the testing effect, scaffolding, cueing, and embodiment. Large, consistent effects were found for text + diagrams across factual, inferential, and transfer outcomes. Less-consistent effects were found for animation, games, and simulations, with smaller effects on factual learning and on average larger effects on inferential and transfer outcomes, but no significant effects for virtual reality. We identified two boundary conditions in tests of design principle x DV type interactions and Multimedia type x DV type interactions. We close by interpreting various findings in phases of Mayer’s work, characterized by collaborators and educational technologies. We also contextualize Mayer’s findings within recent meta-analyses of the larger published research on various design principles.
more »
« less
- Award ID(s):
- 1661231
- PAR ID:
- 10637927
- Publisher / Repository:
- Elsevier
- Date Published:
- Journal Name:
- Educational Research Review
- Volume:
- 49
- Issue:
- C
- ISSN:
- 1747-938X
- Page Range / eLocation ID:
- 100730
- Subject(s) / Keyword(s):
- animation games video text and diagrams virtual reality transfer of learning
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Greene, J A; Linnenbrink-Garcia, L (Ed.)Misinformation around scientific issues is rampant on social media platforms, raising concerns among educators and science communicators. A variety of approaches have been explored to confront this growing threat to science literacy. For example, refutations have been used both proactively as warning labels and in attempts to inoculate against misconceptions, and retroactively to debunk misconceptions and rebut science denialism. Refutations have been used by policy makers and scientists when communicating with the general public, yet little is known about their effectiveness or consequences. Given the interest in refutational approaches, we conducted a comprehensive, pre-registered meta-analysis comparing the effect of refutation texts to non-refutation texts on individuals’ misconceptions about scientific information. We selected 71 articles (53 published and 18 unpublished) that described 76 studies, 111 samples, and 294 effect sizes. We also examined 26 moderators. Overall, our findings show a consistent and statistically significant advantage of refutation texts over non-refutation texts in controlled experiments confronting scientific misconceptions. We also found that moderators neither enhanced nor diminished the impact of the refutation texts. We discuss the implications of using refutations in formal and informal science learning contexts and in science communications from three theoretical perspectives.more » « less
-
Silva, Daniel de (Ed.)Quantitatively summarizing results from a collection of primary studies with meta-analysis can help answer ecological questions and identify knowledge gaps. The accuracy of the answers depends on the quality of the meta-analysis. We reviewed the literature assessing the quality of ecological meta-analyses to evaluate current practices and highlight areas that need improvement. From each of the 18 review papers that evaluated the quality of meta-analyses, we calculated the percentage of meta-analyses that met criteria related to specific steps taken in the meta-analysis process (i.e., execution) and the clarity with which those steps were articulated (i.e., reporting). We also re-evaluated all the meta-analyses available from Pappalardo et al. [1] to extract new information on ten additional criteria and to assess how the meta-analyses recognized and addressed non-independence. In general, we observed better performance for criteria related to reporting than for criteria related to execution; however, there was a wide variation among criteria and meta-analyses. Meta-analyses had low compliance with regard to correcting for phylogenetic non-independence, exploring temporal trends in effect sizes, and conducting a multifactorial analysis of moderators (i.e., explanatory variables). In addition, although most meta-analyses included multiple effect sizes per study, only 66% acknowledged some type of non-independence. The types of non-independence reported were most often related to the design of the original experiment (e.g., the use of a shared control) than to other sources (e.g., phylogeny). We suggest that providing specific training and encouraging authors to follow the PRISMA EcoEvo checklist recently developed by O’Dea et al. [2] can improve the quality of ecological meta-analyses.more » « less
-
Abstract About 70% of people with osteogenesis imperfecta (OI) experience hearing loss. There is no cure for OI, and therapies to ameliorate hearing loss rely on conventional treatments for auditory impairments in the general population. The success rate of these treatments in the OI population with poor collagenous tissues is still unclear. Here, we conduct a systematic review and meta-analysis on the efficacy of treatments addressing hearing loss in OI. This study conforms to the reporting standards of the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA). Data sources include published articles in Medline via PubMed, Web of Science, Scopus, and Embase, from their inception to November 2020. Studies included individuals with OI undergoing a hearing loss treatment, having pre- and postoperative objective assessment of hearing function at a specified follow-up length. Our search identified 1144 articles, of which 67 were reviewed at full-text screening. A random-effects meta-analysis was conducted on the selected articles (n = 12) of people with OI that underwent stapes surgery. Success was assessed as the proportion of ears with a postoperative Air–Bone Gap (ABG) ≤ 10 dB. A systematic review was conducted on the remaining articles (n = 13) reporting on other treatments. No meta-analysis was conducted on the latter due to the low number of articles on the topic and the nature of single case studies. The meta-analysis shows that stapes surgeries have a low success rate of 59.08 (95% CI 45.87 to 71.66) in the OI population. The systematic review revealed that cochlear implants, bone-anchored hearing aids, and other implantable hearing aids proved to be feasible, although challenging, in the OI population, with only 2 unsuccessful cases among the 16 reviewed single cases. This analysis of published data on OI shows poor clinical outcomes for the procedures addressing hearing loss. Further studies on hearing loss treatments for OI people are needed. Notably, the mechanisms of hearing loss in OI need to be determined to develop successful and possibly non-invasive treatment strategies.more » « less
-
This protocol is for a study that examines personalized adaptive learning (PAL) of undergraduate students with a focus on mathematics outcomes. This protocol addresses two meta-analytic studies: 1) Examination of PAL within the population of undergraduate students enrolled in mathematics courses broadly (across all mathematics courses with the type of course as a moderator); and 2) Specifically in college algebra, which is a STEM pipeline mathematics course. Undergraduate mathematics courses can be roadblocks to college degree attainment. PAL offers the opportunity to develop mathematical skills in a personalized way, while enabling instructors to identify students’ struggle areas and take remedial action.Partnering with academic librarians, databases likely to index relevant studies were identified, the search strategy was refined, and keywords and subject indexing terms were identified. Titles and abstracts were independently screened by two reviewers. Full-text review is in progress. Retained studies will have data extracted that will allow the results to be meta-analyzed.The standardized mean difference will be the main effect size computed. Multilevel meta-analysis will be used to address the hierarchical data structure. Robust variance estimation will be used to estimate standard errors and hypothesis tests, addressing dependencies of effect sizes within studies. Prediction intervals for effect sizes, for study-average effect sizes, and heterogeneity statistics will be reported. Subgroup analyses will evaluate the relationship between moderators and the magnitude of the effect size. Cochrane’s criteria for assessing the risk of bias in studies with a separate control group will be applied. Reporting bias will be assessed visually with funnel plots.more » « less
