An official website of the United States government
The purposes of this study included conducting a meta-analysis and reviewing the study reporting quality of math interventions implemented in informal learning environments (e.g., the home) by children’s caregivers. This meta-analysis included 25 preschool to third-grade math interventions with 83 effect sizes that yielded a statistically significant summary effect (g = 0.26, 95% CI [0.07, 0.45) on children’s math achievement. Significant moderators of the treatment effect included the intensity of caregiver training and type of outcome measure. There were larger average effects for interventions with caregiver training that included follow-up support and for outcomes that were comprehensive early numeracy measures. Studies met 58.0% of reporting quality indicators, and analyses revealed that quality of reporting has improved in recent years. The results of this study offer several recommendations for researchers and practitioners, particularly given the growing evidence base of math interventions conducted in informal learning environments.
more »
« less
A meta-evaluation of the quality of reporting and execution in ecological meta-analyses
Quantitatively summarizing results from a collection of primary studies with meta-analysis can help answer ecological questions and identify knowledge gaps. The accuracy of the answers depends on the quality of the meta-analysis. We reviewed the literature assessing the quality of ecological meta-analyses to evaluate current practices and highlight areas that need improvement. From each of the 18 review papers that evaluated the quality of meta-analyses, we calculated the percentage of meta-analyses that met criteria related to specific steps taken in the meta-analysis process (i.e., execution) and the clarity with which those steps were articulated (i.e., reporting). We also re-evaluated all the meta-analyses available from Pappalardo et al. [1] to extract new information on ten additional criteria and to assess how the meta-analyses recognized and addressed non-independence. In general, we observed better performance for criteria related to reporting than for criteria related to execution; however, there was a wide variation among criteria and meta-analyses. Meta-analyses had low compliance with regard to correcting for phylogenetic non-independence, exploring temporal trends in effect sizes, and conducting a multifactorial analysis of moderators (i.e., explanatory variables). In addition, although most meta-analyses included multiple effect sizes per study, only 66% acknowledged some type of non-independence. The types of non-independence reported were most often related to the design of the original experiment (e.g., the use of a shared control) than to other sources (e.g., phylogeny). We suggest that providing specific training and encouraging authors to follow the PRISMA EcoEvo checklist recently developed by O’Dea et al. [2] can improve the quality of ecological meta-analyses.
more »
« less
- Award ID(s):
- 1655426
- PAR ID:
- 10479615
- Editor(s):
- Silva, Daniel de
- Publisher / Repository:
- PlosOne
- Date Published:
- Journal Name:
- PLOS ONE
- Volume:
- 18
- Issue:
- 10
- ISSN:
- 1932-6203
- Page Range / eLocation ID:
- e0292606
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract In ecological meta‐analyses, nonindependence among observed effect sizes from the same source paper is common. If not accounted for, nonindependence can seriously undermine inferences. We compared the performance of four meta‐analysis methods that attempt to address such nonindependence and the standard random‐effect model that ignores nonindependence. We simulated data with various types of within‐paper nonindependence, and assessed the standard deviation of the estimated mean effect size and Type I error rate of each method. Although all four methods performed substantially better than the standard random‐effects model that assumes independence, there were differences in performance among the methods. A two‐step method that first summarizes the multiple observed effect sizes per paper using a weighted mean and then analyzes the reduced data in a standard random‐effects model, and a robust variance estimation method performed consistently well. A hierarchical model with both random paper and study effects gave precise estimates but had a higher Type I error rates, possibly reflecting limitations of currently available meta‐analysis software. Overall, we advocate the use of the two‐step method with a weighted paper mean and the robust variance estimation method as reliable ways to handle within‐paper nonindependence in ecological meta‐analyses.more » « less
-
null (Ed.)Impairments related to figurative language understanding have been considered to be one of the diagnostic and defining features of autism. Metaphor comprehension and production in autism spectrum disorder (ASD) as compared to typically developing (TD) individuals have been investigated for around thirty years, generally showing an overall advantage for TD groups. We present a preregistered systematic review and meta-analysis including a total of 15 studies that fulfilled our set of inclusion criteria (notably, ASD and TD groups matched in chronological age and verbal- or full-scale IQ). Along with accuracy, we also analyzed group differences in reaction time in the studies that reported them. The results revealed a medium-to-large group difference favoring TD over ASD groups based on accuracy measures, as well as a similar overall advantage for TD groups based on reaction times. There was reliable heterogeneity in effect sizes for group differences in accuracy, which was mostly explained by the effect of verbal intelligence, with differences in metaphor processing being smaller for participants with better verbal skills. Some of the variation in effect sizes may also be attributed to differences in types of metaphor processing tasks. We also evaluated the quality of the studies included in the meta-analysis, and the evidence relating to the potential presence of publication bias.more » « less
-
This protocol is for a study that examines personalized adaptive learning (PAL) of undergraduate students with a focus on mathematics outcomes. This protocol addresses two meta-analytic studies: 1) Examination of PAL within the population of undergraduate students enrolled in mathematics courses broadly (across all mathematics courses with the type of course as a moderator); and 2) Specifically in college algebra, which is a STEM pipeline mathematics course. Undergraduate mathematics courses can be roadblocks to college degree attainment. PAL offers the opportunity to develop mathematical skills in a personalized way, while enabling instructors to identify students’ struggle areas and take remedial action.Partnering with academic librarians, databases likely to index relevant studies were identified, the search strategy was refined, and keywords and subject indexing terms were identified. Titles and abstracts were independently screened by two reviewers. Full-text review is in progress. Retained studies will have data extracted that will allow the results to be meta-analyzed.The standardized mean difference will be the main effect size computed. Multilevel meta-analysis will be used to address the hierarchical data structure. Robust variance estimation will be used to estimate standard errors and hypothesis tests, addressing dependencies of effect sizes within studies. Prediction intervals for effect sizes, for study-average effect sizes, and heterogeneity statistics will be reported. Subgroup analyses will evaluate the relationship between moderators and the magnitude of the effect size. Cochrane’s criteria for assessing the risk of bias in studies with a separate control group will be applied. Reporting bias will be assessed visually with funnel plots.more » « less
-
Abstract Explaining the maintenance of genetic variation in fitness‐related traits within populations is a fundamental challenge in ecology and evolutionary biology. Frequency‐dependent selection (FDS) is one mechanism that can maintain such variation, especially when selection favours rare variants (negative FDS). However, our general knowledge about the occurrence of FDS, its strength and direction remain fragmented, limiting general inferences about this important evolutionary process. We systematically reviewed the published literature on FDS and assembled a database of 747 effect sizes from 101 studies to analyse the occurrence, strength, and direction of FDS, and the factors that could explain heterogeneity in FDS. Using a meta‐analysis, we found that overall, FDS is more commonly negative, although not significantly when accounting for phylogeny. An analysis of absolute values of effect sizes, however, revealed the widespread occurrence of modest FDS. However, negative FDS was only significant in laboratory experiments and non‐significant in mesocosms and field‐based studies. Moreover, negative FDS was stronger in studies measuring fecundity and involving resource competition over studies using other fitness components or focused on other ecological interactions. Our study unveils key general patterns of FDS and points in future promising research directions that can help us understand a long‐standing fundamental problem in evolutionary biology and its consequences for demography and ecological dynamics.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1371/journal.pone.0292606
