Search for: All records

Abstract Traditional approaches to the modeling of multiple‐choice item response data (e.g., 3PL, 4PL models) emphasize slips and guesses as random events. In this paper, an item response model is presented that characterizes both disjunctively interacting guessing and conjunctively interacting slipping processes as proficiency‐related phenomena. We show how evidence for this perspective is seen in the systematic form of invariance violations for item slip and guess parameters under four‐parameter IRT models when compared across populations of different mean proficiency levels. Specifically, higher proficiency populations tend to show higher guess and lower slip probabilities than lower proficiency populations. The results undermine the use of traditional models for IRT applications that require invariance and would suggest greater attention to alternatives. 

Abstract Item difficulty and dimensionality often correlate, implying that unidimensional IRT approximations to multidimensional data (i.e., reference composites) can take a curvilinear form in the multidimensional space. Although this issue has been previously discussed in the context of vertical scaling applications, we illustrate how such a phenomenon can also easily occur within individual tests. Measures of reading proficiency, for example, often use different task types within a single assessment, a feature that may not only lead to multidimensionality, but also an association between item difficulty and dimensionality. Using a latent regression strategy, we demonstrate through simulations and empirical analysis how associations between dimensionality and difficulty yield a nonlinear reference composite where the weights of the underlying dimensionschangeacross the scale continuum according to the difficulties of the items associated with the dimensions. We further show how this form of curvilinearity produces systematic forms of misspecification in traditional unidimensional IRT models (e.g., 2PL) and can be better accommodated by models such as monotone‐polynomial or asymmetric IRT models. Simulations and a real‐data example from the Early Childhood Longitudinal Study—Kindergarten are provided for demonstration. Some implications for measurement modeling and for understanding the effects of 2PL misspecification on measurement metrics are discussed. 

Abstract In computer‐based tests allowing revision and reviews, examinees' sequence of visits and answer changes to questions can be recorded. The variable‐length revision log data introduce new complexities to the collected data but, at the same time, provide additional information on examinees' test‐taking behavior, which can inform test development and instructions. In the current study, we used recently proposed statistical learning methods for sequence data to provide an exploratory analysis of item‐level revision and review log data. Based on the revision log data collected from computer‐based classroom assessments, common prototypes of revisit and review behavior were identified. The relationship between revision behavior and various item, test, and individual covariates was further explored under a Bayesian multivariate generalized linear mixed model. 

Background/Context:Schools are increasingly using scripted curricula that limit teacher autonomy. These limitations are exacerbated when scripted curricula are enacted in fully standardized, asynchronous online course environments with no mechanisms for student–teacher communication. Purpose:This study extends understanding of how teacher discretion, identity, and the relationship between those two components shape students’ educational experiences online. Research Design:Within a sequential mixed method design, we identified spaces for teacher discretion using critical discourse analysis. By coding lesson transcripts, we developed a typology of common strategies: friendly, directive, personalized, and procedural. We used the resulting typology to run statistical models examining associations among teacher identity, discretionary acts, and student achievement. Lastly, we turned back to the qualitative data to confirm findings, test hypotheses, and provide nuance. Findings:Teachers presenting as Black were significantly more likely to use a procedural approach and significantly less likely to use friendly strategies. Students scored higher on their end-of-lesson quiz when their teacher used personalized strategies, such as sharing relevant personal experiences, and scored lower when teachers used friendly or directive strategies. Conclusions:Findings have implications for understanding and enacting equitable educational practices in asynchronous, scripted online environments. The isolation of discretionary acts feasible within the virtual learning environment studied contributes nuance to knowledge of the mechanisms through which teacher discretion might result in more favorable learning outcomes for students belonging to minoritized groups. 

Abstract Private nonprofit colleges are increasingly using tuition resets, or a decrease in sticker price by at least 5%, to attract new students and counter declining demand. While discounting tuition with institutional aid is a common practice to get accepted students to matriculate and to increase affordability, a tuition reset is a more transparent approach that moves colleges away from a high aid/high tuition model. The authors find minimal evidence that these policies increase student enrollment in the long run, but that there may be short-term impacts. As expected, institutional aid decreases and varies directly with the size of the sticker price reduction. The average net price students pay decreases, but this effect may be driven by changes in the estimated non-tuition elements of the total cost of attendance. Finally, net tuition revenue appears unrelated to tuition resets. These findings call into question the efficacy of this practice. 

Using administrative data of center-based child care providers in North Carolina from 2005 to 2018, we provide the first direct evidence on the effects of competition on provider quality and student outcomes in the context of early care and education, taking advantage of quality measures from the state’s Quality Rating and Improvement System (QRIS). We found that competition was associated with higher quality ratings and a higher probability to achieve a five-star rating—the highest tier in the QRIS. More competition increased providers’ probability to improve their rating and reduced the time to improve. Compared to public schools, private providers were responsive to competition. However, we did not find any effects of competition on district-level student third-grade academic performance. 

Causal decomposition analysis is among the rapidly growing number of tools for identifying factors (“mediators”) that contribute to disparities in outcomes between social groups. An example of such mediators is college completion, which explains later health disparities between Black women and White men. The goal is to quantify how much a disparity would be reduced (or remain) if we hypothetically intervened to set the mediator distribution equal across social groups. Despite increasing interest in estimating disparity reduction and the disparity that remains, various estimation procedures are not straightforward, and researchers have scant guidance for choosing an optimal method. In this article, the authors evaluate the performance in terms of bias, variance, and coverage of three approaches that use different modeling strategies: (1) regression-based methods that impose restrictive modeling assumptions (e.g., linearity) and (2) weighting-based and (3) imputation-based methods that rely on the observed distribution of variables. The authors find a trade-off between the modeling assumptions required in the method and its performance. In terms of performance, regression-based methods operate best as long as the restrictive assumption of linearity is met. Methods relying on mediator models without imposing any modeling assumptions are sensitive to the ratio of the group-mediator association to the mediator-outcome association. These results highlight the importance of selecting an appropriate estimation procedure considering the data at hand.