Search for: All records

Selected response items and constructed response (CR) items are often found in the same test. Conventional psychometric models for these two types of items typically focus on using the scores for correctness of the responses. Recent research suggests, however, that more information may be available from the CR items than just scores for correctness. In this study, we describe an approach in which a statistical topic model along with a diagnostic classification model (DCM) was applied to a mixed item format formative test of English and Language Arts. The DCM was used to estimate students’ mastery status of reading skills. These mastery statuses were then included in a topic model as covariates to predict students’ use of each of the latent topics in their written answers to a CR item. This approach enabled investigation of the effects of mastery status of reading skills on writing patterns. Results indicated that one of the skills, Integration of Knowledge and Ideas, helped detect and explain students’ writing patterns with respect to students’ use of individual topics. 

In this paper, we focus on the design of assessments of mathematics teachers’ knowledge by emphasising the importance of identifying the purpose for the assessment, defining the specific construct to be measured, and considering the affordances of particular psychometric models on the development of assessments as well as how they are able to communicate learning or understanding. We add to the literature by providing illustrations of the interactions among these critical considerations in determining what inferences can be drawn from an assessment. We illustrate how the considerations shape assessments by discussing both existing and ongoing research projects. We feature discussion of two projects on which the authors of this paper are collaborating to demonstrate the affordances of attending to all three considerations in designing assessments of mathematics teachers’ knowledge to provide readers with opportunities to see those considerations in use.