skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: A Multivariate Probit Model for Learning Trajectories: A Fine-Grained Evaluation of an Educational Intervention
Advances in educational technology provide teachers and schools with a wealth of information about student performance. A critical direction for educational research is to harvest the available longitudinal data to provide teachers with real-time diagnoses about students’ skill mastery. Cognitive diagnosis models (CDMs) offer educational researchers, policy makers, and practitioners a psychometric framework for designing instructionally relevant assessments and diagnoses about students’ skill profiles. In this article, the authors contribute to the literature on the development of longitudinal CDMs, by proposing a multivariate latent growth curve model to describe student learning trajectories over time. The model offers several advantages. First, the learning trajectory space is high-dimensional and previously developed models may not be applicable to educational studies that have a modest sample size. In contrast, the method offers a lower dimensional approximation and is more applicable for typical educational studies. Second, practitioners and researchers are interested in identifying factors that cause or relate to student skill acquisition. The framework can easily incorporate covariates to assess theoretical questions about factors that promote learning. The authors demonstrate the utility of their approach with an application to a pre- or post-test educational intervention study and show how the longitudinal CDM framework can provide fine-grained assessment of experimental effects.  more » « less
Award ID(s):
1758688
PAR ID:
10531329
Author(s) / Creator(s):
 ;  
Publisher / Repository:
SAGE Publications
Date Published:
Journal Name:
Applied Psychological Measurement
Volume:
44
Issue:
7-8
ISSN:
0146-6216
Format(s):
Medium: X Size: p. 515-530
Size(s):
p. 515-530
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Student research in STEM education is an important learning component for both undergraduate and graduate students. It is not sufficient for students to learn passively in lecture-based classrooms without engaging and immersing themselves in the educational process through real-world research learning. Experiential learning for STEM students can involve conducting research, alongside and through the guidance of their professors, early in a student’s undergraduate or graduate program. The authors consider such experiences to be the hallmark of a high-quality STEM education and something every student, undergraduate and graduate, should have during the course of their programs. The purpose of this case study is to document the faculty authors’ experiences in student-faculty research and provide guidance and recommendations for best practices based upon the authors’ experience, data, and literature findings. Moreover, the study presents the experience of the faculty authors’ international student researchers in STEM with focus on two student researchers, one undergraduate and one graduate, who are international STEM. The students served as co-authors on this project. Findings from this case study indicate that students were highly engaged in the research process and found these skills valuable preparation for further study and career. Moreover, the students expressed enthusiasm and engagement for the research process. 
    more » « less
  2. Society’s increased reliance on technology has simultaneously increased the demand for people who can develop and design these new advancements. This has led to an influx of students looking to learn how to code and gain the technological skill set that is currently among the most marketable. Learning to code is challenging; without the right tools, resources, and assistance, it can be tough to build the foundation needed to understand key computer science fundamentals. The existing web platforms focused on assisting K-12 learners are competitive from an educational and technical perspective. There is a huge lack of virtual educational platforms that can deliver resources to students with disabilities through innovative accessible features and provide guidance to K-12 teachers that are trying to support this area. This lack of guidance is especially evident when examining resources available to teachers about increasing access and engagement of struggling learners including students with disabilities. The motivation of this paper is to introduce a prototype of a centralized portal, Accessible Virtual Learning, that implements user experience strategies and accessible usability principles aiming to be accessible to any student and also educators who need guidance on finding suitable materials. The success of this portal relies heavily on its ability to allow teachers and self-directed learners to facilitate curriculums effectively while maximizing student engagement, ease of learning, and digital assistance for students at various ages with different learning abilities, both physical and cognitive. 
    more » « less
  3. Schools’ overt or explicit practices are a dominant lens through which education researchers and policymakers attempt to understand how schools are racially inequitable. Yet, Lewis and Diamond argue that contemporary racial inequalities are largely sustained through implicit factors, like institutional practices and structural inequalities. Ray’s framework on racialized organizations similarly outlines how our racialized sociopolitical structure becomes embedded in organizations, legitimating and perpetuating the racialized hierarchy. We apply illustrative cluster analysis techniques to rich data on schools, teachers, and students from the nationally representative High School Longitudinal Study of 2009 to find that structural inequities (e.g., student body, sector, average achievement) appear to be most salient in delineating the racialization of US high schools, whereas the characteristics of schools and teachers that are typically emphasized for closing racial inequities in educational outcomes (e.g., teacher qualifications, courses offered, stratification practices) are not salient differentiators across schools. 
    more » « less
  4. Jones, D.; Ryan, Q.; Pawl, A. (Ed.)
    Data modeling and graphing skill sets are foundational to science learning and careers, yet students regularly struggle to master these basic competencies. Further, although educational researchers have uncovered numerous approaches to support sense-making with mathematical models of motion, teachers sometimes struggle to enact them due to a variety of reasons, including limited time and materials for lab-based teaching opportunities and a lack of awareness of student learning difficulties. In this paper, we introduce a free smartphone application that uses LiDAR data to support motion-based physics learning with an emphasis on graphing and mathematical modeling. We tested the embodied technology, called LiDAR Motion, with 106 students in a non-major, undergraduate physics classroom at a mid-sized, private university on the U.S. East Coast. In identical learning assessments issued both before and after the study, students working with LiDAR Motion improved their scores by a more significant margin than those using standard issue sonic rangers. Further, per a voluntary survey, students who used both technologies expressed a preference for LiDAR Motion. This mobile application holds potential for improving student learning in the classroom, at home, and in alternative learning environments. 
    more » « less
  5. his work-in-progress paper expands on a collaboration between engineering education researchers and machine learning researchers to automate the analysis of written responses to conceptually challenging questions in statics and dynamics courses (Authors, 2022). Using the Concept Warehouse (Koretsky et al., 2014), written justifications of ConcepTests (CTs) were gathered from statics and dynamics courses in a diverse set of two- and four-year institutions. Written justifications for CTs have been used to support active learning pedagogies which makes them important to investigate how students put together their problem-solving narratives of understanding. However, despite the large benefit that analysis of student written responses may provide to instructors and researchers, manual review of responses is cumbersome, limits analysis, and can be prone to human bias. In efforts to improve the analysis of student written responses, machine learning has been used in various educational contexts to analyze short and long texts (Burstein et al., 2020; Burstein et al., 2021). Natural Language Processing (NLP) uses transformer-based machine learning models (Brown et al., 2020; Raffel et al., 2019) which can be used through fine-tuning or in-context learning methods. NLP can be used to train algorithms that can automate the coding of written responses. Only a few studies for educational applications have leveraged transformer-based machine learning models further prompting an investigation into its use in STEM education. However, work in NLP has been criticized for heightening the possibility to perpetuate and even amplify harmful stereotypes and implicit biases (Chang et al., 2019; Mayfield et al., 2019). In this study, we detail the aim to use NLP for linguistic justice. Using methods like text summary, topic modeling, and text classification, we identify key aspects of student narratives of understanding in written responses to mechanics and statics CTs. Through this process, we seek to use machine learning to identify different ways students talk about a problem and their understanding at any point in their narrative formation process. Thus, we hope to help reduce human bias in the classroom and through technology by giving instructors and researchers a diverse set of narratives that include insight into their students’ histories, identities, and understanding. These can then be used towards connecting technological knowledge to students’ everyday lives. 
    more » « less