An official website of the United States government
Caring assessments is an assessment design framework that considers the learner as a whole and can be used to design assessment opportunities that learners find engaging and appropriate for demonstrating what they know and can do. This framework considers learners’ cognitive, meta-cognitive, intra-and inter-personal skills, aspects of the learning context, and cultural and linguistic backgrounds as ways to adapt assessments. Extending previous work on intelligent tutoring systems that “care” from the field of artificial intelligence in education (AIEd), this framework can inform research and development of personalized and socioculturally responsive assessments that support students’ needs. In this article, we (a) describe the caring assessment framework and its unique contributions to the field, (b) summarize current and emerging research on caring assessments related to students’ emotions, individual differences, and cultural contexts, and (c) discuss challenges and opportunities for future research on caring assessments in the service of developing and implementing personalized and socioculturally responsive interactive digital assessments.
more »
« less
This content will become publicly available on April 23, 2026
Validation for Personalized Assessments: A Threats‐to‐Validity Approach
Personalized assessments are of increasing interest because of their potential to lead to more equitable decisions about the examinees. However, one obstacle to the widespread use of personalized assessments is the lack of a measurement toolkit that can be used to analyze data from these assessments. This article takes one step toward building such a toolkit by proposing a validation framework for personalized assessments. The framework is built on the threats‐to‐validity approach. We demonstrate applications of the suggested framework using the AP 3D Art and Design Portfolio examination and a more restrictive culturally relevant assessment as examples.
more »
« less
- Award ID(s):
- 2243041
- PAR ID:
- 10586854
- Editor(s):
- Wang, Chun
- Publisher / Repository:
- Wiley
- Date Published:
- Journal Name:
- Journal of Educational Measurement
- Edition / Version:
- 1
- ISSN:
- 0022-0655
- Page Range / eLocation ID:
- 1-29
- Subject(s) / Keyword(s):
- Comparability, Kane's validity framework, Standardized tests
- Format(s):
- Medium: X Size: 2MB Other: pdf
- Size(s):
- 2MB
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The Academic Vigilance Environment (AVE) presented is a combination of two innovative tools. AchieveUp's micro-credentialing system identifies and showcase students' skills, while KnowGap's provides personalized learning content that fills knowledge gaps. To meet the growing demand for micro-credentials, AchieveUp integrates this capability into established courses using online quizzes to evaluate skills from a predefined test bank. By leveraging responses from digitized quiz-based assessments, we have developed a synergistic approach with online assessment and remediation protocols. Our Python-based toolkit enables undergraduate tutors to identify and address knowledge gaps among at-risk learners in higher-education courses. Through digitized assessments, personalized tutoring, and automated skill analysis scripts integrated into Canvas LMS, students receive skill-specific badges that provide incremental motivation and enhance their self-efficacy. In a required electrical and computer engineering course here at UCF, the implemented software allowed for the distribution of 17 unique digital badges suitable for LinkedIn posting, benefiting both students and employers by verifying skills, while also providing instructors with insights to improve course instruction.more » « less
-
null (Ed.)Digital health technology is becoming more ubiquitous in monitoring individuals’ health as both device functionality and overall prevalence increase. However, as individuals age, challenges arise with using this technology particularly when it involves neurodegenerative issues (e.g., for individuals with Parkinson’s disease, Alzheimer’s disease, and ALS). Traditionally, neurodegenerative diseases have been assessed in clinical settings using pen-and-paper style assessments; however, digital health systems allow for the collection of far more data than we ever could achieve using traditional methods. The objective of this work is the formation and implementation of a neurocognitive digital health system designed to go beyond what pen-and-paper based solutions can do through the collection of (a) objective, (b) longitudinal, and (c) symptom-specific data, for use in (d) personalized intervention protocols. This system supports the monitoring of all neurocognitive functions (e.g., motor, memory, speech, executive function, sensory, language, behavioral and psychological function, sleep, and autonomic function), while also providing methodologies for personalized intervention protocols. The use of specifically designed tablet-based assessments and wearable devices allows for the collection of objective digital biomarkers that aid in accurate diagnosis and longitudinal monitoring, while patient reported outcomes (e.g., by the diagnosed individual and caregivers) give additional insights for use in the formation of personalized interventions. As many interventions are a one-size-fits-all concept, digital health systems should be used to provide a far more comprehensive understanding of neurodegenerative conditions, to objectively evaluate patients, and form personalized intervention protocols to create a higher quality of life for individuals diagnosed with neurodegenerative diseases.more » « less
-
Within residences, normative messaging interventions have been gaining interest as a cost-effective way to promote energy-saving behaviors. Behavioral reference groups are one important factor in determining the effectiveness of normative messages. More personally relevant and meaningful groups are likely to promote behavior change. Using readily available energy-use profiles in a non-invasive manner permits the creation of highly personalized reference groups. Unfortunately, how data granularity (e.g., minute and hour) and aggregation (e.g., one week and one month) affect the performance of energy profile-based reference group categorization is not well understood. This research evaluates reference group categorization performance across different levels of data granularity and aggregation. We conduct a clustering analysis using one-year of energy use data from 2248 households in Holland, Michigan USA. The clustering analysis reveals that using six-hour intervals results in more personalized energy profile-based reference groups compared to using more granular data (e.g., 15 min). This also minimizes computational burdens. Further, aggregating energy-use data over all days of twelve weeks increases the group similarity compared to less aggregated data (e.g., weekdays of twelve weeks). The proposed categorization framework enables interveners to create personalized and scalable normative feedback messages.more » « less
-
The career paths of PhD scientists often deviate from their doctoral theses. As a result, the need to integrate student-centered career and professional development training is important to meet the needs of doctoral students. Qualifying exams (QEs) represent a significant milestone in progression toward graduation within most PhD Programs in the United States. These exams are commonly administered 2–3 years into a PhD program following the completion of coursework, with the primary objective of evaluating whether the candidate possesses the necessary knowledge and skills to progress with their dissertation research. To enhance the value of QEs and intentionally align them with the diverse career trajectories of our students, we explored the inclusion of student-centered assessments in a track with a Pharmaceutical Sciences PhD program. In this PhD program, one component of QEs is a series of monthly, written cumulative exams focused on recent scientific literature in the faculty and students’ discipline. To create a student-centered QE, the student and a faculty member collaborated to develop personalized assessments focused on career exploration and in alignment with individual student’s career goals. All students enrolled in the PhD track (n = 8) were invited to participate in a survey about their experience with the redesigned QE. A combination of Likert scale and short answer questions were collected; quantitative items were analyzed with descriptive statistics and qualitative items with thematic coding. A subset of survey participants (n = 5) participated in a focus group regarding their experience with both the Traditional Model QE and the redesigned Pilot Model QE. Two faculty interviews were conducted regarding the design, content, procedures, and evaluation of student QEs. The study design and analysis were grounded in the cognitive apprenticeship framework, with a focus on how the QEs were situated within the four domains of this framework: content, methods, sequencing, and sociology. Results revealed that this student-centered QE approach was perceived to be more aligned with student career aspirations and to have a high interest level and value for students without placing a substantial additional burden on participants. This suggests that it is a feasible mechanism for integrating student-centered assessment into QEs.more » « less
