An official website of the United States government
An instructor-facing Graphical User Interface (GUI) for micro-credential formation and tracking within STEM curricula is designed, prototyped, and evaluated. It enables instructors to readily associate skills with response-level elements within digitized assessments of Canvas LMS for issuance of micro-credential badges, without the need for computer programming expertise. An instructor survey of preference and acceptance levels of skill tracking was conducted. Survey results were used to design an input flow and interface to tag skills. The Tkinter Python-based computer coding library was utilized to create rapid check buttons and the Matplotlib backend was deployed to generate pie chart views of instantaneous course performance against tagged skills. A case study demonstrates the process for highest and lowest achieved skills via an auto-appending CSV file, allowing the instructor to identify and react to semester-long performance of learners by tracking badge issuance rates up to 17.3% within a required Computer Engineering undergraduate course.
more »
« less
This content will become publicly available on April 11, 2026
Automated Micro-Credentialing and AI-Enhanced Remediation for Online STEM Assessments
The Academic Vigilance Environment (AVE) presented is a combination of two innovative tools. AchieveUp's micro-credentialing system identifies and showcase students' skills, while KnowGap's provides personalized learning content that fills knowledge gaps. To meet the growing demand for micro-credentials, AchieveUp integrates this capability into established courses using online quizzes to evaluate skills from a predefined test bank. By leveraging responses from digitized quiz-based assessments, we have developed a synergistic approach with online assessment and remediation protocols. Our Python-based toolkit enables undergraduate tutors to identify and address knowledge gaps among at-risk learners in higher-education courses. Through digitized assessments, personalized tutoring, and automated skill analysis scripts integrated into Canvas LMS, students receive skill-specific badges that provide incremental motivation and enhance their self-efficacy. In a required electrical and computer engineering course here at UCF, the implemented software allowed for the distribution of 17 unique digital badges suitable for LinkedIn posting, benefiting both students and employers by verifying skills, while also providing instructors with insights to improve course instruction.
more »
« less
- Award ID(s):
- 1953606
- PAR ID:
- 10581767
- Publisher / Repository:
- 2025 Florida Online Innovation Summit (FOIS)
- Date Published:
- Format(s):
- Medium: X
- Location:
- Orlando, FL, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
A model that maps the requisite skills, or knowledge components, to the contents of an online course is necessary to implement many adaptive learning technologies. However, developing a skill model and tagging courseware contents with individual skills can be expensive and error prone. We propose a technology to automatically identify latent skills from instructional text on existing online courseware called Smart (Skill Model mining with Automated detection of Resemblance among Texts). Smart is capable of mining, labeling, and mapping skills without using an existing skill model or student learning (aka response) data. The goal of our proposed approach is to mine latent skills from assessment items included in existing courseware, provide discovered skills with human-friendly labels, and map didactic paragraph texts with skills. This way, mapping between assessment items and paragraph texts is formed. In doing so, automated skill models produced by Smart will reduce the workload of courseware developers while enabling adaptive online content at the launch of the course. In our evaluation study, we applied Smart to two existing authentic online courses. We then compared machine-generated skill models and human-crafted skill models in terms of the accuracy of predicting students’ learning. We also evaluated the similarity between machine-generated and human-crafted skill models. The results show that student models based on Smart-generated skill models were equally predictive of students’ learning as those based on human-crafted skill models— as validated on two OLI (Open Learning Initiative) courses. Also, Smart can generate skill models that are highly similar to human-crafted models as evidenced by the normalized mutual information (NMI) values.more » « less
-
A model that maps the requisite skills, or knowledge components, to the contents of an online course is necessary to implement many adaptive learning technologies. However, developing a skill model and tagging courseware contents with individual skills can be expensive and error prone. We propose a technology to automatically identify latent skills from instructional text on existing online courseware called Smart (Skill Model mining with Automated detection of Resemblance among Texts). Smart is capable of mining, labeling, and mapping skills without using an existing skill model or student learning (aka response) data. The goal of our proposed approach is to mine latent skills from assessment items included in existing courseware, provide discovered skills with human-friendly labels, and map didactic paragraph texts with skills. This way, mapping between assessment items and paragraph texts is formed. In doing so, automated skill models produced by Smart will reduce the workload of courseware developers while enabling adaptive online content at the launch of the course. In our evaluation study, we applied Smart to two existing authentic online courses. We then compared machine-generated skill models and human-crafted skill models in terms of the accuracy of predicting students’ learning. We also evaluated the similarity between machine-generated and human-crafted skill models. The results show that student models based on Smart-generated skill models were equally predictive of students’ learning as those based on human-crafted skill models— as validated on two OLI (Open Learning Initiative) courses. Also, Smart can generate skill models that are highly similar to human-crafted models as evidenced by the normalized mutual information (NMI) values.more » « less
-
null (Ed.)Background. It is well recognized that current graduate education is too narrowly focused on thesis research. Graduate students have a strong desire to gain skills for their future career success beyond thesis research. This obvious gap in professional skill training in current graduate study also leads to the common student perception that professional skills beyond academic knowledge should only be gained after completion of thesis research. Purpose. A new program is being developed to rigorously integrate professional skills training with thesis research. The approach is to establish learning communities of Graduates for Advancing Professional Skills (GAPS) to incorporate project management skill training from industry into academic research. The GAPS program seeks to address two fundamental education research questions: How can project management skill training be integrated with thesis research in graduate education? What is the role/value of learning communities in enhancing the training and retention of professional skills and the effectiveness of thesis research? Our proposed solution is that graduate student learning communities engaging in a blended online and classroom approach will promote learning of professional skills such as project and time management in thesis research activities. The purpose of this session is to establish the connection between project management and thesis research, and demonstrate the beginning progress of the GAPS program towards. Methodology/approach. The following progress is being made to establish GAPS learning communities through which to teach and practice professional skills. A website has been developed to introduce the program, recruit participants, provide information on the online modules, and survey results of participants’ current levels of knowledge and skills related to project management. A new course, “Introduction of Project Management for Thesis Research”, has been added to the course catalog and open to enrollment for students from different majors. In addition, learning modules including project charter, scheduling, communication, teamwork, critical path method, and lean concept are developed. Case studies and examples have been developed to help students learn how to utilize project management skills in their thesis research. Conclusions. The concept of integrating professional skills training with thesis research through learning communities has been demonstrated. There are multiple advantages of this approach, including efficient utilization of the current resources, and faculty buy-in. Preliminary data from the first cohort are being collected and analyzed to identify students’ needs, benefits of the program, and areas of improvement for future cohort iterations. Implications. The GAPS program will improve professional skill training for graduate students through communities of practice. This new learning model has the potential to fundamentally change the culture of graduate education. We believe the method demonstrated here can be broadly applied to different engineering majors, and even broadly to all thesis research.more » « less
-
Abstract Undergraduate STEM lecture courses enroll hundreds who must master declarative, conceptual, and applied learning objectives. To support them, instructors have turned to active learning designs that require students to engage inself-regulated learning(SRL). Undergraduates struggle with SRL, and universities provide courses, workshops, and digital training to scaffold SRL skill development and enactment. We examined two theory-aligned designs of digital skill trainings that scaffold SRL and how students’ demonstration of metacognitive knowledge of learning skills predicted exam performance in biology courses where training took place. In Study 1, students’ (n = 49) responses to training activities were scored for quality and summed by training topic and level of understanding. Behavioral and environmental regulation knowledge predicted midterm and final exam grades; knowledge of SRL processes did not. Declarative and conceptual levels of skill-mastery predicted exam performance; application-level knowledge did not. When modeled by topic at each level of understanding, declarative knowledge of behavioral and environmental regulation and conceptual knowledge of cognitive strategies predicted final exam performance. In Study 2 (n = 62), knowledge demonstrated during a redesigned video-based multimedia version of behavioral and environmental regulation again predicted biology exam performance. Across studies, performance on training activities designed in alignment with skill-training models predicted course performances and predictions were sustained in a redesign prioritizing learning efficiency. Training learners’ SRL skills –and specifically cognitive strategies and environmental regulation– benefited their later biology course performances across studies, which demonstrate the value of providing brief, digital activities to develop learning skills. Ongoing refinement to materials designed to develop metacognitive processing and learners’ ability to apply skills in new contexts can increase benefits.more » « less
