More Like this

1. Improving Automated Evaluation of Student Text Responses Using GPT-3.5 for Text Data Augmentation

   Cochran, Keith ; Cohn, Clayton ; Rouet, J.F. ; Hastings, Peter ( June 2023 , Springer, Cham.)

   Wang, N. (Ed.)

   In education, intelligent learning environments allow students to choose how to tackle open-ended tasks while monitoring performance and behavior, allowing for the creation of adaptive support to help students overcome challenges. Timely feedback is critical to aid students’ progression toward learning and improved problem-solving. Feedback on text-based student responses can be delayed when teachers are overloaded with work. Automated evaluation can provide quick student feedback while easing the manual evaluation burden for teachers in areas with a high teacher-to-student ratio. Current methods of evaluating student essay responses to questions have included transformer-based natural language processing models with varying degrees of success. One main challenge in training these models is the scarcity of data for student-generated data. Larger volumes of training data are needed to create models that perform at a sufficient level of accuracy. Some studies have vast data, but large quantities are difficult to obtain when educational studies involve student-generated text. To overcome this data scarcity issue, text augmentation techniques have been employed to balance and expand the data set so that models can be trained with higher accuracy, leading to more reliable evaluation and categorization of student answers to aid teachers in the student’s learning progression. This paper examines the text-generating AI model, GPT-3.5, to determine if prompt-based text-generation methods are viable for generating additional text to supplement small sets of student responses for machine learning model training. We augmented student responses across two domains using GPT-3.5 completions and used that data to train a multilingual BERT model. Our results show that text generation can improve model performance on small data sets over simple self-augmentation. 

   more » « less
2. The information won't just sink in: Helping teachers provide technology‐assisted data literacy instruction in social studies

   https://doi.org/10.1111/bjet.13255  

   Shreiner, Tamara L. ; Guzdial, Mark ( July 2022 , British Journal of Educational Technology)

   In this study, support for teaching data literacy in social studies is provided through the design of a pedagogical support system informed by participatory design sessions with both pre‐service and in‐service social studies teachers. It provides instruction on teaching and learning data literacy in social studies, examples of standards‐based lesson plans, made‐to‐purpose data visualization tools and minimal manuals that put existing online tools in a social studies context. Based on case studies of eleven practicing teachers, this study provides insight into features of technology resources that social studies teachers find usable and useful for using data visualizations as part of standards‐ and inquiry‐based social studies instruction, teaching critical analysis of data visualizations and helping students create data visualizations with online computing tools. The final result, though, is that few of our participating teachers have yet adopted the provided resources into their own classrooms, which highlights weaknesses of the technology acceptance model for describing teacher adoption.

   What is already known about this topic

   Data literacy is an important part of social studies education in the United States.

   Most teachers do not teach data literacy as a part of social studies.

   Teachers may adopt technology to help them teach data literacy if they think it is useful and usable.

   What this paper adds

   Educational technology can help teachers learn about data literacy in social studies.

   Social studies teachers want simple tools that fit with their existing curricula, give them new project ideas and help students learn difficult concepts.

   Making tools useful and usable does not predict adoption; context plays a large role in a social studies teachers' adoption.

   Implications for practice and/or policy

   Designing purpose‐built tools for social studies teachers will encourage them to teach data literacy in their classes.

   Professional learning opportunities for teachers around data literacy should include opportunities for experimentation with tools.

   Teachers are not likely to use tools if they are not accompanied by lesson and project ideas.

    

   more » « less
3. Toward a Theoretical Framework for Data Fluency Teaching and Learning in Middle School STEM

   Wong N. ; Elsayed, R. ; Perez, L. R. ; Daehler, K. R. ; & Chen, P. ( March 2024 , Ninety-seventh annual international conference of the National Association for Research in Science Teaching (NARST))

   This paper presents findings from a qualitative study of eleven experienced STEM educators who worked alongside developers to design and implement data-rich lessons in their grades 6–9 mathematics and science classrooms. In the context of a project that seeks to develop professional learning for data fluency, researchers documented the co-development process to articulate a model of what teachers need to know and be able to do in order to support their students’ data fluency. The project team distilled key findings into two framing documents: 1) a description of high-leverage areas of focus for PL which highlight challenges faced by teachers, which are common, important for data fluency, and represent opportunities for supporting teacher and student growth; and 2) a logic model that describes how the PL course under development is expected to influence teacher, classroom, and student outcomes. This paper contributes to the larger education community by defining the professional learning needs of educators who wish to integrate data into their STEM classrooms. These frameworks provide designers and researchers with touchpoints to structure and study PL experiences, lesson materials, and other classroom resources for both new and veteran educators. These tools can provide STEM teachers with guidance for reflecting on their current knowledge, skills, beliefs, and teaching practices that help their students become more data fluent. 

   more » « less
4. Leveraging Auxiliary Data from Similar Problems to Improve Automatic Open Response Scoring

   Rivera-Bergollo, Raysa ; Baral, Sami ; Botelho, Anthony ; Heffernan, Neil ( July 2022 , Conference for Educational Data Mining)

   As computer-based learning platforms have become ubiquitous, there is a growing need to better support teachers. Particularly in mathematics, teachers often rely on openended questions to assess students’ understanding. While prior works focusing on the development of automated openended work assessments have demonstrated their potential, many of those methods require large amounts of student data to make reliable estimates. We explore whether a problem specific automated scoring model could benefit from auxiliary data collected from similar problems to address this “cold start” problem. We examine factors such as sample size and the magnitude of similarity of utilized problem data. We find the use of data from similar problems not only provides benefits to improve predictive performance by increasing sample size, but also leads to greater overall model performance than using data solely from the original problem when sample size is held constant. 

   more » « less
5. Leveraging Auxiliary Data from Similar Problems to Improve Automatic Open Response Scoring

   Rivera-Bergollo, Raysa ; Baral, Sami ; Botelho, Anthony F ; Heffernan, Neil T ( June 2022 , Proceedings of the 15th International Conference on Educational Data Mining,)

   As computer-based learning platforms have become ubiq- uitous, there is a growing need to better support teachers. Particularly in mathematics, teachers often rely on open- ended questions to assess students’ understanding. While prior works focusing on the development of automated open- ended work assessments have demonstrated their potential, many of those methods require large amounts of student data to make reliable estimates. We explore whether a prob- lem specific automated scoring model could benefit from auxiliary data collected from similar problems to address this “cold start” problem. We examine factors such as sam- ple size and the magnitude of similarity of utilized problem data. We find the use of data from similar problems not only provides benefits to improve predictive performance by in- creasing sample size, but also leads to greater overall model performance than using data solely from the original prob- lem when sample size is held constant. 

   more » « less