An official website of the United States government
his work-in-progress paper expands on a collaboration between engineering education researchers and machine learning researchers to automate the analysis of written responses to conceptually challenging questions in statics and dynamics courses (Authors, 2022). Using the Concept Warehouse (Koretsky et al., 2014), written justifications of ConcepTests (CTs) were gathered from statics and dynamics courses in a diverse set of two- and four-year institutions. Written justifications for CTs have been used to support active learning pedagogies which makes them important to investigate how students put together their problem-solving narratives of understanding. However, despite the large benefit that analysis of student written responses may provide to instructors and researchers, manual review of responses is cumbersome, limits analysis, and can be prone to human bias. In efforts to improve the analysis of student written responses, machine learning has been used in various educational contexts to analyze short and long texts (Burstein et al., 2020; Burstein et al., 2021). Natural Language Processing (NLP) uses transformer-based machine learning models (Brown et al., 2020; Raffel et al., 2019) which can be used through fine-tuning or in-context learning methods. NLP can be used to train algorithms that can automate the coding of written responses. Only a few studies for educational applications have leveraged transformer-based machine learning models further prompting an investigation into its use in STEM education. However, work in NLP has been criticized for heightening the possibility to perpetuate and even amplify harmful stereotypes and implicit biases (Chang et al., 2019; Mayfield et al., 2019). In this study, we detail the aim to use NLP for linguistic justice. Using methods like text summary, topic modeling, and text classification, we identify key aspects of student narratives of understanding in written responses to mechanics and statics CTs. Through this process, we seek to use machine learning to identify different ways students talk about a problem and their understanding at any point in their narrative formation process. Thus, we hope to help reduce human bias in the classroom and through technology by giving instructors and researchers a diverse set of narratives that include insight into their students’ histories, identities, and understanding. These can then be used towards connecting technological knowledge to students’ everyday lives.
more »
« less
Sociolinguistic prompts in the 21 st century: Uniting past approaches and current directions
Abstract As technology (particularly smartphone and computer technology) has advanced, sociolinguistic methodology has likewise adapted to include remote data collection. Remote methods range from approximating the traditional sociolinguistic interview via synchronous video conferencing to developing new methods for asynchronous self‐recording (Boyd et al., 2015; Leeman et al., 2020). In this paper, we take a close look at the question prompts sent to participants in an asynchronous, remote self‐recording project (“MI Diaries”). We discuss how some of the techniques initially developed for obtaining a range of styles in a traditional in‐person sociolinguistic interview can be fruitfully adapted to a remote context. Of this range of styles, we give particular focus toNarratives of Personal Experience(Labov & Waletzky, 1967), and provide an analysis of how the theme, style, and development of prompts can encourage narratives from participants. We end with a short discussion of prompts that have successfully elicited other speech styles, and prompts that are especially fruitful with child participants.
more »
« less
- Award ID(s):
- 2119975
- PAR ID:
- 10414755
- Publisher / Repository:
- Wiley-Blackwell
- Date Published:
- Journal Name:
- Language and Linguistics Compass
- Volume:
- 17
- Issue:
- 3
- ISSN:
- 1749-818X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract BackgroundWhile laboratory practices have traditionally been conducted in-person, online asynchronous laboratory learning has been growing in popularity due to increased enrollments and the recent pandemic, creating opportunities for accessibility. In remote asynchronous learning environments, students have more autonomy to choose how they participate with other students in their laboratory classes. Communities of practice and self-efficacy may provide insights into why students are making their participation choices and how they are interacting with peers in asynchronous physics laboratory courses. ResultsIn this mixed methods, explanatory sequential study, students in an introductory physics remote asynchronous laboratory (N = 272) were surveyed about their social learning perceptions and their physics laboratory self-efficacy. Three groups of students were identified based upon their self-reported participation level of communication with peers in asynchronous courses: (1)contributors, who communicated with peers via instant messaging software and posted comments; (2)lurkers, who read discussions on instant messaging software without posting comments; and (3)outsiders, who neither read nor posted comments to peer discussions. Analysis of variance with post hoc Tukey tests showed significant differences in social learning perceptions among contributors, lurkers, and outsiders, with a large effect size, and differences between contributing and lurking students’ self-efficacy, with a small effect size. Qualitative findings from open-ended survey responses indicated contributors felt the structure of the learning environment, or their feeling of connectedness with other students, facilitated their desire to contribute. Many lurkers felt they could get what they needed through vicarious learning, and many expressed their lack of confidence to post relevant, accurate comments. Outsiders felt they did not have to, did not want to, or could not connect with other students. ConclusionsWhile the classroom laboratory traditionally requires all students to participate in the learning process through active socialization with other students, students in a remote asynchronous laboratory may still gain the benefits of participation through lurking. Instructors may consider lurking in an online or remote science laboratory as a legitimate form of participation and engagement.more » « less
-
null (Ed.)The outbreak of COVID-19 and sudden transition to remote learning brought many changes and challenges to higher education campuses across the nation. This paper evaluates the impact of the transition to remote learning on the engineering-related social cognitions of self-efficacy (belief in one’s abilities to successfully accomplish tasks in engineering) and outcome expectations (beliefs about the consequences of performing engineering behaviors). These social cognitions can be attributed to important academic and career outcomes, such as the development of STEM interests and goals (Lent et al., 2019) and may be especially important in the success of women in non-traditional fields such as engineering. As an extension to a NSF RIEF (Research Initiation in Engineering Formation) study evaluating engineering social cognitions, students in 8 engineering classes were surveyed at the beginning of Spring 2020 semester (N=224), shortly after the transition to remote learning (N = 190), and at the end of the semester (N=101). The classes surveyed included a common early engineering class at the sophomore level (Engineering Statics) and required junior level courses in different departments. The students were surveyed using reliable and validated instruments to measure engineering self-efficacy (Lent et al. 2005, Frantz et al. 2011), engineering outcome expectations (Lent et al. 2003, Lee et al. 2018), and engineering persistence intentions (Lent et al. 2003). The results show a gradual increase in the mean scores on the engineering self-efficacy and outcome expectation measures through the semester. Two tailed t-tests of matched participants showed no significance when comparing the data between the beginning and mid-semester surveys, as well as the mid-semester and end surveys. However, significance was found in the two engineering self-efficacy measures between the beginning and end of semester surveys. Results are compared across courses at different levels and across gender. Results indicate that despite the sudden change in instructional mode, students’ perceptions of engineering self-efficacy and outcome expectations showed a slight increase or no change.more » « less
-
Abstract BackgroundWhile studies examining graduate engineering student attrition have grown more prevalent, there is an incomplete understanding of the plight faced by persisting students. As mental health and well‐being crises emerge in graduate student populations, it is important to understand how students conceptualize their well‐being in relation to their decisions to persist or depart from their program. Purpose/HypothesisThe purpose of this article is to characterize the well‐being of students who endured overwhelming difficulties in their doctoral engineering programs. The PERMA‐V framework of well‐being theory proposes that well‐being is a multifaceted construct comprised ofpositive emotion,engagement,relationships,meaning,accomplishment, andvitality. Design/MethodData were collected in a mixed‐methods research design through two rounds of qualitative semistructured interviews and a survey‐based PERMA‐V profiling instrument. Interview data were analyzed thematically using the PERMA‐V framework as an a priori coding schema and narrative configuration and analysis. ResultsThe narratives demonstrated the interconnectedness between the different facets of well‐being and how they were influenced by various experiences the participants encountered. The participants in this study faced prolonged and extreme adversity. By understanding how the multiple dimensions of well‐being theory manifested in their narratives, we better understood and interpreted how these participants chose to persist.more » « less
-
Ali, Raian; Lugrin, Birgit; Charles, Fred (Ed.)The widespread adoption of intelligent voice assistants (IVAs), like Amazon’s Alexa or Google’s Assistant, presents new opportunities for designers of persuasive technologies to explore how to support people’s behavior change goals and habits with voice technology. In this work, we explore how to use planning prompts, a technique from behavior science to make specific and effective plans, with IVAs. We design and conduct usability testing (N = 13) on a voice app called Planning Habit that encourages users to formulate daily plans out loud. We identify strategies that make it possible to successfully adapt planning prompts to voice format. We then conduct a week-long online deployment (N = 40) of the voice app in the context of daily productivity. Overall, we find that traditional forms of planning prompts can be adapted to and enhanced by IVA technology.more » « less
