More Like this

1. Examining the effect of automated assessments and feedback on students’ written science explanations

   Puntambekar, Sadhana ( July 2023 , Computer Supported Collaborative Learning Conference)

   Writing scientific explanations is a core practice in science. However, students find it difficult to write coherent scientific explanations. Additionally, teachers find it challenging to provide real-time feedback on students’ essays. In this study, we discuss how PyrEval, an NLP technology, was used to automatically assess students’ essays and provide feedback. We found that students explained more key ideas in their essays after the automated assessment and feedback. However, there were issues with the automated assessments as well as students’ understanding of the feedback and revising their essays. 

   more » « less
2. Refining a Taxonomy for Categorizing the Quality of Engineering Student Questions

   Goldberg, S ; Venters C ; Masnick, A ( July 2021 , 2021 ASEE Annual Conference and Exposition)

   The ability to identify one’s own confusion and to ask a question that resolves it is an essential metacognitive skill that supports self-regulation (Winne, 2005). Yet, while students receive substantial training in how to answer questions, little classroom time is spent training students how to ask good questions. Past research has shown that students are able to pose more high-quality questions after being instructed in a taxonomy for classifying the quality of their questions (Marbach‐Ad & Sokolove, 2000). As pilot data collection in preparation for a larger study funded through NSF-DUE, we provided engineering statics students training in writing high-quality questions to address their own confusions. The training emphasized the value of question-asking in learning and how to categorize questions using a simple taxonomy based on prior work (Harper et al., 2003). The taxonomy specifies five question levels: 1) an unspecific question, 2) a definition question, 3) a question about how to do something, 4) a why question, and 5) a question that extends knowledge to a new circumstance. At the end of each class period during a semester-long statics course, students were prompted to write and categorize a question that they believed would help them clarify their current point of greatest confusion. Through regular practice writing and categorizing such questions, we hoped to improve students' abilities to ask questions that require higher-level thinking. We collected data from 35 students in courses at two institutions. Over the course of the semester, students had the opportunity to write and categorize twenty of their own questions. After the semester, the faculty member categorized student questions using the taxonomy to assess the appropriateness of the taxonomy and whether students used it accurately. Analysis of the pilot data indicates three issues to be addressed: 1) Student compliance in writing and categorizing their questions varied. 2) Some students had difficulty correctly coding their questions using the taxonomy. 3) Some student questions could not be clearly characterized using the taxonomy, even for faculty raters. We will address each of these issues with appropriate refinements in our next round of data collection: 1) Students may have been overwhelmed with the request to write a question after each class period. In the future, we will require students to write and categorize at least one question per week, with more frequent questions encouraged. 2) To improve student use of the taxonomy in future data collection, students will receive more practice with the taxonomy when it is introduced and more feedback on their categorization of questions during the semester. 3) We are reformulating our taxonomy to accommodate questions that may straddle more than one category, such as a question about how to extend a mathematical operation to a new situation (which could be categorized as either a level 3 or 5). We are hopeful that these changes will improve accuracy and compliance, enabling us to use the intervention as a means to promote metacognitive regulation and measure changes as a result, which is the intent of the larger scope of the project. 

   more » « less
3. Combining Student and Teacher Feedback for Effective Science Writing

   Goss, William ; Singh, Purushratha ; Puntambekar, Sadhana ; Gnesdilow, Dana ; Kim, ChanMin ; Passonneau, Rebecca J. ( July 2022 , International Society for the Learning Sciences Conference)

   This is a contribution to a Symposium This symposium will provide opportunities for discussion about how Artificial Intelligence can support ambitious learning practices in CSCL. To the extent that CSCL can be a lever for educational equitable educational change, AI needs to be able to support the kinds of practices that afford agency to students and teachers. However, AI also brings to the fore the need to consider equity and ethics. This interactive session will provide opportunities to discuss these issues in the context of the examples presented here. Our contribution is focused on two participatory design studies we conducted with 14 teachers to understand the kinds of automatic feedback they thought would support their students’ science explanation writing as well as how they would like summaries of information from students’ writing presented in a teacher’s dashboard. We also discuss how we developed our system, PyrEval, for automated writing support based on historical data and scoring from manual coding rubrics. 

   more » « less
4. Asking Clarifying Questions Based on Negative Feedback in Conversational Search

   https://doi.org/10.1145/3471158.3472232  

   Ai, Q. ; Bi, K. ; Croft, W. B. ( July 2021 , Proceedings of the 7th ACM International Conference on the Theory of Information Retrieval (ICTIR 2021))

   null (Ed.)

   Users often need to look through multiple search result pages or reformulate queries when they have complex information-seeking needs. Conversational search systems make it possible to improve user satisfaction by asking questions to clarify users’ search intents. This, however, can take significant effort to answer a series of questions starting with “what/why/how”. To quickly identify user intent and reduce effort during interactions, we propose an intent clarification task based on yes/no questions where the system needs to ask the correct question about intents within the fewest conversation turns. In this task, it is essential to use negative feedback about the previous questions in the conversation history. To this end, we propose a Maximum-Marginal-Relevance (MMR) based BERT model (MMR-BERT) to leverage negative feedback based on the MMR principle for the next clarifying question selection. Experiments on the Qulac dataset show that MMR-BERT outperforms state-of-the-art baselines significantly on the intent identification task and the selected questions also achieve significantly better performance in the associated document retrieval tasks. 

   more » « less
5. The principle of minimal power

   https://doi.org/10.1088/1361-6595/ac8b31  

   Kolobov, Vladimir I ; Golubovskii, Yuri B ( September 2022 , Plasma Sources Science and Technology)

   Abstract This article is devoted to the memory of Yuri P Raizer, who passed away in 2021. He left a noticeable trace in gas discharge physics. The principle of minimal power (the state that requires minimal power is most probable) is thoroughly used in his books. Although the fundamental laws of physics do not imply this ad hoc principle, a detailed analysis of underlying phenomena can often reveal why nature prefers this path. Raizer illustrated this principle for plasma stratification, formation of electrode spots, discharge constriction, the shape of an arc channel, etc. We argue that the nonlinearity of equations describing gas discharges can often justify the realization of a plasma state maintained at minimal electric power. This nonlinearity appears because small groups of energetic electrons often control the ionization processes. The number of these electrons depends strongly on the ratio of the electric field to gas density, E / N . Under certain conditions, the ionization rate can also depend nonlinearly on electron density due to stepwise ionization and Coulomb collisions. We use the principle of minimal power to illustrate some of Raizer’s contributions to gas discharge physics from a single point of view. We demonstrate that nonlinearity of ionization processes in gas discharges can substantiate this principle for plasma stratification. However, striations of s , p , and r types in neon could exist with minimal or no ionization enhancement. This reminds us of Raizer’s warning that applying the minimal power principle could lead to erroneous predictions, and a proper theory is required in each case to justify its use. ‘The phenomenon of striations satisfies the principle of minimal power’ – Yuri Raizer 

   more » « less