Search for: All records

The current paper discusses conversation-based assessments (CBAs) created with prompt engineering for LLMs based on Evidence-Centered Design (ECD). Conversation-based assessments provide students the opportunity to discuss a given topic with artificial agent(s). These conversations elicit evidence of students’ knowledge, skills and abilities that may not be uncovered by traditional tests. We discuss our previous method of creating such conversations with regular expressions and latent semantic analysis in an expensive methodology requiring time and various expertise. Thus, in this novel work, we created a prompt-engineered version of CBAs based on evidence-centered design that remains on the domain topic throughout the conversation as well as provides evidence of the student knowledge in a less expensive way. We present the methodology for creating these prompts, compare responses to various student speech acts between the previous version and the prompt engineered version, and discuss the evidence gleaned from the conversation and based on the prompt. Finally, limitations, conclusions and implications of this work are discussed. 

Abstract We must be able to predict and mitigate against geomagnetically induced current (GIC) effects to minimize socio‐economic impacts. This study employs the space weather modeling framework (SWMF) to model the geomagnetic response over Fennoscandia to the September 7–8, 2017 event. Of key importance to this study is the effects of spatial resolution in terms of regional forecasts and improved GIC modeling results. Therefore, we ran the model at comparatively low, medium, and high spatial resolutions. The virtual magnetometers from each model run are compared with observations from the IMAGE magnetometer network across various latitudes and over regional‐scales. The virtual magnetometer data from the SWMF are coupled with a local ground conductivity model which is used to calculate the geoelectric field and estimate GICs in a Finnish natural gas pipeline. This investigation has lead to several important results in which higher resolution yielded: (1) more realistic amplitudes and timings of GICs, (2) higher amplitude geomagnetic disturbances across latitudes, and (3) increased regional variations in terms of differences between stations. Despite this, substorms remain a significant challenge to surface magnetic field prediction from global magnetohydrodynamic modeling. For example, in the presence of multiple large substorms, the associated large‐amplitude depressions were not captured, which caused the largest model‐data deviations. The results from this work are of key importance to both modelers and space weather operators. Particularly when the goal is to obtain improved regional forecasts of geomagnetic disturbances and/or more realistic estimates of the geoelectric field.