Integrating engineering into the K‐12 science curriculum continues to be a focus in national reform efforts in science education. Although there is an increasing interest in research in and practice of integrating engineering in K‐12 science education, to date only a few studies have focused on the development of an assessment tool to measure students’ understanding of engineering design. Most of the existing measures focus only on knowledge and understanding of engineering design concepts using multiple‐choice items with the exception of the mixed‐format Engineering Concept Assessment (ECA). Also, advanced measurement models are lacking application in the testing of such mixed‐format assessments in science education. This study applied many‐faceted Rasch measurement to the modified ECA for eighth‐grade (ECA/M8) and a newly constructed rubric applied by five judges across 497 eighth‐grade students’ responses after experiencing an integrated learning unit on the engineering design process. The results supported the fit of the items and rubric rating scales to the Rasch specifications. Recommendations are made for item wording, and further reliability and validity testing of the ECA/M8, and use of the ECA/M8 in science education and research.
- Award ID(s):
- 1625215
- NSF-PAR ID:
- 10229736
- Date Published:
- Journal Name:
- American Journal of Evaluation
- Volume:
- 41
- Issue:
- 1
- ISSN:
- 1098-2140
- Page Range / eLocation ID:
- 145 to 155
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
Wyoming recently mandated that computer science instruction be provided in K-12 schools by 2022, and there is an urgent need for designing instruction that can integrate computer science into the teaching of other subjects. This project assembles a network improvement community comprised of partners from the University of Wyoming, community colleges, Wyoming school districts, the Wyoming Library System, the Wyoming Department of Education, and local software development firms. The community meets once monthly over the duration of the project to collaborate stakeholder agendas for meeting the project goals. The community enlists K-8 teachers from across the state to experience professional development and collaborate on integrating computer science into their instruction of STEM and social science topics. The project is producing units for teachers, who are implementing these units with support from master teachers and educational scholars. The community serves as a forum for teachers to debrief and learn from each other about ways to improve their instruction and design of the curricular units. Libraries in the state system act as partners for dissemination to rural areas of the innovative instructional approaches. WySLICE prepares 150 K-8 teachers and state librarians from all disciplines to integrate computer science into their teaching. The project is reaching almost half of all K-8 students in Wyoming. The research questions address how teachers use modeling practices as supports for student understanding of algorithms and coding in a variety of ways. The curricula involve cybersecurity as well as other topics relevant to measurement in mathematics and social studies topics that involve social concerns like voting. Data sources include teacher lesson plans and recordings of their instructional implementation, scoring of each of these according to a rubric, meeting notes of monthly meetings, and results from pre-post student assessments. The evaluation focuses on the meeting of project goals and the quality of the management of the network improvement community. This project is jointly funded by CS for All and the Established Program to Stimulate Competitive Research (EPSCoR). This work is supported by the National Science Foundation under DRL Grant #1923542 "CS For All:RPP - Booting Up Computer Science in Wyoming."more » « less
-
more » « less
Abstract High‐throughput phenotyping (HTP) with unoccupied aerial systems (UAS), consisting of unoccupied aerial vehicles (UAV; or drones) and sensor(s), is an increasingly promising tool for plant breeders and researchers. Enthusiasm and opportunities from this technology for plant breeding are similar to the emergence of genomic tools ∼30 years ago, and genomic selection more recently. Unlike genomic tools, HTP provides a variety of strategies in implementation and utilization that generate big data on the dynamic nature of plant growth formed by temporal interactions between growth and environment. This review lays out strategies deployed across four major staple crop species: cotton (
Gossypium hirsutum L.), maize (Zea mays L.), soybean (Glycine max L.), and wheat (Triticum aestivum L.). Each crop highlighted in this review demonstrates how UAS‐collected data are employed to automate and improve estimation or prediction of objective phenotypic traits. Each crop section includes four major topics: (a) phenotyping of routine traits, (b) phenotyping of previously infeasible traits, (c) sample cases of UAS application in breeding, and (d) implementation of phenotypic and phenomic prediction and selection. While phenotyping of routine agronomic and productivity traits brings advantages in time and resource optimization, the most potentially beneficial application of UAS data is in collecting traits that were previously difficult or impossible to quantify, improving selection efficiency of important phenotypes. In brief, UAS sensor technology can be used for measuring abiotic stress, biotic stress, crop growth and development, as well as productivity. These applications and the potential implementation of machine learning strategies allow for improved prediction, selection, and efficiency within breeding programs, making UAS HTP a potentially indispensable asset. -
A Mavragani (Ed.)more » « less
Background Posttraumatic stress disorder (PTSD) is a serious public health concern. However, individuals with PTSD often do not have access to adequate treatment. A conversational agent (CA) can help to bridge the treatment gap by providing interactive and timely interventions at scale. Toward this goal, we have developed PTSDialogue—a CA to support the self-management of individuals living with PTSD. PTSDialogue is designed to be highly interactive (eg, brief questions, ability to specify preferences, and quick turn-taking) and supports social presence to promote user engagement and sustain adherence. It includes a range of support features, including psychoeducation, assessment tools, and several symptom management tools.
Objective This paper focuses on the preliminary evaluation of PTSDialogue from clinical experts. Given that PTSDialogue focuses on a vulnerable population, it is critical to establish its usability and acceptance with clinical experts before deployment. Expert feedback is also important to ensure user safety and effective risk management in CAs aiming to support individuals living with PTSD.
Methods We conducted remote, one-on-one, semistructured interviews with clinical experts (N=10) to gather insight into the use of CAs. All participants have completed their doctoral degrees and have prior experience in PTSD care. The web-based PTSDialogue prototype was then shared with the participant so that they could interact with different functionalities and features. We encouraged them to “think aloud” as they interacted with the prototype. Participants also shared their screens throughout the interaction session. A semistructured interview script was also used to gather insights and feedback from the participants. The sample size is consistent with that of prior works. We analyzed interview data using a qualitative interpretivist approach resulting in a bottom-up thematic analysis.
Results Our data establish the feasibility and acceptance of PTSDialogue, a supportive tool for individuals with PTSD. Most participants agreed that PTSDialogue could be useful for supporting self-management of individuals with PTSD. We have also assessed how features, functionalities, and interactions in PTSDialogue can support different self-management needs and strategies for this population. These data were then used to identify design requirements and guidelines for a CA aiming to support individuals with PTSD. Experts specifically noted the importance of empathetic and tailored CA interactions for effective PTSD self-management. They also suggested steps to ensure safe and engaging interactions with PTSDialogue.
Conclusions Based on interviews with experts, we have provided design recommendations for future CAs aiming to support vulnerable populations. The study suggests that well-designed CAs have the potential to reshape effective intervention delivery and help address the treatment gap in mental health.
-
This paper examines how 17 secondary mathematics teacher candidates (TCs) in four university teacher preparation programs implemented technology in their classrooms to teach for conceptual understanding in online, hybrid, and face to face classes during COVID-19. Using the Professional Development: Research, Implementation, and Evaluation (PrimeD) framework, TCs, classroom mentor teachers, field experience supervisors, and university faculty formed a Networked Improvement Community (NIC) to discuss a commonly agreed upon problem of practice and a change idea to implement in the classroom. Through Plan-Do-Study-Act cycles, participants documented their improvement efforts and refinements to the change idea and then reported back to the NIC at the subsequent monthly meeting. The Technology Pedagogical Content Knowledge framework (TPACK) and the TPACK levels rubric were used to examine how teacher candidates implemented technology for Mathematics conceptual understanding. The Mathematics Classroom Observation Protocol for Practices (MCOP2) was used to further examine how effective mathematics teaching practices (e.g., student engagement) were implemented by TCs. MCOP2 results indicated that TCs increased their use of effective mathematics teaching practices. However, growth in TPACK was not significant. A relationship between TPACK and MCOP2 was not evident, indicating a potential need for explicit focus on using technology for mathematics conceptual understanding.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1177/1098214019878784
