Research based assessments have a productive and storied history in PER. While useful for conducting research on student learning, their utility is limited for instructors interested in improving their own courses. We have developed a new assessment design process that leverages three-dimensional learning, evidence-centered design, and self-regulated learning to deliver actionable feedback to instructors about supporting their students' learning. We are using this approach to design the Thermal and Statistical Physics Assessment (TaSPA), which also allows instructors to choose learning goals that align with their teaching. Perhaps more importantly, this system will be completely automated when it is completed, making the assessment scalable with minimal burden on instructors and researchers. This work represents an advancement in how we assess physics learning at a large scale and how the PER community can better support physics instructors and students.
more »
« less
gPortfolios: A pragmatic approach to online asynchronous assignments
Purpose: We gathered examples from our extended collaboration to move educators move online while avoiding synchronous meetings. “gPortfolios” are public (to the class) pages where students write responses to carefully constructed engagement routines. Students then discuss their work with instructors and peers in threaded comments. gPortfolios usually include engagement reflections, formative self-assessments, and automated quizzes. These assessments support and document learning while avoiding instructor “burnout” from grading. gPortfolios can be implemented using Google Docs and Forms or any learning management system.
Methodology. We report practical insights gained from design-based implementation research. This research explored the late Randi Engle’s principles for productive disciplinary engagement and expansive framing. Engle used current theories of learning to foster student discussions that were both authentic to the academic discipline at hand and productive for learning. This research also used new approaches to assessment to support Engle’s principles. This resulted in a comprehensive approach to online instruction and assessment that is effective and efficient for both students and teachers.
Findings. Our approach “frames” (i.e., contextualizes) online engagement using each learners’ own experiences, perspectives, and goals. Writing this revealed how this was different in different courses. Secondary biology students framed each assignment independently. Secondary English and history students framed assignments as elements of a personalized capstone presentation; the history students further used a self-selected “historical theme.” Graduate students framed each assignment in an educational assessment course using a real or imagined curricular aim and context.
Originality. Engle’s ideas have yet to be widely taken up in online education.
more »
« less
- Award ID(s):
- 1915498
- NSF-PAR ID:
- 10299958
- Date Published:
- Journal Name:
- Information and learning science
- Volume:
- 121
- Issue:
- 5/6
- ISSN:
- 2398-5348
- Page Range / eLocation ID:
- 273-283
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
An increasingly global environment expects graduating Engineering students to perform, live and work across cultures. Most intercultural competence research and associated global engineering education is focused on developing the global engineering skill set through long-term travel experiences such as study abroad programs. These programs can be expensive from both a time and money standpoint, limiting the participation to more privileged members of a community, and are not scalable to support broader participation. This work-in-progress addresses this research gap by focusing on the development of the students’ global learner mindset without requiring extensive travel. The project will investigate four different global engagement interventions, including the use of engineering case studies, the intentional formation of multi-national student teams, a Collaborative Online International Learning (COIL) research project, and a community engaged project within a short course. These interventions can be used to develop a holistic global learner mindset and global engineering education approach to foster global competence in undergraduate engineering students. The four global engagement interventions will be grounded in the global engineering competency (GEC) theoretical framework and assessed for their ability to foster a global learner mindset in engineering students. A mixed-methods approach will be used to assess students’ global learner mindset and skill set. This research will use the Global Engagement Survey (GES), the Global Engineering Competency Scale (GECS) and specific questions developed by the researchers to evaluate improvements in the participating students’ global engineering skill set and answer specific research questions including: 1) To what extent can global competence be developed in engineering students through the use of the proposed global engagement interventions; and 2) what are the relative strengths of each of the proposed global engagement interventions in developing global engineering competence? Combined, these research measures will provide both an accurate picture of how each global engagement intervention impacts the formation of a global learner mindset in engineering education, and also its associated ability to develop and/or improve global engineering skills. The outcomes of this study will generate valuable knowledge to understand how each global engagement intervention impacts the formation of global engineering competence. In this work-in-progress study, the authors discuss the four global engagement interventions with specific learning objectives that have been mapped to the overall student outcomes for the project. These objectives have also been mapped to the GES and GECS instruments. Finally the faculty members have developed qualitative tools to augment the GES and GECS to identify the global engineering skill sets each intervention is generating. This paper lays the foundation before implementing the interventions and performing their associated assessments over the several subsequent semesters.more » « less
-
An increasingly global environment expects graduating Engineering students to perform, live and work across cultures. Most intercultural competence research and associated global engineering education is focused on developing the global engineering skill set through long-term travel experiences such as study abroad programs. These programs can be expensive from both a time and money standpoint, limiting the participation to more privileged members of a community, and are not scalable to support broader participation. This work-in-progress addresses this research gap by focusing on the development of the students’ global learner mindset without requiring extensive travel. The project will investigate four different global engagement interventions, including the use of engineering case studies, the intentional formation of multi-national student teams, a Collaborative Online International Learning (COIL) research project, and a community engaged project within a short course. These interventions can be used to develop a holistic global learner mindset and global engineering education approach to foster global competence in undergraduate engineering students. The four global engagement interventions will be grounded in the global engineering competency (GEC) theoretical framework and assessed for their ability to foster a global learner mindset in engineering students. A mixed-methods approach will be used to assess students’ global learner mindset and skill set. This research will use the Global Engagement Survey (GES), the Global Engineering Competency Scale (GECS) and specific questions developed by the researchers to evaluate improvements in the participating students’ global engineering skill set and answer specific research questions including: 1) To what extent can global competence be developed in engineering students through the use of the proposed global engagement interventions; and 2) what are the relative strengths of each of the proposed global engagement interventions in developing global engineering competence? Combined, these research measures will provide both an accurate picture of how each global engagement intervention impacts the formation of a global learner mindset in engineering education, and also its associated ability to develop and/or improve global engineering skills. The outcomes of this study will generate valuable knowledge to understand how each global engagement intervention impacts the formation of global engineering competence. In this work-in-progress study, the authors discuss the four global engagement interventions with specific learning objectives that have been mapped to the overall student outcomes for the project. These objectives have also been mapped to the GES and GECS instruments. Finally the faculty members have developed qualitative tools to augment the GES and GECS to identify the global engineering skill sets each intervention is generating. This paper lays the foundation before implementing the interventions and performing their associated assessments over the several subsequent semesters.more » « less
-
As K-12 engineering education becomes more ubiquitous in the U.S, increased attention has been paid to preparing the heterogeneous group of in-service teachers who have taken on the challenge of teaching engineering. Standards have emerged for professional development along with research on teacher learning in engineering that call for teachers to facilitate and support engineering learning environments. Given that many teachers may not have experienced engineering practice calls have been made to engage teaches K-12 teachers in the “doing” of engineering as part of their preparation. However, there is a need for research studying more specific nature of the “doing” and the instructional implications for engaging teachers in “doing” engineering. In general, to date, limited time and constrained resources necessitate that many professional development programs for K-12 teachers to engage participants in the same engineering activities they will enact with their students. While this approach supports teachers’ familiarity with curriculum and ability to anticipate students’ ideas, there is reason to believe that these experiences may not be authentic enough to support teachers in developing a rich understanding of the “doing” of engineering. K-12 teachers are often familiar with the materials and curricular solutions, given their experiences as adults, which means that engaging in the same tasks as their students may not be challenging enough to develop their understandings about engineering. This can then be consequential for their pedagogy: In our prior work, we found that teachers’ linear conceptions of the engineering design process can limit them from recognizing and supporting student engagement in productive design practices. Research on the development of engineering design practices with adults in undergraduate and professional engineering settings has shown significant differences in how adults approach and understand problems. Therefore, we conjectured that engaging teachers in more rigorous engineering challenges designed for adult engineering novices would more readily support their developing rich understandings of the ways in which professional engineers move through the design process. We term this approach meaningful engineering for teachers, and it is informed by work in science education that highlights the importance of learning environments creating a need for learners to develop and engage in disciplinary practices. We explored this approach to teachers’ professional learning experiences in doing engineering in an online graduate program for in-service teachers in engineering education at Tufts University entitled the Teacher Engineering Education Program (teep.tufts.edu). In this exploratory study, we asked: 1. How did teachers respond to engaging in meaningful engineering for teachers in the TEEP program? 2. What did teachers identify as important things they learned about engineering content and pedagogy? This paper focuses on one theme that emerged from teachers’ reflections. Our analysis found that teachers reported that meaningful engineering supported their development of epistemic empathy (“the act of understanding and appreciating someone's cognitive and emotional experience within an epistemic activity”) as a result of their own affective experiences in doing engineering that required significant iteration as well as using novel robotic materials. We consider how epistemic empathy may be an important aspect of teacher learning in K-12 engineering education and the potential implications for designing engineering teacher education.more » « less
-
Motivation: This is a complete paper. There was a sudden shift from traditional learning to online learning in Spring 2020 with the outbreak of COVID-19. Although online learning is not a new topic of discussion, universities, faculty, and students were not prepared for this sudden change in learning. According to a recent article in ‘The Chronicle of Higher Education, “even under the best of circumstances, virtual learning requires a different, carefully crafted approach to engagement”. The Design Thinking course under study is a required freshmen level course offered in a Mid-western University. The Design Thinking course is offered in a flipped format where all the content to be learned is given to students beforehand and the in-class session is used for active discussions and hands-on learning related to the content provided at the small group level. The final learning objective of the course is a group project where student groups are expected to come up with functional prototypes to solve a real-world problem following the Design Thinking process. There were eighteen sections of the Design Thinking course offered in Spring 2020, and with the outbreak of COVID-19, a few instructors decided to offer synchronous online classes (where instructors were present online during class time and provided orientation and guidance just like a normal class) and a few others decided to offer asynchronous online classes (where orientation from the instructor was delivered asynchronous and the instructor was online during officially scheduled class time but interactions were more like office hours). Students were required to be present synchronously at the team level during the class time in a synchronous online class. In an asynchronous online class, students could be synchronous at the team level to complete their assignment any time prior to the deadline such that they could work during class time but they were not required to work at that time. Through this complete paper, we are trying to understand student learning, social presence and learner satisfaction with respect to different modes of instruction in a freshmen level Design Thinking course. Background: According to literature, synchronous online learning has advantages such as interaction, a classroom environment, and better course quality whereas asynchronous online learning has advantages such as self-controlled and self-directed learning. The disadvantages of synchronous online learning include the learning process, technology issues, and distraction. Social isolation, lack of interaction, and technology issue are a few disadvantages related to asynchronous online learning. Problem Being Addressed: There is a limited literature base investigating different modes of online instruction in a Design Thinking course. Through this paper, we are trying to understand and share the effectiveness of synchronous and asynchronous modes of instruction in an online Flipped Design Thinking Course. The results of the paper could also help in this time of pandemic by shedding light on the more effective way to teach highly active group-based classrooms for better student learning, social presence, and learner satisfaction. Method/Assessment: An end of semester survey was monitored in Spring 2020 to understand student experiences in synchronous and asynchronous Design Thinking course sections. The survey was sent to 720 students enrolled in the course in Spring 2020 and 324 students responded to the survey. Learning was measured using the survey instrument developed by Walker (2003) and the social presence and learner satisfaction was measured by the survey modified by Richardson and Swan (2003). Likert scale was used to measure survey responses. Anticipated Results: Data would be analyzed and the paper would be completed by draft paper submission. As the course under study is a flipped and active course with a significant component of group work, the anticipated results after analysis could be that one mode of instruction has higher student learning, social presence, and learner satisfaction compared to the other.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
