skip to main content


Title: Better Organization or a Source of Distraction?: Introducing Digital Peer Feedback to a Paper-Based Classroom
Peer feedback is a central activity for project-based design education. The prevalence of devices carried by students and the emergence of novel peer feedback systems enables the possibility of collecting and sharing feedback immediately between students during class. However, pen and paper is thought to be more familiar, less distracting for students, and easier for instructors to implement and manage. To evaluate the efficacy of in-class digital feedback systems, we conducted a within-subjects study with 73 students during two weeks of a game design course. After short student presentations, while instructors provided verbal feedback, peers provided feedback either on paper or through a device. The study found that both methods yielded comments of similar quality and quantity, but the digital approach provided additional ways for students to participate and required less effort from the instructors. While both methods produced similar behaviors, students held inaccurate perceptions about their behavior with each method. We discuss design implications for technologies to support in-class feedback exchange.  more » « less
Award ID(s):
1821590
NSF-PAR ID:
10117959
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
Page Range / eLocation ID:
5545 to 5555
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Evidence has shown that facilitating student-centered learning (SCL) in STEM classrooms enhances student learning and satisfaction [1]–[3]. However, despite increased support from educational and government bodies to incorporate SCL practices [1], minimal changes have been made in undergraduate STEM curriculum [4]. Faculty often teach as they were taught, relying heavily on traditional lecture-based teaching to disseminate knowledge [4]. Though some faculty express the desire to improve their teaching strategies, they feel limited by a lack of time, training, and incentives [4], [5]. To maximize student learning while minimizing instructor effort to change content, courses can be designed to incorporate simpler, less time-consuming SCL strategies that still have a positive impact on student experience. In this paper, we present one example of utilizing a variety of simple SCL strategies throughout the design and implementation of a 4-week long module. This module focused on introductory tissue engineering concepts and was designed to help students learn foundational knowledge within the field as well as develop critical technical skills. Further, the module sought to develop important professional skills such as problem-solving, teamwork, and communication. During module design and implementation, evidence-based SCL teaching strategies were applied to ensure students developed important knowledge and skills within the short timeframe. Lectures featured discussion-based active learning exercises to encourage student engagement and peer collaboration [6]–[8]. The module was designed using a situated perspective, acknowledging that knowing is inseparable from doing [9], and therefore each week, the material taught in the two lecture sessions was directly applied to that week’s lab to reinforce students’ conceptual knowledge through hands-on activities and experimental outcomes. Additionally, the majority of assignments served as formative assessments to motivate student performance while providing instructors with feedback to identify misconceptions and make real-time module improvements [10]–[12]. Students anonymously responded to pre- and post-module surveys, which focused on topics such as student motivation for enrolling in the module, module expectations, and prior experience. Students were also surveyed for student satisfaction, learning gains, and graduate student teaching team (GSTT) performance. Data suggests a high level of student satisfaction, as most students’ expectations were met, and often exceeded. Students reported developing a deeper understanding of the field of tissue engineering and learning many of the targeted basic lab skills. In addition to hands-on skills, students gained confidence to participate in research and an appreciation for interacting with and learning from peers. Finally, responses with respect to GSTT performance indicated a perceived emphasis on a learner-centered and knowledge/community-centered approaches over assessment-centeredness [13]. Overall, student feedback indicated that SCL teaching strategies can enhance student learning outcomes and experience, even over the short timeframe of this module. Student recommendations for module improvement focused primarily on modifying the lecture content and laboratory component of the module, and not on changing the teaching strategies employed. The success of this module exemplifies how instructors can implement similar strategies to increase student engagement and encourage in-depth discussions without drastically increasing instructor effort to re-format course content. Introduction. 
    more » « less
  2. null (Ed.)
    Laboratory notebooks perform important roles in the engineering disciplines. They at once record an engineer’s work, serve as an important reference for future reports and/or articles, and perform as a kind of journal that enables questioning presuppositions, considering new approaches, and generating new ideas. Given the importance of notebooks, there is surprisingly little scholarship on how to teach their use. Stanley and Lewandowski (2016) surveyed students in undergraduate laboratory courses and evaluated how their notebooks were being used. They found that “few [students] … thought that their lab classes successfully taught them the benefit of maintaining a lab notebook.” Moreover, the authors’ later survey of the literature and of college faculty led them to conclude that in undergraduate lab courses “little formal attention has been paid to addressing what is considered ‘best practice’ for scientific documentation …[or] how researchers come to learn these practices” (Stanley and Lewandowski, 2018). At XXX University, two courses, Interfacing the Digital Domain with the Analog World (AEP 2640) and Engineering Communications (ENGRC 2640) are taught in conjunction. In AEP 2640, students use a computer to control equipment and acquire measurements in an engineering design and experimentation laboratory. Laboratory activities such as the development of a computer interface for an oscilloscope, a set of motors, and a photodiode culminate in the realization of an automated laser scanning microscope system. In ENGRC 2640, students receive instruction and feedback on their lab notebook entries and, in turn, use those notebooks as a resource for preparing a Progress Report and an Instrument Design Report. The instructors encourage peer review in order to facilitate improvement of students’ skills in the art of notebook use while allowing them to develop these skills and personal style through trial and error during the research. The primary learning objectives are: 1) to enable students to engage in real laboratory research; and 2) to develop proficiency with select genres associated with that research. The educational research objectives are: 1) to study students’ developing proficiency in order to generate best practices for teaching and learning scientific documentation; and 2) to better understand the contribution of scientific documentation to the teaching and learning of authentic research. This study is a work-in-progress. We will present the study design. That design involves, first, developing a self-efficacy scale for both conducting laboratory research and performing those genres associated with that research. Self-efficacy or a “person’s awareness of their ability to accomplish a goal” (Kolar et. al, 2013) has proven to be a powerful predictor of achievement. Our intent is to track learner agency. Second, the design also involves conducting a content analysis of students’ laboratory notebooks and reports. Content analysis is a methodology that encourages inferencing "across distinct domains, from particulars of one kind to particulars of another kind" (Krippendorff,, 2019). Our intent is to learn about students' mastery of the engineering design and experimentation process through analyzing their lab notebooks. We will present the results of a preliminary content analysis of a select sample of those notebooks and genres. 
    more » « less
  3. This paper reports on the initial implementation of a two student “tiger team” in an engineering capstone design class. A tiger team is a small group of individuals that covers a range of expertise and is assigned when challenges arise that helps address the root issues causing the challenge. The term was coined in the 1960’s in the Cold War; tiger teams are used in industry, government, and military organizations. While tiger teams in these situations are usually formed around an issue then disbanded, in the capstone class the tiger team was formed for the duration of the two semester long class; details on formation and the larger context and organization of the class are discussed in the paper. The rationale for the tiger team was the observation over many years of a capstone class that as projects are functionally decomposed and subsystems assigned to individual students, a not insignificant fraction of students become “stuck” at some point in time – the concept of “stuckness” is further derived in the full paper. The result is that if delays accumulate on critical parts of the project, teams often struggle to get the project back on track and end up with a cascading series of missed deadlines. The rationale for the tiger team is to help teams identify when parts of the project are getting behind schedule and to have additional, short-term help available. In the initial implementation described here, the tiger team was two students—one from electrical and one from computer engineering—who volunteered for the position and were confirmed in that role by the other students in the class. Initial data shows that during the problem identification phase of the project the tiger team attended team meetings, helped evaluation project milestone reviews, worked to solve individual and team issues, and regularly met with the faculty. Early in the semester the two tiger team students described their role as unclear and worried their technical exposure would be limited. Later, as the teams developed technical representations, the tiger team provided independent feedback and addressed multiple technical challenges. Finally, as teams started to build technical prototypes the tiger team role again shifted to helping individuals with specific aspects of their project; this role continued throughout the remainder of the year-long course. This in-depth case-study of the experience of implementing a tiger team draws on observations from students, faculty, the tiger team members, and an external ethnographer. This work may help other capstone instructors who may be considering similar interventions. 
    more » « less
  4. This Complete Evidence-based Practice paper will describe how three different public urban research universities designed, executed, and iterated Summer Bridge programming for a subset of incoming first-year engineering students over the course of three consecutive years. There were commonalities between each institution’s Summer Bridge, as well as unique aspects catering to the specific needs and structures of each institution. Both these commonalities and unique aspects will be discussed, in addition to the processes of iteration and improvement, target student populations, and reported student outcomes. Finally, recommendations for other institutions seeking to launch or refine similar programming will be shared. Summer Bridge programming at each of the three institutions shared certain communalities. Mostly notably, each of the three institutions developed its Summer Bridge as an additional way to provide support for students receiving an NSF S-STEM scholarship. The purpose of each Summer Bridge was to build community among these students, prepare them for the academic rigor of first-year engineering curriculum, and edify their STEM identity and sense of belonging. Each Summer Bridge was a 3-5 day experience held in the week immediately prior to the start of the Fall semester. In addition to these communalities, each Summer Bridge also had its own unique features. At the first institution, Summer Bridge is focused on increasing college readiness through the transition from summer break into impending coursework. This institution’s Summer Bridge includes STEM special-interest presentations (such as biomedical or electrical engineering) and other development activities (such as communication and growth mindset workshops). Additionally, this institution’s Summer Bridge continues into the fall semester via a 1-credit hour First Year Seminar class, which builds and reinforces student networking and community beyond the summer experience. At the second institution, all students receiving the NSF S-STEM scholarship (not only those who are first-year students) participate in Summer Bridge. This means that S-STEM scholars at this institution participate in Summer Bridge multiple years in a row. Relatedly, after the first year, Summer Bridge transitioned to a student-led and student-delivered program, affording sophomore and junior students leadership opportunities, which not only serve as marketable experience after graduation, but also further builds their sense of STEM identity and belonging. At the third institution, a special focus was given to building community. This was achieved through several means. First, each day of Summer Bridge included a unique team-oriented design challenge where students got to work together and know each other within an engineering context, also reinforcing their STEM identities. Second, students at this institution’s Summer Bridge met their future instructors in an informal, conversational, lunch setting; many students reported this was one of their favorite aspects of Summer Bridge. Finally, Summer Bridge facilitated a first connect between incoming first-year students and their peer mentors (sophomore and junior students also receiving the NSF S-STEM scholarship), with whom they would meet regularly throughout the following fall and spring semesters. Each of the three institutions employed processes of iteration and improvement for their Summer Bridge programming over the course of two or three consecutive years. Through each version and iteration of Summer Bridge, positive student outcomes are demonstrated, including direct student feedback indicating built community among students and the perception that their time spent during Summer Bridge was valuable. Based on the experiences of these three institutions, as well as research on other institutions’ Summer Bridge programming, recommendations for those seeking to launch or refine similar Summer Bridge programming will also be shared. 
    more » « less
  5. The drive to encourage young people to pursue degrees and careers in engineering has led to an increase in student populations in engineering programs. For some institutions, such as large public research institutions, this has led to large class sizes for courses that are commonly taken across multiple programs. While this decision is reasonable from an operational and resource management perspective, research on large classes have shown that students suffer decreased engagement, motivation and achievement. Instructors, on the other hand, report having difficulty establishing rapport with their students and a growing inability to monitor students’ learning gains and provide quality individualized feedback. To address these issues, our project draws from Lattuca and Stark’s Academic Plan model, which incorporates a thorough consideration of factors influencing curricular activities that can be applied at the course, program, and institutional levels, and assumes that instructors are key actors in curriculum development and revision. We aim to revitalize feedback loops to help instructors and departments continuously improve. Recognizing that we must understand both individual and systems level perspectives, we prioritize regular engagement between faculty and institutional support structures to collaboratively identify problems and systematically establish continuous improvement. In the first phase of this NSF IUSE Institutional Transformation project, we focus on specifically prompting and studying the experiences of 8 instructors of foundational engineering courses usually taught in large class sizes across 4 different departments at a large public research institution. We collected qualitative data (semi-structured interviews, reflective journals, course-related documents) and quantitative data (student surveys and institution-provided transcript data) to answer research questions (e.g., what data do faculty teaching large foundational undergraduate engineering courses identify as being useful so that they may enhance students’ experiences and outcomes within the classes that they teach and across students’ multiple large classes?) at the intersection of learning analytics and faculty change. The data was used as a baseline to further refine data collection protocols, identify data that faculty consider meaningful and useful for managing large foundational engineering courses, and consider ways of productively leveraging institutional data to improve the learning experience in these courses. Data collection for the first phase is ongoing and will continue through the Spring 2018 semester. Findings for this paper will include high-level insights from Fall interviews with instructors as well as data visualizations created from the population-level data characterizing student performance in the foundational courses within the context of pre-college characteristics (e.g., SAT scores) and/or other academic outcomes (e.g., major switching within or out of engineer, degree attainment). 
    more » « less