An official website of the United States government
A challenge instructors face is developing and accurately assessing technical communication skills to ensure students can apply and transfer the skills from the academic context into the context of engineering practice. By intentionally balancing teaching transferrable communication skills relevant to engineering practice and evaluating student understanding, engineering educators can foster competence and prepare students for the expectations of their professional careers. This study addresses two questions: (1) how can chemical engineering instructors reliably and consistently assess student communication skills, and (2) are instructor expectations aligned with those of practicing engineers? The use of well-designed rubrics is important for setting clear expectations for students, providing constructive feedback, and in team taught courses, grading consistently. This study discusses how a rubric for assessing technical communication skills in senior-level chemical engineering laboratory reports was validated and demonstrated reliability across five chemical engineering instructors. Additionally, five industry partners evaluated student reports for comparison to instructor rubric scores. Expectations and perceptions of the quality of student work align between instructors and practicing engineers, but practicing engineers prioritized safety and abstract clarity, while instructors prioritized the students’ abilities to interpret results and draw conclusions.
more »
« less
Rubric Development for Technical Reports in Chemical Engineering Unit Operations Laboratory Courses
The purpose of this work was to test the inter-rater reliability (IRR) of a rubric used to grade technical reports in a senior-level chemical engineering laboratory course that has multiple instructors that grade deliverables. The rubric consisted of fifteen constructs that provided students detailed guidance on instructor expectations with respect to the report sections, formatting and technical writing aspects such as audience, context and purpose. Four student reports from previous years were scored using the rubric, and IRR was assessed using a two-way mixed, consistency, average-measures intraclass correlation (ICC) for each construct. Then, the instructors met as a group to discuss their scoring and reasoning. Multiple revisions were made to the rubric based on instructor feedback and constructs rated by ICC as poor. When fair or poor constructs were combined, the ICCs improved. In addition, the overall score construct continued to be rated as excellent, indicating that while different instructors may have variation at the individual construct level, they evaluate the overall quality of the report consistently. A key learning from this process was the importance of the instructor discussion around their reasoning for the scores and the importance of an ‘instructor orientation’ involving discussion and practice using the rubrics in the case of multiple instructors or a change in instructors. The developed rubric has the potential for broad applicability to engineering laboratory courses with technical writing components and could be adapted for alternative styles of technical writing genre.
more »
« less
- Award ID(s):
- 2120775
- PAR ID:
- 10437367
- Date Published:
- Journal Name:
- ASEE Annual Conference proceedings
- ISSN:
- 1524-4644
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Laboratory report writing instructional modules have been developed and refined using a community of practice (CoP) approach. Supported by the National Science Foundation Improving Undergraduate STEM Education initiative, researchers at three institutions have refined and reorganized a series of scaffolded laboratory writing modules based on the work of faculty and graduate students at a CoP meeting. This paper documents the process used at the CoP meeting where draft modules were made available and a model laboratory session was considered. Other published laboratory report writing resources were evaluated alongside the draft modules to determine areas of overlap and novelty and to ensure the completeness of the revised modules. The process of revising instructional modules was valuable for both the quality of the modules and the development of the community of practice. The modules are now organized into two guides, published at http://labs.wsu.edu/engineeringlab-report-writing/. An Instructor’s Guide to Engineering Lab Writing, targets instructors and provides model lab writing and data analysis learning outcomes for consideration when planning a laboratory session, as well as approaches for course organization and teaching to support lab writing outcomes. A library of lab report types and a model rubric for lab report scoring complete the instructor-oriented resource. A Student’s Guide to Engineering Lab Writing, supports students who are learning lab report writing for the first time or are advancing as technical writers. It is organized according to traditional lab report format and is aligned with the learning outcomes in the instructor modules. The content in the student-oriented modules is scaffolded to support continuous development. The modules are arranged in order of increasing cognitive difficulty, first addressing formatting conventions and arrangement, then section contents and methods of data analysis, and finally effective methods of interpretation, reasoning, and conclusion writing. This paper demonstrates the mutually reinforcing nature of collaboratively developed instructional material and the growth of a community of practice. The CoP approach to structuring a meeting was effective for gathering targeted and relevant feedback in a short period of time as well as for developing the CoP itself. The instructional modules revised at the CoP meeting were significantly improved creating a sense of ownership and inclusion by those participating in the meeting. They are now publicly available to serve a growing community of practice focused on engineering lab writing.more » « less
-
Laboratory reports are a genre of writing that students are exposed to early in their engineering curriculum. Varied student writing preparation ensures that students need differentiated support in laboratory writing to achieve learning outcomes. Supported by the National Science Foundation Improving Undergraduate STEM Education initiative, researchers at three institutions have developed a series of scaffolded laboratory writing modules related to different components of a laboratory report. The module contents were informed by prior research into student performance in laboratory report writing in multiple engineering disciplines and with varied writing preparation. The modules provide definitions and guidance for novice report writers and instructor support for developing assignments and rubrics for laboratory reports. The scaffolded modules treat elements of a laboratory report at fundamental, intermediate, and advanced levels. Fundamental modules include audience expectations, lab report organization and conventions, simple statistics, and data presentation in tables and graphs. Intermediate modules address primary and secondary sources of data, trendlines, summary and conclusion writing, and referencing secondary sources. Advanced modules address logical appeals and encourage student writers to consider error analysis and error propagation. This paper describes the structure and content of the modules as well as the process used to develop them. Initial assessments by instructors as module users are presented. Other publicly available writing-support resources are catalogued to demonstrate the novelty and value of the lab report writing modulesmore » « less
-
Laboratory reports are a genre of writing that students are exposed to early in their engineering curriculum. Varied student writing preparation ensures that students need differentiated support in laboratory writing to achieve learning outcomes. Supported by the National Science Foundation Improving Undergraduate STEM Education initiative, researchers at three institutions have developed a series of scaffolded laboratory writing modules related to different components of a laboratory report. The module contents were informed by prior research into student performance in laboratory report writing in multiple engineering disciplines and with varied writing preparation. The modules provide definitions and guidance for novice report writers and instructor support for developing assignments and rubrics for laboratory reports. The scaffolded modules treat elements of a laboratory report at fundamental, intermediate, and advanced levels. Fundamental modules include audience expectations, lab report organization and conventions, simple statistics, and data presentation in tables and graphs. Intermediate modules address primary and secondary sources of data, trendlines, summary and conclusion writing, and referencing secondary sources. Advanced modules address logical appeals and encourage student writers to consider error analysis and error propagation. This paper describes the structure and content of the modules as well as the process used to develop them. Initial assessments by instructors as module users are presented. Other publicly available writing-support resources are catalogued to demonstrate the novelty and value of the lab report writing modules.more » « less
-
This study aims to identify the linguistic feature characteristics of multiple writing assignments completed by engineering undergraduates, including entry-level engineering laboratory reports and writing produced in non-engineering courses. We used Biber’s multidimensional analysis (MDA) method as the analysis tool for the student writing artifacts. MDA is a corpus-analysis methodology that utilizes language processing software to analyze text by parts of speech (e.g. nouns, verbs, prepositions, etc.). MDA typically identifies six “dimensions” of linguistic features that a text may perform in, and each dimension is rated along a continuum. The dimensions used in this study include Dimension 1: Informational vs involved, Dimension 3: Context dependence, Dimension 4: Overt persuasion, and Dimension 5: Abstract vs. non-abstract information. In AY 2019-2020, total of 97 student artifacts (N = 97) were collected. For this analysis, we grouped documents into similar assignment genres: research-papers (n = 45), technical reports and analyses (n = 7) and engineering laboratory reports (n = 35), with individual engineering students represented at least once in the laboratory report and once in another category. Findings showed that engineering lab reports are highly informational, minimally-persuasive, and used deferred elaboration. Students’ research papers in academic writing courses, conversely, were highly involved, highly persuasive, and featured more immediate elaboration on claims and data. The analyses above indicate that students are generally performing as expected in lab report writing in entry-level engineering lab classes, and that this performance is markedly different from their earlier academic writing courses, such as first-year-composition (FYC) and technical communication/writing, indicating that students are not merely “writing like engineers” from their first day at college. However, similarities in context dependence suggest that engineering students must still learn to modulate their languages in writing dramatically depending on the writing assignment. While some students show little growth from one context to another, others are able to change their register or other linguistic/structural features to meet the needs of their audience.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
