skip to main content

Title: Work in Progress: Content Validation of an Engineering Process Safety Decision Making Instrument
Process safety is at the heart of operation of many chemical processing companies. However, the Chemical Safety Board (CSB) has still documented over 800 investigations of process safety failures since the year 2000. While not all of these incidents were severe, some did lead to employee injuries or death and environmental harm. As a result, chemical engineering companies are increasingly dedicated to process safety through training programs and detailed vigilance as part of their operations practice. AIChE and OSHA also offer courses in process safety to help support the industry. These efforts illustrate the paramount importance that chemical engineering graduates have an appreciation and understanding of process safety as they transition from their degree program into industrial positions. Previous studies have shown that despite difficulties due to course load constraints, process safety has been incorporated into chemical engineering curriculum through either the addition of new courses, incorporation of the content within existing classes, or a combination of the two methods. A review performed in Process Safety Progress suggested that a key step for departments moving forward is to perform an assessment of the process safety culture within their institution in order to determine how faculty and students view process safety. An more » issue with completing this task is the lack of assessment tools that can be used to determine how students are developing their understanding of process safety decision making. This observation led to the development of the Engineering Process Safety Research Instrument (EPSRI). This instrument is modeled after the Defining Issues Test version 2 (DIT2) and the Engineering Ethical Reasoning Instrument (EERI). Similar to these instruments, the EPSRI provides dilemmas, three decisions, and 12 additional considerations that individuals must rate based on their relative importance to their decision making process. The dilemmas developed in the EPSRI are based on case studies and investigations from process safety failures that have occurred in industry to provide a realistic context for the decision making decisions that engineers may be faced with upon employment. The considerations provided after the scenario are derived to reflect pre-conventional, conventional, and post-conventional decision making thinking as described by Kohlberg’s Moral Development Theory. Pre-conventional decision making thinking focuses particularly on what is right/wrong or good/bad from an individual level, whereas post-conventional thinking seeks to determine what is correct from moral and value perspectives at the society level. This WIP paper describes the content validity study conducted while developing the EPSRI. Dilemmas were examined by context experts including professionals in the process industry, chemical engineering departments, and learning sciences field. Content experts reviewed the dilemmas and determined whether they represented accurate examples of process safety decision making that individuals may face in real-world engineering settings. The experts also reviewed the 12 considerations for each dilemma for their accuracy in capturing pre-conventional, conventional and post-conventional thinking. This work represents the first step in the overall instrument validation that will take place over the next academic year. « less
Authors:
; ; ; ;
Award ID(s):
1711866
Publication Date:
NSF-PAR ID:
10089269
Journal Name:
American Society for Engineering Education: Proceedings of the 2018 ASEE Annual Conference and Exposition
Sponsoring Org:
National Science Foundation
More Like this
  1. Process safety is at the heart of operation of many chemical processing companies. However, the Chemical Safety Board (CSB) has still documented over 800 investigations of process safety failures since the year 2000. While not all of these incidents were severe, some did lead to employee injuries or death and environmental harm. As a result, chemical engineering companies are increasingly dedicated to process safety through training programs and detailed vigilance as part of their operations practice. AIChE and OSHA also offer courses in process safety to help support the industry. These efforts illustrate the paramount importance that chemical engineering graduatesmore »have an appreciation and understanding of process safety as they transition from their degree program into industrial positions. Previous studies have shown that despite difficulties due to course load constraints, process safety has been incorporated into chemical engineering curriculum through either the addition of new courses, incorporation of the content within existing classes, or a combination of the two methods. A review performed in Process Safety Progress suggested that a key step for departments moving forward is to perform an assessment of the process safety culture within their institution in order to determine how faculty and students view process safety. An issue with completing this task is the lack of assessment tools that can be used to determine how students are developing their understanding of process safety decision making. This observation led to the development of the Engineering Process Safety Research Instrument (EPSRI). This instrument is modeled after the Defining Issues Test version 2 (DIT2) and the Engineering Ethical Reasoning Instrument (EERI). Similar to these instruments, the EPSRI provides dilemmas, three decisions, and 12 additional considerations that individuals must rate based on their relative importance to their decision making process. The dilemmas developed in the EPSRI are based on case studies and investigations from process safety failures that have occurred in industry to provide a realistic context for the decision making decisions that engineers may be faced with upon employment. The considerations provided after the scenario are derived to reflect pre-conventional, conventional, and post-conventional decision making thinking as described by Kohlberg’s Moral Development Theory. Pre-conventional decision making thinking focuses particularly on what is right/wrong or good/bad from an individual level, whereas post-conventional thinking seeks to determine what is correct from moral and value perspectives at the society level. This WIP paper describes the content validity study conducted while developing the EPSRI. Dilemmas were examined by context experts including professionals in the process industry, chemical engineering departments, and learning sciences field. Content experts reviewed the dilemmas and determined whether they represented accurate examples of process safety decision making that individuals may face in real-world engineering settings. The experts also reviewed the 12 considerations for each dilemma for their accuracy in capturing pre-conventional, conventional and post-conventional thinking. This work represents the first step in the overall instrument validation that will take place over the next academic year.« less
  2. Process safety is at the heart of operation of many chemical processing companies. However, the Chemical Safety Board (CSB) has still documented over 800 investigations of process safety failures since the year 2000. While not all of these incidents were severe, some did lead to employee injuries or death and environmental harm. As a result, chemical engineering companies are increasingly dedicated to process safety through training programs and detailed vigilance as part of their operations practice. AIChE and OSHA also offer courses in process safety to help support the industry. These efforts illustrate the paramount importance that chemical engineering graduatesmore »have an appreciation and understanding of process safety as they transition from their degree program into industrial positions. Previous studies have shown that despite difficulties due to course load constraints, process safety has been incorporated into chemical engineering curriculum through either the addition of new courses, incorporation of the content within existing classes, or a combination of the two methods. A review performed in Process Safety Progress suggested that a key step for departments moving forward is to perform an assessment of the process safety culture within their institution in order to determine how faculty and students view process safety. An issue with completing this task is the lack of assessment tools that can be used to determine how students are developing their understanding of process safety decision making. This observation led to the development of the Engineering Process Safety Research Instrument (EPSRI). This instrument is modeled after the Defining Issues Test version 2 (DIT2) and the Engineering Ethical Reasoning Instrument (EERI). Similar to these instruments, the EPSRI provides dilemmas, three decisions, and 12 additional considerations that individuals must rate based on their relative importance to their decision making process. The dilemmas developed in the EPSRI are based on case studies and investigations from process safety failures that have occurred in industry to provide a realistic context for the decision making decisions that engineers may be faced with upon employment. The considerations provided after the scenario are derived to reflect pre-conventional, conventional, and post-conventional decision making thinking as described by Kohlberg’s Moral Development Theory. Pre-conventional decision making thinking focuses particularly on what is right/wrong or good/bad from an individual level, whereas post-conventional thinking seeks to determine what is correct from moral and value perspectives at the society level. This WIP paper describes the content validity study conducted while developing the EPSRI. Dilemmas were examined by context experts including professionals in the process industry, chemical engineering departments, and learning sciences field. Content experts reviewed the dilemmas and determined whether they represented accurate examples of process safety decision making that individuals may face in real-world engineering settings. The experts also reviewed the 12 considerations for each dilemma for their accuracy in capturing pre-conventional, conventional and post-conventional thinking. This work represents the first step in the overall instrument validation that will take place over the next academic year.« less
  3. Process safety is at the heart of operation of many chemical processing companies. However, the Chemical Safety Board (CSB) has still documented over 800 investigations of process safety failures since the year 2000. While not all of these incidents were severe, some did lead to employee injuries or death and environmental harm. As a result, chemical engineering companies are increasingly dedicated to process safety through training programs and detailed vigilance as part of their operations practice. AIChE and OSHA also offer courses in process safety to help support the industry. These efforts illustrate the paramount importance that chemical engineering graduatesmore »have an appreciation and understanding of process safety as they transition from their degree program into industrial positions. Previous studies have shown that despite difficulties due to course load constraints, process safety has been incorporated into chemical engineering curriculum through either the addition of new courses, incorporation of the content within existing classes, or a combination of the two methods. A review performed in Process Safety Progress suggested that a key step for departments moving forward is to perform an assessment of the process safety culture within their institution in order to determine how faculty and students view process safety. An issue with completing this task is the lack of assessment tools that can be used to determine how students are developing their understanding of process safety decision making. This observation led to the development of the Engineering Process Safety Research Instrument (EPSRI). This instrument is modeled after the Defining Issues Test version 2 (DIT2) and the Engineering Ethical Reasoning Instrument (EERI). Similar to these instruments, the EPSRI provides dilemmas, three decisions, and 12 additional considerations that individuals must rate based on their relative importance to their decision making process. The dilemmas developed in the EPSRI are based on case studies and investigations from process safety failures that have occurred in industry to provide a realistic context for the decision making decisions that engineers may be faced with upon employment. The considerations provided after the scenario are derived to reflect pre-conventional, conventional, and post-conventional decision making thinking as described by Kohlberg’s Moral Development Theory. Pre-conventional decision making thinking focuses particularly on what is right/wrong or good/bad from an individual level, whereas post-conventional thinking seeks to determine what is correct from moral and value perspectives at the society level. This WIP paper describes the content validity study conducted while developing the EPSRI. Dilemmas were examined by context experts including professionals in the process industry, chemical engineering departments, and learning sciences field. Content experts reviewed the dilemmas and determined whether they represented accurate examples of process safety decision making that individuals may face in real-world engineering settings. The experts also reviewed the 12 considerations for each dilemma for their accuracy in capturing pre-conventional, conventional and post-conventional thinking. This work represents the first step in the overall instrument validation that will take place over the next academic year.« less
  4. Process safety has become a critical component of chemical engineering education. However, students may find it difficult to fully understand the ramifications of decisions they make during classroom exercises due to their lack of real world experience. Use of an immersive digital environment where students could role play as chemical engineering employees making process safety decisions could be one method of achieving this goal. Through this experience, students could observe the outcomes of their decisions in a safe, controlled environment without the disastrous real-world consequences that could come from making a mistake. This digital environment could have further features, suchmore »as time constraints or interactions with other characters, to make the experience feel more authentic than an in-class discussion or case study. In order to evaluate the efficacy of such a virtual environment, a portion of this work centered around the creation of the Engineering Process Safety Research Instrument (EPSRI). The instrument asks participants to evaluate process safety dilemmas and rank a set of considerations based on how influential they were in their decision-making process. The instrument then classifies each decision based on the stages of Kohlberg’s moral development theory, ranging from pre-conventional (i.e. more self-centered) thinking to post-conventional (i.e. more global) thinking. This instrument will be used to assess how students’ thinking about process safety decisions changes as a result of engaging in the virtual safety decision making environment. This paper will summarize the progress since the project’s start in summer 2017, highlighting the work completed in development and validation of the EPSRI. This process included content validation, think-aloud studies to improve clarity of the instrument, and factor analysis based on a large scale implementation at multiple universities. The paper will also discuss the development of the minimum viable product digital process safety experience, including establishment of learning outcomes and the mechanics that reinforce those outcomes. By presenting these findings, we intend to spread awareness of the EPSRI, which can evaluate the safety decisions of chemical engineering students while having the potential to launch discussions about safety and ethics in other engineering disciplines. We also hope that these results will provide educators with insights into how to translate educational objectives to elements of a digital learning environment through collaboration with digital media companies.« less
  5. Process safety has become a critical component of chemical engineering education. However, students may find it difficult to fully understand the ramifications of decisions they make during classroom exercises due to their lack of real world experience. Use of an immersive digital environment where students could role play as chemical engineering employees making process safety decisions could be one method of achieving this goal. Through this experience, students could observe the outcomes of their decisions in a safe, controlled environment without the disastrous real-world consequences that could come from making a mistake. This digital environment could have further features, suchmore »as time constraints or interactions with other characters, to make the experience feel more authentic than an in-class discussion or case study. In order to evaluate the efficacy of such a virtual environment, a portion of this work centered around the creation of the Engineering Process Safety Research Instrument (EPSRI). The instrument asks participants to evaluate process safety dilemmas and rank a set of considerations based on how influential they were in their decision-making process. The instrument then classifies each decision based on the stages of Kohlberg’s moral development theory, ranging from pre-conventional (i.e. more self-centered) thinking to post-conventional (i.e. more global) thinking. This instrument will be used to assess how students’ thinking about process safety decisions changes as a result of engaging in the virtual safety decision making environment. This paper will summarize the progress since the project’s start in summer 2017,, highlighting the work completed in development and validation of the EPSRI. This process included content validation, think-aloud studies to improve clarity of the instrument, and factor analysis based on a large scale implementation at multiple universities. The paper will also discuss the development of the minimum viable product digital process safety experience, including establishment of learning outcomes and the mechanics that reinforce those outcomes. By presenting these findings, we intend to spread awareness of the EPSRI, which can evaluate the safety decisions of chemical engineering students while having the potential to launch discussions about safety and ethics in other engineering disciplines. We also hope that these results will provide educators with insights into how to translate educational objectives to elements of a digital learning environment through collaboration with digital media companies.« less