skip to main content

Explore Research Products in the NSF-PAR

Title: Development of a situational judgment test for global engineering competency
Abstract Background

As globalization continues to impact the engineering profession, many programs aim to prepare current and future engineers to work across national and cultural boundaries. Yet there remains a lack of quality tools for assessing global competency among engineers and other technical professionals, including their behavioral tendencies in global work situations.

 Purpose

We introduce development of a situational judgment test (SJT) covering three dimensions of global engineering competency (GEC) in Chinese national/cultural context. The main aim of this article is to describe how the SJT was developed through a systematic multistep process. Secondarily, we explore relationships between SJT performance and other theoretically relevant variables.

 Methods

After generating a large initial pool of SJT scenarios and behavioral response items, we used ratings from subject matter experts (SMEs) to select six SJT scenarios and create scoring keys for 26 response items. To further explore the instrument's validity, we deployed the SJT items, other relevant measures, and a demographic survey to a sample of practicing engineers (n= 400).

 Results

SME ratings provide strong evidence for the content relevance of the GEC–SJT tool. Survey results also suggest positive relationships between SJT performance and Chinese cultural knowledge, age, and years of work experience. However, more validity and reliability evidence is needed before recommending wider use of the instrument.

 Conclusions

Our findings suggest the SJT format as a promising behavior‐based approach to measuring global competency and other professional attributes in engineering. We also discuss directions for future research and training efforts related to assessing and developing global competency.

  more » « less
NSF-PAR ID:
10377769
Author(s) / Creator(s):
 ;  ;  ;  
Publisher / Repository:
Wiley Blackwell (John Wiley & Sons)
Date Published:
Journal Name:
Journal of Engineering Education
Volume:
109
Issue:
3
ISSN:
1069-4730
Page Range / eLocation ID:
p. 470-490
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ( , Journal of Engineering Education)
    Abstract Background

    Engineers operate in an increasingly global environment, making it important that engineering students develop global engineering competency to prepare them for success in the workplace. To understand this learning, we need assessment approaches that go beyond traditional self‐report surveys. A previous study (Jesiek et al.,Journal of Engineering Education2020; 109(3):1–21) began this process by developing a situational judgment test (SJT) to assess global engineering competency based in the Chinese context and administering it to practicing engineers.

     Purpose

    We built on this previous study by administering the SJT to engineering students to explore what prior experiences related to their SJT performance and how their SJT performance compared with practicing engineers' performance on the SJT.

     Method

    Engineering students completed a survey including the SJT and related self‐report survey instruments. We collected data from three groups of students: those who had studied abroad in China; those who had studied abroad elsewhere; and those who had not studied abroad.

     Results

    We found that students' SJT performance did not relate to their scores on the self‐report instruments, but did relate to their participation in study abroad programs. The students also performed better on the SJT when compared to the practicing engineers.

     Conclusions

    Our results highlight the need to use multiple forms of assessment for global engineering competence. Although building evidence for the validity of the Global Engineering Competency China SJT is an ongoing process, this data collection technique may provide new insights on global engineering competency compared to traditionally used assessments.

     
    more » « less
  2. ( , Techné: Research in Philosophy and Technology)
    Engineering is more cross-cultural and international than ever before, presenting challenges and opportunities in the way engineering ethics is conceived and delivered. To assist in providing more effective ethics education to increasingly diverse groups, this paper shares three related projects implemented at the University of Michigan-Shanghai Jiao Tong University Joint Institute (China). These projects are united in their attempts to address challenges arising from the increasingly global nature of engineering. The first is a course on global engineering ethics, developed for and attended by engineering students from diverse backgrounds. The second is a website hosting contents on global engineering ethics education and conducting research related to cross-cultural moral psychology. The third explores methods of assessing engineering ethics and moral development, using paradigms of ethical decision-making. Although these projects were developed in a Chinese-US collaboration with university students, these contexts could facilitate the adoption of similar programs elsewhere, with practicing engineers. 
    more » « less
  3. ; ; ; ; ( , Journal of Engineering Education)
    Abstract Background

    A better understanding of departmental climate and its relationship to engineering identity is needed to diversify engineering and improve marginalized students' experiences.

     Purpose/Hypothesis

    We investigated whether undergraduate engineering students from 16 social identity groups perceived departmental climate differently from one another and examined psychological and behavioral factors contributing to these perceptions and their relationship to engineering identification.

     Design/Method

    We surveyed 398 undergraduate engineering students about departmental climate and engineering identity, testing structural models across race and gender. Qualitative analysis of open‐ended items complemented quantitative results.

     Results

    Students rated climate for dominant identities (White, male, and/or US‐born) as more welcoming than for 14 nondominant identities, broadening the notion of “nondominant” identities in engineering. In structural models, invariant across race and gender, students' perceptions of bias, safety, and faculty support predicted climate ratings; peer relations and microaggressions predicted engineering identity. There were mean differences in perceptions across intersections of race and gender, but students in all groups perceived a climate gap favoring dominant identities. Open‐ended responses highlighted students' desire for a more diverse, inclusive program and the importance of peer relations.

     Conclusions

    Departmental climate can be less welcoming for engineering students with many different nondominant identities. Attending to both students' own social positioning and their perceptions of climate for other students can open opportunities for change in engineering departments. Results suggest that efforts to improve peer relations in group work could be important in promoting disciplinary identification in historically marginalized groups.

     
    more » « less
  4. ; ; ; ; ; ; ; ; ( , 2022 ASEE Annual Conference & Exposition)
    Many of the National Academy of Engineering’s grand challenges are related to environmental engineering. There is broad recognition that these challenges will require environmental engineers to integrate concepts from the natural and physical sciences, social sciences, business, and communications to find solutions at the individual, company, community, national and global levels. Montana State University is in the process of revolutionizing the curriculum and culture of its environmental engineering program to prepare and inspire a new generation of engineers through a project sponsored by the Revolutionizing Engineering Departments program at the National Science Foundation. At the core of the approach is transformation of the hierarchical, topic-focused course structure into a model of team taught, integrated, and project-based learning courses grouped around the key knowledge threads of systems thinking, professionalism, and sustainability. Multi-disciplinary faculty developed specific and detailed program outcomes after review of ABET program outcomes; the Fundamentals of Engineering exam; Body of Knowledge documents from the American Academy of Environmental Engineers (AAEE), the American Society of Civil Engineers (ASCE) and the American Society for Engineering Management (ASEM); the Engineering for One Planet report sponsored by the Lemelson Foundation; and the KEEN Framework on the Entrepreneurial Mindset. The resulting outcomes were organized into competency strands and competency domains. Currently, outcomes spanning the spectrum of content are being crafted into integrated and project-based courses in each year of the undergraduate curriculum. This paper reviews the lessons learned from the process of developing knowledge threads, competency strands and domains, and specific program outcomes with a multidisciplinary group of faculty, as well as the challenges of developing integrated and project-based courses within an established undergraduate curriculum. 
    more » « less
  5. ; ; ; ; ; ; ( , American Society For Engineering Education Annual Conference & Exposition Proceedings)
    The purpose of this work is to determine if global engagement interventions without extended international travel can help engineering students develop a global learner mindset and build towards the overarching goal of developing a holistic global engineering educational approach to meet the current and future needs of the engineering profession. The global learner mindset refers to how engineers perceive and interpret the global environment, and is assumed to be foundational in developing global engineering competence, how they define problems and formulate and implement solutions. This project has focused on assessing the global learner mindset elements, which include cultural humility, global citizenship, and critical reflection within the context of four distinctly different global engagement interventions. These interventions include international engineering case studies in a quantitative analysis course, intentional formation of multi-national student teams within a capstone design course, a Collaborative Online International Learning (COIL) research project in a fluid flow course, and an engineering short course coupled to a community engaged project. The PIs conducted pilot implementations of the four interventions during the spring 2023 semester and collected pre and post assessment data from the Global Engagement Survey (GES) and Global Engineering Competency Scale (GECS) instruments. The results have been used to determine a path forward to improve the next implementation of the interventions during the Spring 2024 semester. This path includes the development of a focus group with students participating in each intervention to obtain a deeper understanding through qualitative data, specifically targeting global engineering mindset formation that will help better contextualize the quantitative results from the GES and GECS instruments. This work aspires to expand the required development of global competencies in engineering beyond the current research focused on the development of intercultural competence in international or study-abroad experiences. Our focus is on the development of a holistic global engineering education process able to reach all engineering students even when institutions are not able to provide opportunities to fully immerse in other cultures, either because of global crises (such as a pandemic or violent conflicts), financial limitations, or the need for more sustainable methods of globally connecting. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email