skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Paradigm Shift? Preliminary Findings of Engineering Faculty Members’ Mental Models of Assessment in the Era of Generative AI
Title: Paradigm Shift? Preliminary Findings of Engineering Faculty Members’ Mental Models of Assessment in the Era of Generative AI
The emergence of generative artificial intelligence (GAI) has started to introduce a fundamental reexamination of established teaching methods. These GAI systems offer a chance for both educators and students to reevaluate their academic endeavors. Reevaluation of current practices is particularly pertinent in assessment within engineering instruction, where advanced generative text algorithms are proficient in addressing intricate challenges like those found in engineering courses. While this juncture presents a moment to revisit general assessment methods, the actual response of faculty to the incorporation of GAI in their evaluative techniques remains unclear. To investigate this, we have initiated a study delving into the mental constructs that engineering faculty hold about evaluation, focusing on their evolving attitudes and responses to GAI, as reported in the Fall of 2023. Adopting a long-term data-gathering strategy, we conducted a series of surveys, interviews, and recordings targeting the evaluative decision-making processes of a varied group of engineering educators across the United States. This paper presents the data collection process, our participants’ demographics, our data analysis plan, and initial findings based on the participants’ backgrounds, followed by our future work and potential implications. The analysis of the collected data will utilize qualitative thematic analysis in the next step of our study. Once we complete our study, we believe our findings will sketch the early stages of this emerging paradigm shift in the assessment of undergraduate engineering education, offering a novel perspective on the discourse surrounding evaluation strategies in the field. These insights are vital for stakeholders such as policymakers, educational leaders, and instructors, as they have significant ramifications for policy development, curriculum planning, and the broader dialogue on integrating GAI into educational evaluation.  more » « less
Award ID(s):
2113631
PAR ID:
10525800
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
American Society for Engineering Education Annual Conference and Exposition
Date Published:
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, ASEE Conferences)
    This work-in-progress paper explores university students’ perspectives on Generative Artificial Intelligence (GAI) tools, such as ChatGPT, an increasingly prominent topic in the academic community. There is ongoing debate about whether faculty should teach students how to use GAI tools, restrict their usage to maintain academic integrity, or establish regulatory guidelines for sustained integration into higher education. Unfortunately, limited research exists beyond surface-level policies and educator opinions regarding GAI, and its full impact on student learning remains largely unknown. Therefore, understanding students' perceptions and how they use GAI is crucial to ensuring its effective and ethical integration into higher education. As GAI continues to disrupt traditional educational paradigms, this study seeks to explore how students perceive its influence on their learning and problem-solving. As part of a larger mixed-methods study, this work-in-progress paper presents preliminary findings from the qualitative portion using a phenomenological approach that answers the research question: How do university students perceive disruptive technologies like ChatGPT affecting their education and learning? By exploring the implications of Artificial Intelligence (AI) tools on student learning, academic integrity, individual beliefs, and community norms, this study contributes to the broader discourse on the role of emerging technologies in shaping the future of teaching and learning in education. 
    more » « less
  2. ; ; (, American Society for Engineering Education Annual Conference)
    The purpose of this NSF grantees poster is to disseminate initial findings on faculty perception of mastery-based assessment in a project-based engineering program as part of an NSF Broadening Participation award. It is understood that pedagogical approaches influence more than what students learn but also impact their mindsets, motivation, and how they see themselves as engineers. Mastery-based teaching has seen growing popularity in engineering education as faculty strive to support students in achieving learning outcomes linked with continuous improvement to promote performance and persistence. However, this teaching approach has specific challenges as it requires significant restructuring of assessment practices including assignments, exams, evaluation processes, and grading. This work seeks to better understand faculty perspectives of assessment within mastery-based teaching to support a user-oriented perspective that can help other engineering faculty navigate the challenges of using evidence-based teaching practices in their own classrooms. This paper focuses on qualitative findings from an initial pilot study from a larger, NSF-funded Broadening Participation project at a small, Eastern private college. This exploratory pilot study includes the perceptions of two engineering faculty members using mastery teaching and assessment in a project-based engineering program. A semi-structured interview with multiple open-ended questions was used to prompt participants to share their experiences with assessment in relation to their self-efficacy around teaching and their perceptions of assessment in relation to their students’ learning, confidence, and agency. Directed content and thematic analysis were used to identify codes and develop themes in relation to how participants described certain features of assessment in their engineering program. Preliminary results will illustrate features of mastery assessment that faculty highlighted as particularly challenging or successful and related lessons learned. The initial themes and patterns identified in this preliminary pilot study will be used to set up a more focused secondary full data collection phase in the larger study. Additionally, this poster serves as an opportunity to initiate important dialogue around the implementation of mastery-based assessment and project-based learning in engineering programs and to better support engineering faculty in incorporating elements of mastery-based teaching and assessment. 
    more » « less
  3. ; ; ; (, IEEE Frontiers in Education Conference)
    This full research paper documents assessment definitions from engineering faculty members, mainly from Research 1 universities. Assessments are essential components of the engineering learning environment, and how engineering faculty make decisions about assessments in their classroom is a relatively understudied topic in engineering education research. Exploring how engineering faculty think and implement assessments through the mental model framework can help address this research gap. The research documented in this paper focuses on analyzing data from an informational questionnaire that is part of a larger study to understand how the participants define assessments through methods inspired by mixed method strategies. These strategies include descriptive statistics on demographic findings and Natural Language Processing (NLP) and coding on the open-ended response question asking the participants to define assessments, which yielded cluster themes that characterize the definitions. Findings show that while many participants defined assessments in relation to measuring student learning, other substantial aspects include benchmarking, assessing student ability and competence, and formal evaluation for quality. These findings serve as foundational knowledge toward deeper exploration and understanding of assessment mental models of engineering faculty that can begin to address the aforementioned research gap on faculty assessment decisions in classrooms. 
    more » « less
  4. ; ; ; ; (, ASEE Annual Conference proceedings)
    Unlike medicine, the engineering profession establishes new standards for engineering education through a distributed system of governance that mirrors the distributed structure of the profession. In this paper, we present our preliminary findings resulting from early data collected through an NSF-sponsored study of this system. This qualitative study is multi-site and multiscale in its design, and will eventually draw on interviews with faculty and administrators, at different rank, from at least two-dozen different colleges and universities as well as engineering professional organizations. Our interview data is complemented by content analysis of archival documents and published studies, reports, and statements. This paper is designed to introduce our research questions and begin a conversation among engineering educators about how we govern our own educational system. The trends and observations noted in this paper are abstracted from our earliest results, and are described only in general terms. Future papers will explore each of our research questions more fully, taking into account more detailed data. 
    more » « less
  5. ; ; ; ; (, ASEE annual conference & exposition proceedings)
    Unlike medicine, the engineering profession establishes new standards for engineering education through a distributed system of governance that mirrors the distributed structure of the profession. In this paper, we present our preliminary findings resulting from early data collected through an NSF-sponsored study of this system. This qualitative study is multi-site and multiscale in its design, and will eventually draw on interviews with faculty and administrators, at different rank, from at least two-dozen different colleges and universities as well as engineering professional organizations. Our interview data is complemented by content analysis of archival documents and published studies, reports, and statements. This paper is designed to introduce our research questions and begin a conversation among engineering educators about how we govern our own educational system. The trends and observations noted in this paper are abstracted from our earliest results, and are described only in general terms. Future papers will explore each of our research questions more fully, taking into account more detailed data. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email