More Like this

1. Exploring the Viability of Agent-Based Modeling to Extend Qualitative Research: Comparison of Computational Platforms

   Splendido, S.; Berdanier, C.G.P. (January 2023, ASEE annual conference exposition)

   The purpose of this methods paper is to identify the opportunities and applications of agent-based modeling (ABM) methods to interpretative qualitative and educational research domains. The context we explore in this paper considers graduate engineering attrition, which has been a funded research focus of our group for ten years. In attrition research, as with all human research, it is impossible and unethical to imperil real graduate students by subjecting them to acute stressors that are known to contribute to attrition in order to “test” different combinations of factors on persistence and attrition. However, agent-based modeling (ABM) methods have been applied in other human decision-making contexts in which a computer applies researcher programmed logic to digital actors, invoking them to make digital decisions that mimic human decision making. From our research team’s ten years of research studying graduate socialization and attrition and informed from a host of theories that have been used in literature to investigate doctoral attrition, this paper compares the utility of two programming languages, Python and NetLogo, in conducting agent-based modeling to model graduate attrition as a platform. In this work we show that both platforms can be used to simulate attrition and persistence scenarios for thousands of digital agent-students simultaneously to produce results that agree with both with previous qualitative data and that agree with aggregate attrition and persistence statistics from literature. The two languages differ in their integrated development environments (IDE) with the methods of producing the models customizable to fit the needs of the study. Additionally, the size of the intended agent pool impacted the efficiency of the data collection. As computational methods can transform educational research, this work provides both a proof-of-concept and recommendations for other researchers considering employing these methods with these and similar platforms. Ultimately, while there are many programming languages that can perform agent-based modeling tasks, researchers are responsible for translating high quality, theory-driven, interpretive research into a computational model that can model human decision-making processes. 

   more » « less
2. I Graduated, Now What? An Overview of the Academic Engineering Education Research Job Field and Search Process

   https://doi.org/10.18260/1-2--34741  

   McCave, Erin; Bodnar, Cheryl; Smith-Orr, Courtney; Strong, Alexandra; Lee, Walter; Faber, Courtney (January 2021, 2020 ASEE Virtual Annual Conference)

   null (Ed.)

   As the field continues to grow, engineering education is continually challenged with finding engineering education research (EER) positions that align with the broad abilities and interests of its members. EER positions exist in engineering education departments, traditional engineering departments (e.g., mechanical, civil), and in non-degree granting programs (e.g., centers for teaching and learning, engineering programs). These positions vary across their emphasis on research, teaching, and service and provide access to different resources and mechanisms to impact engineering education. Given the range of positions available in EER and the emergence of new EER programs, it can be challenging for graduate students and postdocs to navigate the job search process and identify a position that aligns with their professional goals. The purpose of this research was to better understand the EER job market as it relates to what applicants (i.e., graduates and post-docs) experience as they navigate the job-search and decision-making process. For this study, we conducted interviews with seven transitioning first-year EER faculty members. These individuals were transitioning into various EER faculty positions (e.g. Lecturer, Teaching Fellow, Assistant Professor, Research Assistant Professor) with different backgrounds in EER based on their graduate training experiences which included established EER programs as well as traditional engineering departments with EER advisor(s). We asked questions that focused on the individual’s new faculty position, their perception of the weekly time requirements, their job search process, and factors that influenced their final decision of which job to select. Each interview was conducted by two graduate students and was then transcribed and verified for accuracy. Three faculty members performed holistic coding of the transcripts focused on three areas: EER position types, job search process, and job decision making process. The Qualifying Qualitative research Quality framework (Q3) was used as a guide throughout our data collection and analysis process to ensure reliability and trustworthiness of the data collected. Through our analysis process, we developed a visual representation that provides a guide to assist EER graduate students and postdocs with their job search process. The first figure captures the diversity of positions along with the types of institutions where these positions exist to provide a starting point for individuals on their job search process. The second figure includes a timeline to help capture the average time frames for different phases of the job search process. Factors associated with final decisions based on the interviews conducted are also outlined to provide areas of consideration for individuals undergoing this process in the future. This work provides insight to aspiring academics about the range of opportunities available to those with a background in EER and how they can pursue finding alignment between their interests and positions that are available. 

   more » « less
3. First-year students’ agency related to engineering requirements

   Olewnik, A.; Svihla, V.; Wild, W. (January 2022, Proceedings of the ASEE Annual Conference and Exhibition)

   A key challenge in engineering design problem framing is defining requirements and metrics. This is difficult, in part, because engineers must make decisions about how to treat qualitative and subjective issues, like stakeholder preferences, about how to prioritize different requirements, and about how to maintain tentativeness and ill-structuredness in the solution space. And this is made more challenging in light of the function of requirements in other types of engineering problems, like feasibility analysis, in which the requirements should converge on a decision. Given these challenges, it is unsurprising that there is limited research on how first-year students approach such work, how they make sense of requirements, and how their conceptualizations of requirements change with instruction. Our purpose in this study is to investigate students’ initial understanding and use of engineering requirements in a specific problem solving context. We developed a survey to measure students’ perceptions related to engineering requirements based on constructs derived from the literature on engineering requirements. We implemented the survey in a first-year and in senior courses for the purpose of validating items using factor analysis. Following this, we conducted analysis of survey and interview data restricted to the first-year course, including epistemic beliefs and analysis of students’ agency. Through exploratory factor analysis, we found that factors did not converge around constructs as described in the literature. Rather, factors formed around the forms of information leveraged to develop requirements. Through qualitative analysis of students’ responses on the survey and to interviews, we evaluated the extent to which students expressed agency over their use of requirements to make decisions within a course project. We describe implications of this exploratory study in terms of adapting research instruments to better understand this topic. Further, we consider pedagogical implications for first year programs and beyond in supporting students to develop ownership over decision making related to engineering requirements. 

   more » « less
4. Beyond analytics: Using computer‐aided methods in educational research to extend qualitative data analysis

   https://doi.org/10.1002/cae.22749  

   Vieira, Camilo; Ortega‐Alvarez, Juan D; Magana, Alejandra J; Boutin, Mireille (May 2024, Computer Applications in Engineering Education)

   This study proposes and demonstrates how computer‐aided methods can be used to extend qualitative data analysis by quantifying qualitative data, and then through exploration, categorization, grouping, and validation. Computer‐aided approaches to inquiry have gained important ground in educational research, mostly through data analytics and large data set processing. We argue that qualitative data analysis methods can also be supported and extended by computer‐aided methods. In particular, we posit that computing capacities rationally applied can expand the innate human ability to recognize patterns and group qualitative information based on similarities. We propose a principled approach to using machine learning in qualitative education research based on the three interrelated elements of the assessment triangle: cognition, observation, and interpretation. Through the lens of the assessment triangle, the study presents three examples of qualitative studies in engineering education that have used computer‐aided methods for visualization and grouping. The first study focuses on characterizing students' written explanations of programming code, using tile plots and hierarchical clustering with binary distances to identify the different approaches that students used to self‐explain. The second study looks into students' modeling and simulation process and elicits the types of knowledge that they used in each step through a think‐aloud protocol. For this purpose, we used a bubble plot and a k‐means clustering algorithm. The third and final study explores engineering faculty's conceptions of teaching, using data from semi‐structured interviews. We grouped these conceptions based on coding similarities, using Jaccard's similarity coefficient, and visualized them using a treemap. We conclude this manuscript by discussing some implications for engineering education qualitative research. 

   more » « less
5. A Case for Cognitive Models in Visualization Research

   Padilla, Lace (October 2018, Seventh Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization)

   The visualization community has seen a rise in the adoption of user studies. Empirical user studies systematically test the assumptions that we make about how visualizations can help or hinder viewers’ performance of tasks. Although the increase in user studies is encouraging, it is vital that research on human reasoning with visualizations be grounded in an understanding of how the mind functions. Previously, there were no sufficient models that illustrate the process of decision-making with visualizations. However, Padilla et al. [41] recently proposed an integrative model for decision-making with visualizations, which expands on modern theories of visualization cognition and decision-making. In this paper, we provide insights into how cognitive models can accelerate innovation, improve validity, and facilitate replication efforts, which have yet to be thoroughly discussed in the visualization community. To do this, we offer a compact overview of the cognitive science of decision-making with visualizations for the visualization community, using the Padilla et al. [41] cognitive model as a guiding framework. By detailing examples of visualization research that illustrate each component of the model, this paper offers novel insights into how visualization researchers can utilize a cognitive framework to guide their user studies. We provide practical examples of each component of the model from empirical studies of visualizations, along with visualization implications of each cognitive process, which have not been directly addressed in prior work. Finally, this work offers a case study in utilizing an understanding of human cognition to generate a novel solution to a visualization reasoning bias in the context of hurricane forecast track visualizations. 

   more » « less