Purpose. To make course-based, undergraduate design projects more manageable, instructors often reduce or remove the open-ended quality, which in turn limits opportunities for students to learn to frame design problems. Here we introduce and characterize the construct, framing agency, which involves taking up opportunities to make consequential decisions about design problems and how to proceed in learning and developing solutions.
Methodology. We employed a multi-case study design, selecting cases of student design teams across different sites and levels, all in undergraduate engineering courses. Teams were audio/video recorded during their design process. We adapted a functional linguistics tool [1] to identify markers of agency in students’ design discourse, comparing and contrasting the cases to illuminate the nuances of framing agency. We also identified learning versus task-completion orientations.
Results. All students exhibited agency in some form, but not all exhibited framing agency. Analysis suggests that framing agency is commonly shared across participants and tentative in nature early in the design process. Students who exhibited framing agency tended to adopt a learning rather than task-completion orientation. Students who exhibited agency, but not framing agency, made decisions that foregrounded accuracy and efficiency at the expense of exploring tentative ideas, and tended to treat the problem as having a single right answer.
Conclusions. We argue that how students negotiate design problem framing depends on whether or not they consider the design problem relevant and authentic, the belief that each member brings different and potentially useful information to the task, and the opportunity to iterate design ideas over time. Framing agency provides a lens for understanding the kinds of design learning experiences students need to direct their own learning and negotiate that learning with peers in design projects.
more »
« less
This content will become publicly available on July 1, 2024
Confirmatory factor analysis of the framing agency survey.
In this research paper, we investigate the structure and validity of survey data related to students’
framing agency. In order to promote increased opportunities for students to engage in and learn
to frame design problems that are innovative and empathetic, there is a need for instruments that
can provide information about student progress and the quality of learning experiences. This is a
complex problem because, compared to problem solving, design problem framing is less studied
and harder to predict due to the higher levels of student agency involved. To address this issue,
we developed a survey to measure framing agency, which is defined as opportunities to frame
and reframe design problems and learn in the process. This study extends past research which
focused on the construct of framing agency and developing an instrument to measure it following
best practices in survey design, including using exploratory factor analysis of pilot data, which
recovered six factors related to shared and individual consequentiality, problem structure and
constrainedness, and learning. However, as a pilot, the sample limited generalizability; the
current study addresses this limitation. We used a national cohort that included multiple
engineering disciplines (biomedical, mechanical, chemical, electrical, computer, aerospace),
types of formal design projects (e.g., first-year, design-spine, senior capstone) and institution
types, including private religious; Hispanic-serving; public land-grant; and research flagship
institutions (N=449). We report sample characteristics and used confirmatory factor analysis
(CFA) to provide validity evidence, reporting the chi-square and standardized root mean square
residual as estimates of fit. We report Cronbach’s alpha as a measure of internal consistency.
We found that overall, the CFA aligned with the prior exploratory results, in this case, recovering
four factors, measured on a seven-point scale: shared consequentiality (the extent to which the
student identifies that their understanding of the problem changed as result of a teammate’s
decision, M = 6.15; SD = 1.13); learning as consequentiality (the extent to which the student
identifies learning as the result of decisions, M = 5.88; SD = 0.98); constrainedness (the extent to
which the student reports the ability to make decisions despite design constraints, M = 4.95; SD
= 1.49); and shared tentativeness (the extent to which the student identifies uncertainty about the
problem and solution, M = 4.02; SD = 1.76). This suggests the survey can provide valid data for
instructional decisions and further research into how students learn to frame engineering design
problems and what role framing plays in their professional formation.
more »
« less
- Award ID(s):
- 1751369
- NSF-PAR ID:
- 10495167
- Publisher / Repository:
- Proceedings of the American Society for Engineering Education Annual Conference & Exposition
- Date Published:
- Journal Name:
- Proceedings of the ASEE Annual Conference and Exhibition
- Format(s):
- Medium: X
- Location:
- https://strategy.asee.org/42737
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
A key challenge in engineering design problem framing is defining requirements and metrics. This is difficult, in part, because engineers must make decisions about how to treat qualitative and subjective issues, like stakeholder preferences, about how to prioritize different requirements, and about how to maintain tentativeness and ill-structuredness in the solution space. And this is made more challenging in light of the function of requirements in other types of engineering problems, like feasibility analysis, in which the requirements should converge on a decision. Given these challenges, it is unsurprising that there is limited research on how first-year students approach such work, how they make sense of requirements, and how their conceptualizations of requirements change with instruction. Our purpose in this study is to investigate students’ initial understanding and use of engineering requirements in a specific problem solving context. We developed a survey to measure students’ perceptions related to engineering requirements based on constructs derived from the literature on engineering requirements. We implemented the survey in a first-year and in senior courses for the purpose of validating items using factor analysis. Following this, we conducted analysis of survey and interview data restricted to the first-year course, including epistemic beliefs and analysis of students’ agency. Through exploratory factor analysis, we found that factors did not converge around constructs as described in the literature. Rather, factors formed around the forms of information leveraged to develop requirements. Through qualitative analysis of students’ responses on the survey and to interviews, we evaluated the extent to which students expressed agency over their use of requirements to make decisions within a course project. We describe implications of this exploratory study in terms of adapting research instruments to better understand this topic. Further, we consider pedagogical implications for first year programs and beyond in supporting students to develop ownership over decision making related to engineering requirements.more » « less
-
null (Ed.)Supporting students to frame design problems is one of the most challenging aspects of engineering education, and as faculty, sharing agency with students, such that they have framing agency to make decisions that are consequential to the problem frame is difficult. In this paper, we report on students’ progress framing authentic problems early and after four months of work. Set in a high-agency, co-curricular intramural program where students work on interdisciplinary design projects, we found, using surveys and student work, that early in the process, students reported open-ended problems constrained somewhat by budget or design requirements. Over time, they came to recognize their own limitations as constraining, became more tentative in their treatment of the problem, and reported opportunities to learn from their own and peers’ decisions. Students who reported opportunities to learn also reported working on somewhat more constrained problems yet being able to make consequential decisions. Collectively, this suggests problems that offer a Goldilocks middle ground, that include endemic constraints yet allow students to make consequential decisions may be a key ingredient for developing problem framing capacity. We share instructional implications related to supporting students to differentiate between design requirements and constraints, in shifting from qualitative understandings to quantitative requirements and their role in doing so, and navigating their own limitations.more » « less
-
Supporting students to frame design problems is one of the most challenging aspects of engineering education, and as faculty, sharing agency with students, such that they have framing agency to make decisions that are consequential to the problem frame is difficult. In this paper, we report on students’ progress framing authentic problems early and after four months of work. Set in a high-agency, co-curricular intramural program where students work on interdisciplinary design projects, we found, using surveys and student work, that early in the process, students reported open-ended problems constrained somewhat by budget or design requirements. Over time, they came to recognize their own limitations as constraining, became more tentative in their treatment of the problem, and reported opportunities to learn from their own and peers’ decisions. Students who reported opportunities to learn also reported working on somewhat more constrained problems yet being able to make consequential decisions. Collectively, this suggests problems that offer a Goldilocks middle ground, that include endemic constraints yet allow students to make consequential decisions may be a key ingredient for developing problem framing capacity. We share instructional implications related to supporting students to differentiate between design requirements and constraints, in shifting from qualitative understandings to quantitative requirements and their role in doing so, and navigating their own limitations.more » « less
-
Laboratory experimentation is a key component of the development of professional engineers. However, experiments conducted in chemical engineering laboratory classes are commonly more prescriptive than the problems faced by practicing engineers, who have agency to make consequential decisions across the experiment and communication of results. Thus, understanding how experiments in laboratory courses vary in offering students opportunities to make such decisions, and how students navigate higher agency learning experiences is important for preparing graduates ready to direct these practices. In this study, we sought to answer the following research questions: How do students perceive their agency in course-based undergraduate research experiences? What factors are measured by the Consequential Agency in Laboratory Experiments survey? To better understand student perceptions of their agency in relation to laboratory experiments, we first conducted a case study of a course-based research experience (CURE) in a senior-level chemical engineering laboratory course. We then surveyed six upper-division laboratory courses across two universities using an initial version of the Consequential Agency in Laboratory Experiments survey. We used exploratory factor analysis to investigate the validity of the data from the survey for measuring relevant constructs of authenticity, agency in specific domains, responsibility, and opportunity to make decisions. We found that with instructional support, students in the CURE recognized that failure could itself provide opportunities for learning. They valued having the agency to make consequential decisions, even when they also found the experience challenging. We also found strong support for items measuring agency as responsibility, authenticity, agency in the communication domain, agency in the experimental design domain, and opportunity to make decisions. These findings give us insight into the value of higher agency laboratory experiments, and they provide a foundation for developing a more precise survey capable of measuring agency across various laboratory experiment practices. Such a survey will enable future studies that investigate the impacts of increasing agency in just one domain versus in several. In turn, this can aid faculty in developing higher agency learning experiences that are more feasible to implement, compared to CUREs.more » « less
