<?xml-model href='http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng' schematypens='http://relaxng.org/ns/structure/1.0'?><TEI xmlns="http://www.tei-c.org/ns/1.0">
	<teiHeader>
		<fileDesc>
			<titleStmt><title level='a'>Work-in-Progress: Seizing failure as an opportunity to learn: Undergraduate engineering students’ conceptions of failure and iteration</title></titleStmt>
			<publicationStmt>
				<publisher>American Society of Engineering Education Annual Conference</publisher>
				<date>06/25/2024</date>
			</publicationStmt>
			<sourceDesc>
				<bibl> 
					<idno type="par_id">10526170</idno>
					<idno type="doi"></idno>
					
					<author>Sara A Atwood</author><author>Kelsey Scalaro</author><author>Rebecca Holcombe</author>
				</bibl>
			</sourceDesc>
		</fileDesc>
		<profileDesc>
			<abstract><ab><![CDATA[The purpose of this work-in-progress research paper is to explore how mid-program engineering students perceive mastery-based assessment in a multidisciplinary, project-based engineering program.There have been calls in engineering education to support students beyond learning specific competencies but to also include curricular practices that help students see themselves as people who can do engineering. While it is understood that students’ mindsets and feelings of control over their own actions and outcomes are influential in how students see themselves as engineers, less is known about what specific classroom practices have this effect. Specifically, students experience assessment in relation to their learning throughout their engineering programs, but little is known about how their perceptions of these experiences impact their sense of agency or control over how and what they learn. This work explores how mastery-based learning and its associated assessment can be paired with a multidisciplinary project-based learning approach to influence students’ feelings of control and choice in their own engineering education.This paper focuses on qualitative findings from an initial pilot study for a larger, NSF-funded project at a small, Eastern private college. This exploratory pilot study includes the perceptions of four second-year engineering students enrolled in an undergraduate engineering program designed around integrated, multidisciplinary projects. A semi-structured interview with multiple open-ended questions was used to prompt participants to share their experiences with assessment in relation to their own learning, performance, confidence, and choices. Directed content and thematic analysis were used to identify codes and develop themes in relation to how participants described certain features of assessment in their engineering program.Preliminary results will illustrate students’ beliefs, learning, and perceptions of choice in relation to specific features of mastery-based assessment in a multidisciplinary project-based context. The initial themes and patterns identified in this preliminary pilot study will be used to set up a more focused secondary full data collection phase in the larger study and to initiate important conversation around the impacts of specific pedagogical choices on outcomes parallel to learning. By better understanding students’ perceptions of this pedagogical design, future classroom practices can be designed and oriented to support students in feeling agentic in their own engineering education and in becoming their version of an engineer.]]></ab></abstract>
		</profileDesc>
	</teiHeader>
	<text><body xmlns="http://www.tei-c.org/ns/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink">
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Introduction and Background</head><p>How do we prepare diverse engineers to thrive through challenges? Can we change how students experience iteration and failure by changing the way we teach? Does experiencing learning as iteration support students' reported sense of agency? This work-in-progress paper explores how sophomore engineering students enrolled in an experimental multi-disciplinary program described the impact of key program elements on how they experienced failure, perceived risk, and embraced learning as an iterative process. In this problem-based, hands-on and mastery-assessed program, students iterated and worked through failure as part of their learning process. These real-world applications and projects were inherently multidisciplinary, because they challenged students to draw on and integrate their mastery of essential course outcomes related to physics, statics, calculus 3 and circuits. This pilot was too small to support generalizable inferences, but preliminary findings point to key program elements for future research exploring how the participants redefined failure as an opportunity to learn and built their resilience to challenge and risk.</p><p>The field of engineering education is moving beyond teaching specific content, to thinking about how to holistically develop engineers who are resilient, and who can work through failure, tackle ill-structured problems, and address real-world challenges <ref type="bibr">[1]</ref>- <ref type="bibr">[4]</ref>. In solving these real-world problems, solutions are rarely straightforward nor adequately achieved through a singular first attempt <ref type="bibr">[5]</ref>, <ref type="bibr">[6]</ref>. Thus, the ability to iterate through failure is a critical element of both learning and engineering design.</p><p>How students think about and persist through failure is influenced by their understanding of the role of iteration in both engineering design and their own learning <ref type="bibr">[7]</ref>. In turn, how they approach problems when things do not initially work out as planned or require more effort than imagined is related to their own perception of their ability to learn <ref type="bibr">[7]</ref>. This experimental program employs hands-on, problem based learning and supportive (mastery-based) assessment to prepare students to approach problem solving as an iterative activity.</p><p>Hands-on, problem-based learning and mastery-based assessment are pedagogies that focus on students' learning and include iteration through failure as a defining feature. Problem based learning is an approach in which students solve open-ended, ill-structured, authentic problems in which iteration is treated as a problem-solving process <ref type="bibr">[8]</ref>, <ref type="bibr">[9]</ref>. These ill-structured problems challenge students to draw on multiple disciplines to achieve a solution. For example, students in the program developed foundational knowledge related to statics and circuits while building solar powered stoplights and wind turbines that met certain performance criteria. Mastery-based assessment is an assessment approach that allows multiple, iterative demonstrations to show learning. <ref type="bibr">[10]</ref>- <ref type="bibr">[12]</ref>.</p><p>Although iteration is inherent to both hands-on, problem-based learning, and mastery assessment approaches, there is less research on how students consider iteration and failure in relation to their learning and learning mindsets. In addition, there is less research on how students describe the effect of these pedagogical tools on their agency and willingness to try again when work or learning does not go as expected. This preliminary work examines how students experience iteration when it is layered and embedded in an interdisciplinary engineering program that uses both problem-based learning and mastery-based assessment. Exploratory pilot interviews and thematic analysis were used to start answering the following research questions: RQ1: How do undergraduate engineering students experience iterating through failure in this program? RQ2: How do they connect the process of iteration with incremental learning in this program? RQ3: What programmatic features do students identify as supportive of iteration towards mastery?</p><p>Our understanding of the constructs of learning mindsets and iteration guided data collection, analysis, and initial interpretation towards the development of three preliminary themes: 1) how students redefined failure as an opportunity to learn, and 2) how students reframe difficult tasks as challenging rather than risky, and 3) how the ability to iterate supported student agency. In addition, in these interviews, we explored how these preliminary themes related to the supportive programmatic features that define this experimental program, and specifically, hands on, problem based learning and mastery assessment. Our discussion concludes with initial reflections on how exploratory findings in this pilot study can inform future programmatic decisions and directions for the larger research project.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Theoretical Framework: Growth/Learning Mindsets</head><p>This work leverages learning mindset theory with an emphasis on perseverance through challenge in order to explore how students' conceptions of iteration relate to their beliefs about learning. Students hold beliefs about the nature of intelligence that exist on a continuum between holding a fixed or growth mindset <ref type="bibr">[13]</ref>, <ref type="bibr">[14]</ref>. Someone with a fixed mindset believes that their intelligence is a static, unchanging element of who they are, while someone with a growth mindset believes that their intelligence can change and be developed through practice. These mindsets are initially developed through implicit messaging and are held subconsciously but have been shown to be changeable <ref type="bibr">[15]</ref>- <ref type="bibr">[17]</ref>.</p><p>This work focuses on how student's approach iteration, consistent with their beliefs about intelligence through challenge and effort. Students with a fixed mindset are averse to engaging in challenges or practices that require effort as they feel this is a reflection of their own limited knowledge and avoid situations where they may make mistakes or fail <ref type="bibr">[7]</ref>, <ref type="bibr">[18]</ref>. Those with a growth mindset are more likely to engage in challenging tasks as they feel the effort is in harmony with learning where mistakes are further opportunities to learn <ref type="bibr">[18]</ref>, <ref type="bibr">[19]</ref>. For this study, the process of iteration is operationalized as a part of failing and trying again towards incremental learning. When learning is not an initial success, students engage in iteration as they work towards mastery through repeated practice. This theoretical framing is the context for this study's exploration of how students experience iteration, when iteration through design and learning is embedded as part of a problem-based, mastery-assessed program.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Methods</head><p>This study includes preliminary qualitative data collection and analysis of the experiences of four students who were enrolled in a semester-long, experimental, hands-on, problem-based, and mastery-assessed engineering program. An exploratory approach was taken in this pilot phase to clarify the nature of the research, determine research priorities, and collect data to narrow down on the novel elements of this work <ref type="bibr">[20]</ref>, <ref type="bibr">[21]</ref>. Quality was considered and upheld through reflexive engagement in the research design as well as in making and handling of the data <ref type="bibr">[22]</ref>. Through in depth, semi-structured interviews, researchers explored students' mindsets and understanding of learning, failure, and iteration as part of the learning process. The protocol also explored student perceptions of their learning mindsets in relation to the key program elements, including assessment practices and the hands-on, problem-based nature of the program. This data was analyzed using thematic analysis to identify three main themes and selection of specific programmatic features that supported growth mindsets. Data were collected onsite by a single author, but all interviews were coded and analyzed by two authors to check consistency in inferences about emergent themes.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Institutional Context and Participants</head><p>This pilot study was conducted at a small, mid-Atlantic, private college (Elizabethtown College) that is starting up a new "incubation" location in Vermont (The Greenway Center for Equity and Sustainability in Engineering). The engineering program at the incubation campus was designed around evidence-based practices with proven effectiveness at supporting the success of under-represented students, including:</p><p>&#8226; strong mentorship,</p><p>&#8226; hands-on and problem-based learning,</p><p>&#8226; supportive and mastery-based assessment, and &#8226; a mission-driven focus (sustainability)</p><p>&#8226; close connection to business and industry applications</p><p>Learning was organized around a series of projects with real world applications. Each of these projects was multi-disciplinary and weres designed so that students would have to demonstrate mastery of outcomes from multiple disciplines in order to successfully complete the challenge. Students were challenged over the semester to demonstrate mastery of clearly defined outcomes but had flexibility around how and when to demonstrate mastery, as well as the flexibility to try again when and if they were unsuccessful.</p><p>In the pilot "design" phase, the program had four participants enrolled, all of whom were treated as "co-creators." and all of whom were interviewed as part of this pilot study. Due to the very small size of this population, this work in progress paper does not seek to provide generalizable findings, but rather as initial exploration on how students describe the impact of a mastery-assessed and problem-based learning environment on their perception of failure, their reported sense of agency, and their understanding of learning as iteration.</p><p>Because of the very small sample, identifying demographic information has been redacted and gender neutral pseudonyms (Alex, Jordan, Riley, and Taylor) were assigned by the researchers.</p><p>While the larger goal of this work is to scale this program to serve more students, this is a pilot study whose preliminary findings are intended to initiate discussion, identify promising themes for future work, and to influence future program development and direction.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Data Collection</head><p>To explore participants learning mindsets in relation to iteration in their program, this work used semi-structured interviews. This interview approach is particularly suited for exploratory work as it balances pre-set questions to keep the data collection focused with spontaneous questions to explore, deepen understanding, and clarify answers to earlier questions <ref type="bibr">[22]</ref>. Interviews were conducted by the first author during the latter half of the fall semester and were audio recorded before being transcribed by Otter.ai (Otter.ai Inc, 2023) and edited for clarity by authors 1 and 2.</p><p>Guiding interview questions were derived from theory and prompted participants to reflect on how they were assessed, their own learning of course content, and their perceptions of failure and risk in the program. The questions relevant to this preliminary work are presented in Table <ref type="table">1</ref>.</p><p>The semi-structure nature of these interviews allowed for additional content-oriented follow-up questions that permitted a better understanding of how these concepts were or were not supported by specific programmatic features. </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Data Analysis</head><p>The six steps of thematic analysis outlined by Bruan and Clarke <ref type="bibr">[23]</ref> were leveraged to identify major meaningful patterns in participants' mindsets and interpretation in their engineering programs. First, researchers familiarized themselves with the data through re-reading transcripts, listening to the audio file, and making initial notes on the data <ref type="bibr">[23]</ref>, <ref type="bibr">[24]</ref>. Second, transcripts underwent a round of exploratory coding in which structural codes and open codes were simultaneously applied. Structural codes were derived from the theory guiding the larger study and were used to index the data and provide a theoretical overview <ref type="bibr">[23]</ref>, <ref type="bibr">[25]</ref>. Open coding was included to capture nuances of the experiences that were not adequately captured by the predetermined structural codes or to highlight elements that emerged through participant responses. In this work, coding was used as a heuristic tool that goes beyond labeling data but to link or connect data to data, data to ideas, and ideas to ideas <ref type="bibr">[25]</ref>.</p><p>Third, researchers searched for initial themes in the data by focusing on similarity and overlap between codes. A pattern coding pass was applied to collapse and cluster codes together <ref type="bibr">[23]</ref>, <ref type="bibr">[25]</ref>. Fourth, researchers reviewed the various themes that emerged and discussed whether they were themes, codes, or actually nuances of the same thing and should be combined. Fifth, these codes were defined and named before finally being written and described as the results of this study <ref type="bibr">[23]</ref>. Three major themes were identified that focused on how students experienced their learning through the redefinition of failure and how they perceived risk. These preliminary themes are detailed in the following section along with a description of the programmatic features participants identified as supportive in these themes.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Preliminary Results</head><p>The most significant emergent theme in this exploratory study was that students in this mastery-assessed, hands-on experimental program described the role of failure and iteration using language that was consistent with the theoretical construct of growth mindsets. Consistent with that theory, they described learning as iterative and ability as something that grows with practice. In addition, students identified the work of experimenting, failing and iterating in learning as the same process they expected to engage in as engineers in the workplace. The multidisciplinary nature of the projects appeared to contribute to students' perception that the work of learning in this program was similar to the work students expect to do as engineers. These preliminary results present the three emergent themes centering on reframing failure as par of iteration, the redefinition of risks as challenges, and iteration in support of agency. This section concludes with a brief summary of specific programmatic elements the participants identified as supporting iteration in this work.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>From failure to iteration</head><p>Mastery models shift the emphasis in assessment from demonstrating success on a certain day at a certain time to demonstrating mastery of specified content and skills as they are mastered. In this context, time and the opportunity to reassess were supports that the participants identified when redefining failure as an integral part of iterative learning.</p><p>Students identified this freedom to try again if needed as key to their willingness to persevere. As Alex said: "That changes my experience of learning, because number one, I am not afraid to fail. Number two, I push myself to actually understand the concept, because I am not just trying to memorize the definition out of a book." Alex contrasted this with "traditional" school environments, in which "it's very hard because if students are struggling in one subject, and they feel like they aren't given the chance to really master it, then they give up and they don't try to move forward with it." Alex was representative of the group in suggesting that an assessment environment that rewarded effort and persistence reduced student anxiety around grades and performance, and in particular, failure on those metrics.</p><p>Students reported that mastery assessment reduced risk and gave students time to develop deeper understanding in ways that traditional approaches did not. The ability to take time to practice when needed gave students a greater sense of control over their learning process, and in turn more confidence that they could master course outcomes.</p><p>As Jordan clarified "In a traditional environment I don't have the ability to put something on hold and come back to it later when I'm ready like I do at [college]. So if I'm having trouble in a subject I've got very limited time with the non-mastery based approach. So I've got to figure it out then and there. And I don't think that's the best way to learn. Sometimes you've got to let something sit to learn a bit more before you can come back with a fresh mind. That's harder to do with traditional learning." Like Alex, Jordan observed that failure is in fact an opportunity to try again: "In [college], it just means that I've got to try something new. Because if I'm working on a project and a design I have failed, especially this happened a lot in the truss activity we're doing with the bridges. We've had a lot of designs that didn't meet the criteria that were failed designs. But instead of seeing that thinking, "Oh, I can't do this. This is hard." I just looked at it and said, "Okay, I've got to go back. I've got to find another design to try and see. To test something new and see if that works." So failure at [college] is really an opportunity to try again. It's an opportunity to get better. It's not something that's punished. It's not something that's necessarily bad, it just means there's more work to be done." All four students explained that the project-based nature of the program supported their experience of learning as an iterative process. In particular, they all described learning the most when things did not go as they expected, including when they "failed." All students in this sample reported redefining failure in ways that supported continued growth. As Taylor said "if you get it wrong, you didn't do anything bad. It's a chance to learn." Moreover, "failure" was how students described figuring out where mastery was incomplete. Students described moments of 'failure' as the moments when they figured out where they needed to focus their further efforts to learn. As Alex said, moments of 'failure' were pivotal "...because then I've got to figure out not just what I can do to fix the plan and figure out why it went wrong. If it's wrong, I've got to figure out if there is anything else that I have to change because what I was dealing with isn't going to work, and then figuring out how I can make it work. And with [college] being so project-based, it's very often for me to be in a situation like that where I have to figure out how I can improve something and how I can make things work. And it's great, because that's where I'm doing the most learning.</p><p>Taylor described having things go wrong-or failing-as a support to learning. As Taylor explained: "The ideas and goals [college] strives for &#8230; is that we're able to apply that math in a project, hands-on, and also be able to make mistakes and learn from them. Both of our teams had difficulties with the math in particular and in the calculations, and we were able to go back to our knowledge and be able to fix them and learn from that&#8230;from this process."</p><p>Taylor explicitly redefined failure as an opportunity to learn, saying</p><p>Failure is &#8230;if there is a word that has kind of changed meaning, especially since this program and in this context, it does not mean that I have done something bad or wrong. It just meant that I didn't get the expected outcome that I wanted. And it's a way to learn. you don't learn from succeeding on the first try. It Just means that you know that. Failing means that you have an opportunity to grow, and then being able to take that and continue on with it through mastery-based learning turns a failure into a step toward success. When asked what was the most important thing they had learned in the semester, Riley said: "Probably the mindset that failure is okay. The fact that time management is super duper important. The knowledge that I can jump into something new and learn it." Failure was helpful because students could strategically manage their time in a mastery-based context to maximize their success. Having the time to persevere to a solution, without time pressure, left Alex more confident about mastering new content. In turn, operationalizing learning as iteration made Alex more self-confident. Alex said: "Before starting at [college], my definition of failure was that I'm dumb or I can't achieve something. But here my definition of failure is I'm not there yet. And the key word is yet. It kind of allows me to see that progress is like steps. It's not like a dead end road. Just because you made it halfway up the steps you still don't see the top doesn't mean that you should stop. And so it's definitely changed my view to where I see failure as progress." Flexibility in time thus functioned as a support in and of itself, encouraging students to take time in iteration and practice. Students valued the opportunity to take more time when needed, to ensure they had mastered information before moving on. Without this flexibility to adjust pace to needs, students might not have so clearly framed failure as an opportunity to learn. As Jordan explained: "I do like the whole idea of "we're not grading you on whether you knew this in time for the test." It's just "if you know the material by the time the class is done. If you learn it, then you'll get credit for it." Because it takes everybody a different amount of time to learn things, some people really struggle with some topics. And maybe they need more time than a normal class with standard tests would really give them. But with a mastery-based approach, they can take that extra time they need. And when they actually know the material, they can go and they can take the test, or they can get assessed on it in whatever manner the class uses. And that if they can demonstrate their knowledge, they'll get the points. And they're not being graded on just did they know it, but did they learn it in a week?" Like other students, Alex also embraced the idea that using failure as an opportunity to accelerate learning is essential not only to learning, but to success as an engineer. As Alex concluded: "I'm not afraid to fail&#8230;Not being afraid to fail is very important as an engineer, because to reach a solution, you have to be able to go through many different obstacles. And those obstacles can be completely foreign, to whoever's working on that problem. And so when you are failing at these solutions, you're actually finding new ways to solve the solution and weeding out the ways that don't work and the ways that do work. And that way you can properly buy the structure that works for you and create solutions."</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Practices that redefine risk as challenge</head><p>All four students spoke clearly and specifically about the fact that the hands-on projects and mastery assessment both helped them understand learning as an iterative process AND redefined failure as an opportunity to learn. The nature of hands-on problem solving both provided students with immediate feedback on their mastery, as well as repeated lower stakes opportunities to try again. In particular, students reported that having both clarity about outcomes but also time to practice and try again in an applied way supported deeper learning.</p><p>Having the opportunity to try again changed how students understood the challenge that accompanies learning something hard or new. After describing the experience of struggling to figure out how to build a circuit in class, Alex observed:</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>there's a lot of people who don't get it immediately. And you have to just not give up, just keep doing it. And then you get it. Which improved my confidence a lot. Because now I realize that it's not that I don't understand something, it's that I've given up. I didn't push myself where I needed to be pushed."</head><p>The opportunities provided to try again and continuously improve in this mastery-based setting reinforced the idea of success as the product of effort and iteration, which in turn made challenge feel less like risk, and more like opportunity.</p><p>Similarly, Taylor reflected that "no one's really an expert on everything, but I know I have the skills to do it, and that I can go out there and do it. So very confident in the abilities and picking up like skills and working towards, like higher levels of ability, specific areas, growing skills</p><p>Taylor stated being very confident taking risks to learn, noting that "The only thing that's really held me back in regards to taking larger risks would be the outcomes and time constraints. I have a short time here. And, I have some goals I need to hit. And so I want to make sure that everything gets done, and any unwanted surprises would disrupt that a little bit. But sometimes you want to have little surprises or issues along the way. Because it's good for learning. Risk taking facilitates learning. Making mistakes facilitates learning." In other words, the power of a mastery approach is that it gives students more time to demonstrate growth and relearn as needed. This reduces the perceived risk of tackling a challenging problem. As Taylor added: "I just feel that risk taking here is not at all discouraged. I'd say it's encouraged, where even things that may seem unorthodox or not in a specific direction are&#8230; very much praised. I could do something differently or take a little risk with it, saying "Hey, I'm gonna go out and do this thing" and that's fine. I can do that." This reduced risk associated with learning also left students with more positive feelings about learning in general. As Alex said: "it's definitely a different type of learning. I don't want to say I learned more, because I mean, I could learn in a traditional setting, it's just you apply yourself very differently in both. So with this type of setting, it's different in the sense that I'm not as stressed. I feel more open to explore and learn. I feel like it's very playful and I don't feel pressured to get an A. I just feel like I want to absorb information. And there in a traditional sense, I just kind of feel pressured to get an A, I'm going to learn what I need to learn to get that A and retain as much as I can but I'm going to probably be super stressed the entire time and feel like I'm kind of stuck in a box the entire time."</p><p>Several students echoed this sentiment saying that iteration and reassessment gave them the freedom to explore their own interests as they applied to coursework. In addition, in the supportive context of the mastery approach, students like Jordan were comfortable with struggle, because they expected struggle to lead to success. As Jordan said: "if you're struggling with something that means you're getting better at it. Because if it was easy, then we wouldn't need to take the class."</p><p>The reduced risk students attributed to challenge made students more willing to take on risk and challenge. Taylor reflected on "how failing allowed me to succeed in greater ways than I would have if I had gotten it right the first time." Taylor gave the example of trying to work through a challenging math concept, and having trouble. They explained:</p><p>"So I went back to try it a second time. And while I was going through and taking it, it was going through my head and then I hit this moment where it came back into a realization, where I was able to actually visualize how the math played a role in, actually like, you know, engineering and the physical World, which allowed myself to visualize the math directly and go through and be able to finish the quiz&#8230; If I got it right the first time I never would have really had my brain to take the time to think about how that plays a role in what I do."</p><p>Similarly, Riley observed being very comfortable taking risks in their program. Riley stated that the way faculty employed semi structured problems, they supported risk taking by students by designing programs in ways that students have to stretch to address. They stated: "The instructors help in that way, because they sort of purposefully don't give you&#8230;.They don't just lay out the strategy to do a problem or give you&#8230; and they don't even give you every tool to solve it,...they make sure that you have to dig a little bit, which sort of makes it stick in your brain better. So that kind of in itself makes the students have to take risks. And then personally, I guess I don't mind making something hard for myself to make sure that I try to do it more."</p><p>One of Taylor's final observations was representative of the comments of all students: "&#8230;through this, I was given the time to learn from failures, I was given the opportunity through the mastery based learning to just take the time to figure it out myself..now I have personal direct experience with how that leads towards success. So it's strengthened me."</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Iteration and agency</head><p>The purpose of this experimental program was to identify whether coordinated use of evidence-based strategies would support better outcomes, including persistence in engineering, student agency and engineering identity. In this small sample, all students reported that success at iteration strengthened their sense of both agency and engineering identity.</p><p>Taylor experienced the emphasis on mastery and iteration as a shift from focusing on grades and measuring up, to focusing on deeper learning and application, saying that previously: " ..I was focused on getting a grade&#8230; I feel that doesn't really help me. Here I'm focused on skills and I'm focused on applying them and less on a grade, and feel that any grading .. any letter grading in particular kind of hampers my ability to focus on learning skills and retaining them instead of just getting it right and moving on to the next thing."</p><p>Taylor went on to explain that the experience in the experimental program had "boosted my confidence with myself..That I know that I can take my time to achieve what I want to achieve. And I know that I am allowed to make mistakes&#8230; that no one's perfect, and that's to be expected." In other words, developing mastery through iterative learning left Taylor feeling better prepared and more confident about approaching future challenges as an engineer.</p><p>Alex similarly defined this process of failure and iteration as central to success as an engineer, stating: "..as an engineer, to reach a solution, you have to be able to go through many different obstacles. And those obstacles can be completely foreign, to whoever's working on that problem. And so when you are failing at these solutions, you're actually finding new ways to solve the solution and weeding out the ways that don't work and the ways that do work."</p><p>While Taylor emphasized that repeated iteration boosted confidence, Alex normalized the disciplined process of working through challenges not as failure, but as the process by which engineers achieve success.</p><p>Like others, Jordan also emphasized the confidence and sense of agency that comes with successfully iterating through challenges to success. Jordan described this process as "practicing engineering," effectively internalizing iteration as the work of a successful engineer:</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>"I think I'm a lot more confident than I was going into it. Especially with the projects we've been working on every morning-it gives me an idea on what engineering is actually like and what I would be doing on a daily basis if this was my career. I'd still like to make it my career, I like what I've been doing&#8230;. And because I was actually practicing engineering, instead of just learning the skills, and that definitely helps with confidence."</head><p>Student agency was also evident in how students talked about applying their confidence with iteration and problem solving in the future as engineers. As Alex summarized: "Defining my own problem is very important to my sense as an engineer, because to be able to solve problems that haven't been solved yet, you have to be able to ask the questions that haven't been asked yet." There is nothing rote or scripted in this conception of engineering; Alex has already moved on from contemplating the idea that there is a fixed set of skills and solutions to master, to recognizing the importance of continually having to learn new content and skills.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Program elements that support iterative learning</head><p>All students described the hands-on and project-based nature of the program as conducive to better learning. For example, Jordan mentioned that project work in particular challenged students to stretch, saying "In general, struggling is a sign that if you're learning at least you have something to learn. But at [college] it's really been highlighted especially with the projects, when I'm working I'm doing something, but I know I'm learning. Also it's not necessarily easy. And I do enjoy struggling a little bit." Jordan explained that because the projects approximated the work of engineers, and because Jordan was able to meet and interact with other engineers working on the same types of problems, the work of learning felt like an extension of the kind of problem-solving students would do as professionals.</p><p>Similarly, Alex named a connection between the kind of learning they did on projects and Alex' comfort and sense of anticipation to go to work as an engineer: "I know I'm still a student and I still have a lot to learn. But the push to learn and to apply is not scary anymore. So I feel very motivated to go out and be an engineer. I'm very motivated to go out and see what I can do with this knowledge and where I can apply it. So [college] is very much pushing me." Alex explained that they became more comfortable with challenge and with experiencing failure, because they knew they would have opportunities to relearn and reassess as needed, to reach higher levels of demonstrated mastery.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Initial Discussion</head><p>Learning in this experimental program was organized around hands-on, problem-based learning. The program used a mastery-based approach to capture evidence of student mastery of outcomes, and students had flexibility in how and when they developed and demonstrated mastery. They also had the opportunity to iteratively relearn and be reassessed when they initially struggled or were unsuccessful.</p><p>In the context of this pilot study, students described these program components working in concert to create a supportive learning environment in which students were motivated to persist through challenge, redefine failure as an opportunity to learn, and experience learning as an interactive process. In turn, this supportive and interdisciplinary context also left students feeling more confident and less stressed, such that they redefined risk and challenge as essential drivers of learning. Moreover, they framed iterative learning as similar to iterative design in engineering; they extrapolated from their experience working on real world applications in the classroom to the work they expected to do as engineers in the field.</p><p>The participants in this study also described how their conceptualization and approach of failure changed as they realized this was a mechanism by which they learned. As they reflected that some of their most powerful learning occurred when their efforts did not succeed or work as expected, they described this "failure" as an essential part of checking their understanding and practicing what they needed to master to thrive in work in engineering. Because the stakes of failure were lowered in the mastery-assessed context, they were willing to "fail" in order to learn, and knew that if and when they struggled, they would have time to try again. Thus, while most admitted some nervousness at the start of the pilot semester, they all described developing more of a growth mindset, characterized by iteration through challenge, and in which willingness to risk failure was an essential step in finding good solutions.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Implications for future work</head><p>This exploratory pilot project examined how sophomore engineering students in a mastery-assessed, hands-on and problem-based learning environment experienced iteration through failure, and how they described the effect of iteration through challenges on their confidence and sense of agency as engineers.</p><p>Because of the small sample size in the pilot, this analysis should be repeated on a larger more representative cohort to better understand how program elements shape students' experience of iteration and failure. The findings of this study are intended to guide future research, influence program design, and support discussion in the community.</p><p>Aligning with the goals of a work in progress paper, this preliminary analysis suggested several promising topics for future investigation. For example, one unexpected finding was that students reported that while they felt challenged, they experienced much lower levels of stress and anxiety than they did in other learning contexts. Future research might explore whether the particular program elements in this pilot created a supportive context that reduced anxiety and increased the confidence of students, including underrepresented students, such that they are more likely to thrive in engineering.</p><p>Future study could also explore specifically how students think about iteration as an engineering practice. In this study, students stated that they felt that although they were still in school, they felt like they were already working like the engineers they saw in site visits, and working on the same kinds of problems. In other words, in this kind of hands-on and mastery assessed environment, do students experience and recognize the work of learning, iterating and coping with ill -structured problems as a form of cognitive apprenticeship in engineering?</p><p>This study also raised interesting questions about the difference between having choice in a program and being agentic (control over learning) within a mastery-based learning. Students discussed appreciating having control over how and when they were assessed, which essentially is about choice in how they achieve curricular goals. However, they also discussed how important it was to them to be able to set their own goals and design their own challenges, and describe this as the work of practicing engineers. Do these two types of experiences have a substantively different impact on students' emergent sense of agency?</p><p>Finally, this study suggested that students came to understand iteration, problem-based learning, and failure as defining characteristics of successful engineering. Further research can explore whether and how students who embrace iteration and persisting through failure are associated with greater confidence and success as engineers.</p></div></body>
		</text>
</TEI>
