Prototypes are representations of ideas. They are not always elaborate depictions, and they do not always contain all the fine details of the design. In fact, features in a prototype do not always appear in the final design. However, prototyping, as a process, is an act of externalizing design thinking, embodying it through physical objects. Therefore, studying prototypes and prototyping is one way to studying design thinking. "The reason for prototyping is experimentation," explains Tom Kelley and David Kelley of IDEO, the innovation company: "[is that] the act of creating forces you to ask questions and make choices" [1]. Prototyping provides insights into the design problem that other approaches to design cannot provide.
In academic settings, an increased focus on innovation and entrepreneurship drives the need for project-based learning (PBL), where students can build artifacts, test them with users, and improve their ideas. As early as 1974, the Aalborg University in Denmark was the leader in being founded on the pedagogy of PBL [2,3]. In the US, the National Science Foundation called in its 1997 report, Systemic Engineering Education Reform: An Action, for emphasis on PBL, along with teamwork, and partnership with industry. However, and despite these early efforts, research questions are still open as to how effectively teach, learn and practice design [4]. This paper presents the research design of an exploratory, pilot qualitative study, based on the research question "What are students' conceptions of prototyping in engineering design?" The paper starts with reviewing the relevant literature on the theoretical framework of prototyping in design. Then, the research design rationale is described, followed by the interview protocol development, and participant selection. In addition, data analysis method and preliminary coding are discussed. The implications for this study are described next, with highlights to the future of this work in-progress. This study adds to our understanding of how students conceive of prototyping, which may improve scaffolding activities in design education.
There are different aspects to studying and effectively using prototyping. In the following review of prototyping in design, the relevant literature is organized under three major areas: the role of prototyping in the design process; the role of prototypes in communication between team members in collaborative projects; and the role of prototypes in providing feedback for designers.
According to various studies, prototyping is considered to play an essential role in the design process [5][6][7]. For example, the process of prototyping can bring design issues that alternative approaches to design cannot [8]; can enable designers' hypothesis testing [9]; and facilitate the communication within the design team while keeping the client involved in the design process as well [10].
There are three major ideas about the role of prototyping in the design process. First, there are different kinds of prototypes as interactive artifacts. Second, different perspectives exist as to when prototyping becomes the focus of designers in the design process. Third, there are differences in prototyping behaviors between experts and novices. These three major streams of research are briefly introduced below.
What kinds of prototypes are there?
It is important to note that for the purposes of this research, a prototype is defined as any representation of the design idea, regardless of the medium [11]. In [11], three major aspects were identified that a prototype, as an interactive artifact, can bring to the design process: role, look and feel, and implementation.
When do prototypes appear in the design process?
In [11][12], prototypes were regarded as representation for design before a final solution exists. However, in [13], a holistic framework for prototyping was proposed that suggested prototyping to take place in every step in the design process. Although this model is presented assumes prototyping to take place throughout the design process, in [14] it is suggested that prototyping exhibits a minimalist approach, with many prototyping approaches share the underlying goal to envision the future of the designed artifact [15]. In fact, other research suggests that prototyping should be incorporated early in the design process and used iteratively throughout it [6,[16][17][18].
Research shows that expert designers use different kinds and levels of prototyping during all phases of the design process [19]. Also, expert designers use prototyping to successfully switch between component-and system-level thinking during design [20]. Expert designers exhibit a structured approach for when and how to use prototypes, including awareness of time spent on prototyping and level of complexity [8] and an ability to recall previous knowledge to quickly test ideas [10,21].
In comparison, novice designers lack the ability to scope design problem with sufficient depth and breadth and design strategies while designing [8,20]. Most significantly, novice designers seem to lack the understanding of the dynamic nature of prototyping to help refine and develop ideas [5,8,22].
While prototypes can be regarded as one of the richest approaches to gain insights into the design problem [15] and into future situations [14], prototypes are also used to aid in communication between different stakeholders in the design project. Experts from different backgrounds utilize prototypes as boundary object to bridge gaps of language differences in discussions [23]. In addition, a prototype can be used to demonstrate ideas and persuade a client [6,7]. In [24], the use of prototypes as tools to gain empathy was described.
Overall, in collaborative team settings, prototypes are believed to enhance distributed cognition due to the extension of ideas beyond the individual to encompass the environment, the artifacts, and the people [9,[25][26].
Prototypes can be used as learning vehicles [9,15] as designers refine ideas while interacting with them [27]. The term, "Experience Prototyping" has been proposed to refer to prototyping that enables stakeholders to gain insight of the problem-solution space through experiencing the interaction with the porotype [24]. In this sense, they are the primary feedback provider for designers on their ideas [28].
However, it has been experimentally difficult to isolate and describe the exact impact that prototypes have on the final design outcome [21]. Because of the interaction between though (mind) and action (body) during prototyping [15], embodied cognition theory illustrates the deep learning that takes place when prototyping is used a successful design strategy [29][30][31][32]. Prototypes in design learning is a topic under study [5,22].
The study of prototyping has various aspects. One of the aspects that needs to be explored more is focused on prototyping as it relates to students' learning. While prototyping plays an important role in engineering design thinking, teaching and learning. challenges still exist as to how it can be effectively taught and learned, in order to use it as an integral part of the design process. Studies are lacking to understand students' conceptions; there seems to be a lack of understanding to the centrality of prototyping in design education and learning, especially to explore the problemsolution space, to ideate, and to communicate ideas. While students can be characterized as novice designer, this characterization is loaded with different dimensions of design ability; however, prototyping in design involves less understood areas of research and practice. These include exploring ways to improve students prototyping; using the process as a way to teach and learn design; scaffolding design activities; assessing students work; cultivating reflective practice; and encouraging self-regulated learning. Overall, the aim is to make students' behavior models that of professional, expert designers: learning to build, and building to think. Understanding students' conception is a first step to improve design education and learning.
To study students' conception of prototyping, we recruited students from a sophomore class that teaches engineering design. Taking the learner's perspective, through the students, allows describing their growth to obtain design expertise. The research question we try to answer in this study is: What are the conceptions that students have about the role of prototyping in design thinking? We interviewed three students, and used qualitative research to design this study to answer the research question. Qualitative research has been used to study design thinking and practice [33][34][35][36], because it provides opportunities to understand the underlying concepts, beliefs and motivations of designers [37,38]. Taking a research-informed approach to engineering education allows improving educational practice based on evidence supported by research. The research design follows the guidelines of, and has been approved by, the Institutional Review Board.
In this study, data were collected using artifact elicitation as a method [47]. In this method, students as participants are asked to bring an artifact of their own choice and making to an interview. The interview starts by asking participants to describe what they brought with them: the artifact's features; what does it do; how did they come up with the idea; and where did they learn to make it. Afterwards, the interview moves to ask the participant about other projects and experiences, allowing the elicitation of conceptions about prototyping. The collected data were recorded interviews that were later transcribed for analysis.
As a data collection method, artifact elicitation is a derivative of the photo elicitation method, where participants are typically asked to bring a photo to an interview [39][40][41][42]. The photo enables the participant to physically point to visual cues in it, which can lead to larger discussions related to the research study [43,44]. According to [45], "visual data representing personal understandings of concepts, experiences, beliefs, or behaviors, can be especially useful in helping participants to express complex or abstract ideas or opinions" (p. 513). This method facilitates the critical reflection because it has the "power to capture unfettered visual conceptions of information, especially personal, pleasurable, and profound dimensions, which are increasingly relevant" [46, p. 1350].
While artifact elicitation offers the advantage of providing descriptions about students' conceptions, it also has some disadvantages. As reported by [47], some disadvatages include: the difficulty of coordinating an interview time when it is convenient for a participant to bring the artifact; the challenge of attending to multiple things during the interview, such as the interview protocol, the recording while simultaneosuly being engaged with the participant, and following up with important questions when needed; and being mindful that particpants may become enthused about their project and less reflective in the responses they provide. In order to mitagate some thses shortcomings, the interview protocol was semi-structured so that it moves from the specifics of the artifact at hand, to other contexts such as previous and future projects, and, correspondingly, to more general understanding of conceptions of students' for protyping and the design process.
Artifact elicitation has been used to study Makers [48,49]. In the study in [47], an interview protocol was developed that was based on emerged themes that characterize Makers attributes of knowledge, skills, attitudes, and lifelong learning. In [49], authors described that they "conducted an inductive thematic analysis on the transcribed interviews (generating theory from the data), which fed back to inform questions asked in the interview protocol" (p. 133).
For the purposes of this study, we used a modified version of this protocol. Questions where slightly changed to reflect the nature of the research study. For example, the original protocol asked, "Can you tell me a little bit about what you brought to Maker Faire?" In this study, the question became, "Can you tell me about what you brought to the interview?" Similarly, a question in the Makers study was, "What is your process for designing your invention?" In this study, the question became, "What is your process for designing your prototypes?" In addition, for the purpose of this study, question about receiving feedback where added to this study. Because of the learning emphasis in academia, understanding students' conceptions of feedback on prototypes was important. The questions that were added about learning from feedback were based on the critical incident method, in which participants are encouraged to recall a specific incident relevant to the study and then they are asked to expand on how they see the relevance and why the recalled that particular incident [50]. The additional questions are:
Do you seek feedback on prototypes on models that you make?
What kind of feedback do you seek?
Can you give me an example for a valuable feedback that you have received on a prototype?
How did you use that feedback?
Have you ever received poor feedback on a prototype? What did that look like?
The interview protocol was semi-structured, allowing probe questions so that participants can expand on ideas or contexts they were describing. The questions, however, both the original from the Makers study and the additional ones, were not tested or iterated upon because of the time restriction to conduct and complete this study. The same protocol was used for all participants, with slightly different follow-up questions that depended upon the description of the artifact that each student provided, the students' responses, and experiences.
Participants in this study are students in an undergraduate sophomore design-oriented, project-based learning course in a large southwestern university. Students were invited to participate in a screening survey. Invitations were sent through instructors of the class who were asked to share the survey information with the students in their sections electronically. Following the initial invitation, the main author went to the different sections of the class to invite students to complete the survey, explaining the purpose and process of the study. During the recruitment, the main author avoided using the word "prototype" because it might have different meanings to different students; instead, the main author used words like, "artifact," "object," and "project of your making" when telling students about what to bring to the interview.
To identify potential participants for the study, responses for the recruitment survey were reviewed. Potential participants were identified based on whether they had prototyping experience, either in-class or out-of-class, and whether they could bring an artifact with them to the interview. After potential participants suitable for the purpose of this study were identified, they were invited via email to participate in a follow-up interview. Each interview lasted approximately 20 minutes. Participants were given pseudonyms: Andrew, Benjamin and Charlie. They were all majoring in General Engineering, and they were all in their second year of the program. None of them had professional engineering experience, and the number of project-based classes they took at the time of the interview was three to four classes.
The raw data from the interviews were recorded audio files, which were transcribed for data analysis. For data analysis, we use inductive, open-coding [53,54]. Inductive coding is used iteratively to arrive at patterns, themes and codes. The outcome from the first and second iterations are presented in Table 1 andTable 2, respectively. The two different organizations shed light on two different aspects of the data: in the first, the focus was primarily on the process, while in the second, the focus was on the object.
This exploratory study suggests that there should be ways to link students' understanding of the design process with their practice of prototyping. As suggested in the literature, reflective practice could be one way to do that [29]. Also, students should be given the opportunity to test prototypes to gain insight into their design-problem and solution space. Preliminary findings show that students spend significant time learning prototyping techniques; hence, they should be provided insightful feedback on their activities. Also, interviews show that students start working on projects with the mindset that they are working on a final product. This understanding should be alleviated by intentionally structuring intermediate steps in the design process that result in enhanced understanding of the design problem. Scaffolded prototyping, suggested in [51], provides a systematic way "to support self-regulated learning by offloading feedback from the instructor to students' evaluation of their own built prototype in the context of iterative feedback from a user." Scaffolded prototyping follows Kolb's Experiential Learning Cycle [52], which, in the context of design education, allows students to move reflectively from abstract understanding of the place prototyping has in the design process to concrete understanding, Fig. 1.
Prototyping plays an important role in engineering design thinking, teaching and learning. However, challenges still exist as to how it can be effectively taught and learned, in order to use it as an integral part of the design process. The next step will be analyzing the collected data. In students' conceptions, there seems to be a lack of understanding to the centrality of prototyping in designing, especially to explore the problem-solution space, to ideate, and to communicate ideas. Understanding these (mis)conceptions is the first step to develop learning experiences that allow students to use prototyping to produce high-quality designs.