Science and technology drive innovation, foster economic growth, and are critical to national security. Indeed, the World Bank posits that the number of domestic Ph.D. graduates is indicative of a nation's future innovation potential (Velez Bustillo & A. Patrinos, 2023). Historically, the United States's higher education institutions have relied upon global Science, Technology, Engineering, and Mathematics (STEM) talent to fill engineering and computer science graduate programs. While the number of US students enrolling in computing undergraduate programs is expanding, only a small portion of these students continue on to graduate programs in computing (DesJardin & Libeskind-Hadas, 2021). Because of increased global competition for skilled STEM workers and various barriers along all segments of the STEM pipeline, meaningful and sustained action is required to expand US student engagement in STEM and to specifically address the long-standing underrepresentation of marginalized domestic US communities (Gershenfeld et al., 2021). The National Science Foundation (NSF), in fact, states that expanding access to undergraduate research experiences (REU) is an immediate priority (DesJardin & Libeskind-Hadas, 2021). The Computing Research Association (CRA), moreover, reports that undergraduate students with formal research experience (REU) are twice as likely to pursue graduate studies (Tamer, 2019).
In 2006, as an effort to provide these recommended early research experiences for students, Carnegie Mellon University's Robotics Institute launched a summer undergraduate research robotics program, the RI Summer Scholars Program (RISS). Over time, RISS became a platform to design, pilot, and evaluate educational interventions and learning approaches that could increase student interest and launch students into graduate school. An analysis of student outcomes showed that more than seventy-five percent of RISS alumni from 2012 to 2018 pursued graduate studies in STEM. In this paper, we discuss how the reimagining of CMU's RISS program's undergraduate research learning community structure and goals, which was informed by David Kolb's Experiential Learning Theory (ELT) and learning styles, increased student participation in STEM graduate programs by focusing on the individual learner and peer experience. We use RISS as a case study to explore how coupling educational frameworks more closely with REU programs can improve student researcher outcomes and continuation in STEM studies.
Ensuring that more US students successfully enter graduate STEM programs is essential for our national security and therefore must be a significant part of national STEM education and workforce policies (Alper & National Academies of Sciences, Engineering, and Medicine (U.S.), 2016). US STEM graduate programs at top-ranking research universities, however, are seemingly not as accessible for American students as they used to be. In the CRA study entitled "Addressing the National Need for Increasing the Domestic Ph.D. Yield in Computer Science," the authors noted that the United States has been increasingly relying "on international students to drive innovation and leadership in computing research" (Hambrusch & Pollock, 2020). There is an observable trend of a decreasing ratio of US residents to nonresident students, which rises with each rung of higher education (Table 1). The CRA, therefore, recommends increasing funding and availability of undergraduate research to expand access and participation in undergraduate research programs (Hambrusch & Pollock, 2020). Similarly, the National Science Foundation (NSF) Computer and Information Science and Engineering (CISE) Advisory Committee identifies expanding access to early research experiences as a top strategy to strengthen the national research and innovation pipeline (Figure 1). Another way to help ensure more students are enrolled in STEM graduate programs is establishing REU programs that focus on the learning experience, mentorship, and post-program support. Engaging in REUs has a definitive positive impact on student interest in pursuing graduate studies, as seen in Table 2, making them an essential focus for higher education institutions (Table 2). Clearly, REUs are a highimpact education intervention, but also are not infinitely scalable without a loss of quality and diminishing learning outcomes (Kuh & Schneider, 2008). Note. NSF Computer and Information Science and Engineering | Advisory Committee Report 2019 (DesJardin & Libeskind-Hadas, 2021)
Note. The data percentage reflects numbers reported by CRA and RISS for the year 2018. The CRA percentage was calculated on a total number of participants of 793 (Wright, 2020). The RISS percentage was calculated on a total of 35 participants in 2018.
Expanding the availability of undergraduate research experiences aligns with the policy recommendations across national agencies and leading research units to increase domestic participation in STEM. We observe a tremendous gain in interest to pursue graduate studies after participating in a formal research experience (31% versus 15% per a CRA study) (Tamer, 2019). However, more attention should be paid to identifying models and practices that will increase this number to closer to 100%. We are losing too much talent and must discover where the pipeline leaks and ultimately shatters. Reimagining how we think about and design REU programs and learning experiences could increase REU scholar interest and participation in graduate studies.
Launching: Beginning with a Traditional REU Site Model In 2006, Carnegie Mellon University's Robotics Institute launched a summer undergraduate research program, the CMU Robotics Institute Summer Scholars (RISS), to provide early robotics research exposure. RISS followed the traditional undergraduate research experience (REU) site model, coupling experts (science faculty) with undergraduate students to enable involvement in a research project, providing communications and graduate school preparation workshops, hosting social activities and an end-of-program poster session. The initial RISS program model reflected the traditional REU model: short in length, projectfocused, typical closing activities, the cliff of engagement after the program, and no longitudinal data studies. Reimagining the REU Model We established the REU program with good intentions, a strong commitment to broadening participation, and outstanding scientific content expertise to guide students. However, we soon observed that the traditional 10-week summer undergraduate research program model (henceforth referred to as the traditional REU model) artificially limited the impacts and outcomes for participating students, faculty, and organizations and could even cause harm to students from communities under-represented in STEM. The traditional REU model is often structured in a banking-style model, with the research mentor assigning articulated problems without an intentional framing of the larger problem, its importance, or its impact. The compressed time frame does not allow sufficient exploration of mentees' prior experience, interests, or knowledge. REU experiences can be isolating when individual scholars are matched with an individual mentor or small lab without the benefit of multiple layers of mentorship and peer engagement to cultivate a reflective practice. Because of the traditional REU model's short time frame and limited post-program engagement, the model inevitably yields a high-stakes race to produce a poster or paper rather than providing an opportunity to thoughtfully examine the research's interactive process and collaborative nature. A Guiding Framework: Experiential Learning Theory (ELT) Reimagining the undergraduate research program model is central to developing approaches and practices to effectively increase awareness, access, and participation in robotics. To further this goal, we began with a study of education learning theories and evidence-based student development practices that could be applied to the summer research experience. Experiential learning, dialectical method, peer engagement, and research on the science of effective mentoring in STEM emerged as promising approaches to address long-standing underrepresentation from the traditional science establishment and transform the traditional undergraduate research model. Ultimately, we used David Kolb's Experiential Learning Theory (ELT) as a compass to guide the reimagining of our traditional REU model. David Kolb's Experiential Learning Theory (ELT) presents a framework that supports developing learning identities, flexibility, and resilience. ELT presents learning as a process of creating knowledge through experience and interacting with one's environment. ELT, with its grounding in social justice, democracy, and constructionist theory, strongly influenced the development of Carol Dweck's seminal work on growth versus fixed mindsets. Through a four-part learning cycle (concrete experience, reflective observation, abstract conceptualization, and active experimentation) (figure 2), the learner grasps and transforms experiences. The learner's unique approach is influenced by what Kolb presents as the nine learning styles. According to Kolb, "learning styles are influenced by culture, personality type, educational specialization, career choice, and current role and tasks" (D. A. Kolb, 2015). Drawing upon Kolb's ELT, the RISS program seeks to create a consciousness of one's learning process, help participants develop intentional learning goals, and create a positive learning environment. As described by Kolb, an ideal learning environment should seek to create psychological safety, foster empathetic engagement with and between peers and mentors, provide positive reinforcement, and actively guide the student through the learning cycle (experience, observe, integrate, experiment). This process helps students develop a strong learning identity, better navigate barriers, develop a sense of belonging, reduce imposter syndrome, and ultimately persist in STEM (D. A. Kolb, 2015).
Critics state that Kolb's experiential learning theory can be overly simplistic, may fail to reflect global cultural learning traditions sufficiently, and crowds out other learning theories. However, for our purposes of developing metacognition, empathetic mentoring, and a safe learning environment, ELT, its underpinnings, and its learning cycle provided a clear compass for reimagining RISS's purpose, approach, and process for organizers and participants. Note. The graph represents David Kolb's two continuums. The horizontal axis is called the processing continuum and the vertical axis is called the perception continuum (D. A. Kolb, 2015).
Kolb also stresses the importance of the learning space attained through creating safety, integrating multiple forms of supportive mentorship, facilitating peer engagement and dialectical methods, reducing the risk of failure, and showing the relevance of the activities and framework. The learning space and approach must be adjusted to meet the learner's needs and acknowledge their prior experiences, cultural backgrounds, and identities. The RISS program reframed its purpose to nurture future roboticists from all backgrounds.
Based on Kolb's dimensions of learning space (Figure 3), RISS, iteratively, has embedded the following elements to make the learning proactive and continual:
• Creating Safety • Ensuring Positive Mentorship • Facilitating Peer Engagement & Learning • Reducing Risk of Failure • Communicating Values • Showing Relevance of Process and Outcomes • Creating Space for Reflection
The above-mentioned elements form a guiding checklist to quickly identify gaps in each workshop, process, and experience. Additionally, it allows organizers to modify the language, actively model values and group norms, and place the activity within the Scholar Development Roadmap. Note. The figure represents David Kolb's Dimensions of Learning Space. The five dimensions come together in the experience of the learner (D. Kolb & Kolb, 2013).
Our approach was reframed to facilitate high-quality immersive undergraduate research experiences, professional development, inclusive mentoring, and service learning that can transform lives and open doors. Exploring David Kolb's Dimensions of the Learning Space, we created four distinct phases to the program. The first phase is pre-arrival (setting the foundation, developing a group agreement, use of intentional language, and discussing learning. The second phase is the summer research immersion (reinforcement of group values, norms, and intentional use of language including scholars in the theory). The fourth phase is the wrap-up and launch where we use intentional language to ensure support, discuss iteration, share examples of resilience, and provide one-on-one support. The fifth phase is the post-program support, which has focused on graduate and fellowship application support.
Our vision of engagement is to begin before the program and continue post-program to surround each student with support, just-in-time interventions, and ongoing coaching. Yearround RISS activities focus on education, access, and engagement to co-create expansive and inclusive learning opportunities to support scholar development. Post-program support is essential for processing the summer experience and integrating new knowledge, paths, and ways of being into our individual identities and pathways. However, this fifth phase of scholar engagement is the least developed, funded, and studied.
During the summer portion of the RISS experience, the RISS team, alums, and community of advocates work together to enable students to 1. strengthen their academic foundation (knowledge, skills, and build an influential network of robotics innovators), 2. explore using robotics to impact the world, and 3. craft personalized learning and career roadmaps with multiple pathways and opportunities in robotics.
The 100-plus RISS mentor community that enables this high-touch experience now draws from faculty from the Robotics Institute, Mechanical Engineering, and Electrical Engineering Department, current graduate students, alums, and leaders from across campus and the Pittsburgh community.
Multiple layers of mentorship and engagement in diverse research discussions have been essential to supporting the scholar's identity development. Further, a major aspect in the reframing was the inclusion of research methods and approaches in the discussions (e.g, direct discussions of expectations, approaches, failure, reframing, and iteration on a regular basis). This has been critical in helping the scholar learn and address the objectives and outcomes of their research with active feedback. Consequently, this created a unique space for researchers (mentors and scholars) from across different experience groups to form a freeflowing synergetic collaborative dialogue space where scholars were able to not only learn but also impart their learnings to their fellow peers in other research projects.
RISS is about discovery, growth, and learning. Therefore, we have reframed the traditional REU products and research posters as learning opportunities, not the goal or measure of success. The research products are artifacts that demonstrate scholars' experience, showcasing their research experience and contributions. This reframing combined with the use of intentional language describes the learning experience and exploration as the most important outcomes. This approach encourages active experimentation and iteration rather than creating high-stakes environments in which students are paralyzed by the fear of failure. The greater gain and value is the learning of the subjects, self, and community.
Combined, this framework and the scaffolded experiences allow the scholars to practice key elements of being a researcher or academic in a guided, safe environment with space for reflection, experimentation, and iteration. Table 3 shows a major finding from the ELT case study among students who attended RISS between 2012 and 2018, nearly 80% ended up enrolling in graduate and doctoral programs and 19% of the remaining ended up in the STEM workforce. Thus, establishing the effectiveness of applying the ELT lens in constructing the REU learning space. Note. The stacked bar graph is a breakdown of post-RISS outcome categories from (Table 3) for each year from 2012 to 2018. At the top of each bar, the total number of RISS participants is listed.
Undergraduate research experiences represent a tremendous resource and opportunity to strengthen the national research pipeline and expand the participation of students from underrepresented backgrounds. Expanding access, increasing funding, and studying learning models that support scholar development are essential for the nation's future health, security, and prosperity. However, we need more than expanding access to realize the full potential of these programs. The traditional REU program model can be transformed by adopting a student-centered approach, more effective and supportive mentoring, and systematically applying multiple layers of experiential learning theory. By becoming practitioners and students of ELT design, student outcomes and our understanding of effective learning environments will strengthen.