skip to main content


Title: Interactive Personas: Towards the Dynamic Assessment of Student Motivation within ITS
An intelligent system can provide sufficient collaborative opportunities and support yet fail to be pedagogically effective if the students are unwilling to participate. One of the common ways to assess motivation is using self-report questionnaires, which often do not take the context and the dynamic aspect of motivation into account. To address this, we propose personas, a user-centered design approach. We describe two design iterations where we: identify motivational factors related to students’ collaborative behaviors; and develop a set of representative personas. These personas could be embedded in an interface and be used as an alternative method to assess motivation within ITS.  more » « less
Award ID(s):
1912044
NSF-PAR ID:
10288229
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
International Conference of Artificial Intelligence in Education
Page Range / eLocation ID:
43-47
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Need/Motivation (e.g., goals, gaps in knowledge) The ESTEEM implemented a STEM building capacity project through students’ early access to a sustainable and innovative STEM Stepping Stones, called Micro-Internships (MI). The goal is to reap key benefits of a full-length internship and undergraduate research experiences in an abbreviated format, including access, success, degree completion, transfer, and recruiting and retaining more Latinx and underrepresented students into the STEM workforce. The MIs are designed with the goals to provide opportunities for students at a community college and HSI, with authentic STEM research and applied learning experiences (ALE), support for appropriate STEM pathway/career, preparation and confidence to succeed in STEM and engage in summer long REUs, and with improved outcomes. The MI projects are accessible early to more students and build momentum to better overcome critical obstacles to success. The MIs are shorter, flexibly scheduled throughout the year, easily accessible, and participation in multiple MI is encouraged. ESTEEM also establishes a sustainable and collaborative model, working with partners from BSCS Science Education, for MI’s mentor, training, compliance, and building capacity, with shared values and practices to maximize the improvement of student outcomes. New Knowledge (e.g., hypothesis, research questions) Research indicates that REU/internship experiences can be particularly powerful for students from Latinx and underrepresented groups in STEM. However, those experiences are difficult to access for many HSI-community college students (85% of our students hold off-campus jobs), and lack of confidence is a barrier for a majority of our students. The gap between those who can and those who cannot is the “internship access gap.” This project is at a central California Community College (CCC) and HSI, the only affordable post-secondary option in a region serving a historically underrepresented population in STEM, including 75% Hispanic, and 87% have not completed college. MI is designed to reduce inequalities inherent in the internship paradigm by providing access to professional and research skills for those underserved students. The MI has been designed to reduce barriers by offering: shorter duration (25 contact hours); flexible timing (one week to once a week over many weeks); open access/large group; and proximal location (on-campus). MI mentors participate in week-long summer workshops and ongoing monthly community of practice with the goal of co-constructing a shared vision, engaging in conversations about pedagogy and learning, and sustaining the MI program going forward. Approach (e.g., objectives/specific aims, research methodologies, and analysis) Research Question and Methodology: We want to know: How does participation in a micro-internship affect students’ interest and confidence to pursue STEM? We used a mixed-methods design triangulating quantitative Likert-style survey data with interpretive coding of open-responses to reveal themes in students’ motivations, attitudes toward STEM, and confidence. Participants: The study sampled students enrolled either part-time or full-time at the community college. Although each MI was classified within STEM, they were open to any interested student in any major. Demographically, participants self-identified as 70% Hispanic/Latinx, 13% Mixed-Race, and 42 female. Instrument: Student surveys were developed from two previously validated instruments that examine the impact of the MI intervention on student interest in STEM careers and pursuing internships/REUs. Also, the pre- and post (every e months to assess longitudinal outcomes) -surveys included relevant open response prompts. The surveys collected students’ demographics; interest, confidence, and motivation in pursuing a career in STEM; perceived obstacles; and past experiences with internships and MIs. 171 students responded to the pre-survey at the time of submission. Outcomes (e.g., preliminary findings, accomplishments to date) Because we just finished year 1, we lack at this time longitudinal data to reveal if student confidence is maintained over time and whether or not students are more likely to (i) enroll in more internships, (ii) transfer to a four-year university, or (iii) shorten the time it takes for degree attainment. For short term outcomes, students significantly Increased their confidence to continue pursuing opportunities to develop within the STEM pipeline, including full-length internships, completing STEM degrees, and applying for jobs in STEM. For example, using a 2-tailed t-test we compared means before and after the MI experience. 15 out of 16 questions that showed improvement in scores were related to student confidence to pursue STEM or perceived enjoyment of a STEM career. Finding from the free-response questions, showed that the majority of students reported enrolling in the MI to gain knowledge and experience. After the MI, 66% of students reported having gained valuable knowledge and experience, and 35% of students spoke about gaining confidence and/or momentum to pursue STEM as a career. Broader Impacts (e.g., the participation of underrepresented minorities in STEM; development of a diverse STEM workforce, enhanced infrastructure for research and education) The ESTEEM project has the potential for a transformational impact on STEM undergraduate education’s access and success for underrepresented and Latinx community college students, as well as for STEM capacity building at Hartnell College, a CCC and HSI, for students, faculty, professionals, and processes that foster research in STEM and education. Through sharing and transfer abilities of the ESTEEM model to similar institutions, the project has the potential to change the way students are served at an early and critical stage of their higher education experience at CCC, where one in every five community college student in the nation attends a CCC, over 67% of CCC students identify themselves with ethnic backgrounds that are not White, and 40 to 50% of University of California and California State University graduates in STEM started at a CCC, thus making it a key leverage point for recruiting and retaining a more diverse STEM workforce. 
    more » « less
  2. null (Ed.)
    A recent S-STEM award has allowed the engineering program in a rural, liberal arts institution to offer a need-based scholarship program for its students. The engineering program has a number of veteran, underrepresented minority, transfer, and nontraditional students. Many students are also first-generation college students. The institution and engineering program matriculate a number of under-served populations, students who may have needs that are not well understood in the typical engineering education literature. The scholarship program and its associated mentoring and activities will assist workforce development and will also incorporate a number of research avenues to better understand and serve the student population in this unique setting. To apply for the program, students must fill out an application with four 250 – 500 word essay responses relating to their academic progress, perceived barriers to degree completion, and how this award would help them to complete their degree. This study continues work using personas, a method used in human-centered design. Using the first round of scholarship application essays as a source, three personas were developed, one successful applicant, one unsuccessful applicant, and one general applicant. Personas are detailed profiles of a fake person who could reasonably be in each category of interest. In human-centered design, personas are detailed descriptions of likely clients or end-users, developed to help the engineers focus on who they might be designing for. The profiles developed in this study were used to gain insight into which students were likely to choose to apply and which students may be missing out on this opportunity. It is time for another round of applications for this grant and the use of personas will continue and expand as part of this study. Before reviewing applications, the committee will create two personas as ideal candidates instead of developing a standardized rubric. Subsequently, three personas will be developed from the Fall 2020 applications, one for all applicants, one for successful applicants, and one for unsuccessful applicants. These personas will then be compared to the personas created by the application review committee and the personas created from the Fall 2018 applicants. Similarities and differences across the persona groups will be explored to determine whether the applicants are what the reviewers expected and whether the pool of applicants has evolved or remained mostly the same throughout the scholarship program. Review committee members will also be interviewed in a focus group setting to discuss their experiences using common personas rather than standardized rubrics in the application review process. At this time, the applications are not yet due and the analysis has not yet begun. Initial interest for the grant has been strong and we anticipate at least thirty applications for the nineteen available grants. Results presented will include the student profiles and faculty experiences with the use of personas as a metric for reviewing student applications. 
    more » « less
  3. Abstract

    Our career-forward approach to general chemistry laboratory for engineers involves the use of design challenges (DCs), an innovation that employs authentic professional context and practice to transform traditional tasks into developmentally appropriate career experiences. These challenges are scaled-down engineering problems related to the US National Academy of Engineering’s Grand Challenges that engage students in collaborative problem solving via the modeling process. With task features aligned with professional engineering practice, DCs are hypothesized to support student motivation for the task as well as for the profession. As an evaluation of our curriculum design process, we use expectancy–value theory to test our hypotheses by investigating the association between students’ task value beliefs and self-confidence with their user experience, gender and URM status. Using stepwise multiple regression analysis, the results reveal that students find value in completing a DC (F(5,2430) = 534.96,p < .001) and are self-confident (F(8,2427) = 154.86,p < .001) when they feel like an engineer, are satisfied, perceive collaboration, are provided help from a teaching assistant, and the tasks are not too difficult. We highlight that although female and URM students felt less self-confidence in completing a DC, these feelings were moderated by their perceptions of feeling like an engineer and collaboration in the learning process (F(10,2425) = 127.06,p < .001). When female students felt like they were engineers (gender x feel like an engineer), their self-confidence increased (β = .288) and when URM students perceived tasks as collaborative (URM status x collaboration), their self-confidence increased (β = .302). Given the lack of representation for certain groups in engineering, this study suggests that providing an opportunity for collaboration and promoting a sense of professional identity afford a more inclusive learning experience.

     
    more » « less
  4. null (Ed.)
    This article details the multi-year process of adding a “design thread” to our department’s electrical and computer engineering curricula. We use the conception of a “thread” to mean a sequence of courses that extend unbroken across each year of the undergraduate curriculum. The design thread includes a project-based introduction to the discipline course in the first year, a course in the second year focusing on measurement and fabrication, a course in the third year to frame technical problems in societal challenges, and culminates with our two-semester, client-driven fourth-year capstone design sequence. The impetus to create a design thread arose from preparation for an ABET visit where we identified a need for more “systems thinking” within the curriculum, particularly system decomposition and modularity; difficulty in having students make engineering evaluations of systems based on data; and students’ difficulty transferring skills in testing, measurement, and evaluation from in-class lab scenarios to more independent work on projects. We also noted that when working in teams, students operated more collectively than collaboratively. In other words, rather than using task division and specialization to carry out larger projects, students addressed all problems collectively as a group. This paper discusses the process through which faculty developed a shared conception of design to enable coherent changes to courses in the four year sequence and the political and practical compromises needed to create the design thread. To develop a shared conception of design faculty explored several frameworks that emphasized multiple aspects of design. Course changes based on elements of these frameworks included introducing design representations such as block diagrams to promote systems thinking in the first year and consistently utilizing representations throughout the remainder of the four year sequence. Emphasizing modularity through representations also enabled introducing aspects of collaborative teamwork. While students are introduced broadly to elements of the design framework in their first year, later years emphasize particular aspects. The second year course focuses on skills in fabrication and performance measurement while the third year course emphasizes problem context and users, in an iterative design process. The client-based senior capstone experience integrates all seven aspects of our framework. On the political and organizational side implementing the design thread required major content changes in the department’s introductory course, and freeing up six credit-hour equivalents, one and a half courses, in the curriculum. The paper discusses how the ABET process enabled these discussions to occur, other curricular changes needed to enable the design thread to be implemented, and methods which enabled the two degree programs to align faculty motivation, distribute the workload, and understand the impact the curricular changes had on student learning. 
    more » « less
  5. Ability to effectively work in teams is one of the desired outcomes of engineering and engineering technology programs. Unfortunately, working in teams is still challenging for many students. Rather than contributing to team projects, some students resort to social loafing. Social loafing tends to destroy both teamwork performance and individual learning, especially in solving ill-structured problems, such as design. Furthermore, a bad experience on a past team is a significant concern as it could generate negative feelings toward future team projects. Formation of collaborative teams is a critical first step in team-project-based design courses as team composition directly affects not only teamwork processes and outcomes, but also teamwork skills and experience. This NSF-IUSE sponsored project aims to enhance students’ teamwork experiences and teamwork learning through 1) understanding how to form better student design teams and 2) identifying exercises that will effectively improve team member collaboration. We do this by comparing student characteristics and design task characteristics with the quality of the design team outcome, and examining the resulting correlations. Student characteristics cover six categories: 1) background information, 2) work structure preferences, 3) personality, 4) ability, 5) motivation, and 6) attitude. Task characteristics and design team outcomes are characterized using the Creative Product Semantic Scale. In this article, we present correlations between student/team characteristics and design project outcome, and correlations between task characteristics and design project outcome for 2020-2021 senior design teams at two institutions. For both institutions, we will present correlations between individual student characteristics and team outcome. For one institution, we will also present correlation between team-level characteristics and team outcomes. 
    more » « less