More Like this

1. Board 418: Understanding Context: Propagation and Effectiveness of the Concept Warehouse in Mechanical Engineering at Five Diverse Institutions and Beyond – Results from Year 4

   Self, B. P. ; Koretsky, M. ; Nolen, S. B. ; Dal Bello, D. J. ; Widmann, J. M. ; Prince, M. J. ; Papadopoulos, C. ( June 2023 , 2023 ASEE Annual Conference & Exposition)

   Several consensus reports cite a critical need to dramatically increase the number and diversity of STEM graduates over the next decade. They conclude that a change to evidence-based instructional practices, such as concept-based active learning, is needed. Concept-based active learning involves the use of activity-based pedagogies whose primary objectives are to make students value deep conceptual understanding (instead of only factual knowledge) and then to facilitate their development of that understanding. Concept-based active learning has been shown to increase academic engagement and student achievement, to significantly improve student retention in academic programs, and to reduce the performance gap of underrepresented students. Fostering students' mastery of fundamental concepts is central to real world problem solving, including several elements of engineering practice. Unfortunately, simply proving that these instructional practices are more effective than traditional methods for promoting student learning, for increasing retention in academic programs, and for improving ability in professional practice is not enough to ensure widespread pedagogical change. In fact, the biggest challenge to improving STEM education is not the need to develop more effective instructional practices, but to find ways to get faculty to adopt the evidence-based pedagogies that already exist. In this project we seek to propagate the Concept Warehouse, a technological innovation designed to foster concept-based active learning, into Mechanical Engineering (ME) and to study student learning with this tool in five diverse institutional settings. The Concept Warehouse (CW) is a web-based instructional tool that we developed for Chemical Engineering (ChE) faculty. It houses over 3,500 ConcepTests, which are short questions that can rapidly be deployed to engage students in concept-oriented thinking and/or to assess students’ conceptual knowledge, along with more extensive concept-based active learning tools. The CW has grown rapidly during this project and now has over 1,600 faculty accounts and over 37,000 student users. New ConcepTests were created during the current reporting period; the current numbers of questions for Statics, Dynamics, and Mechanics of Materials are 342, 410, and 41, respectively. A detailed review process is in progress, and will continue through the no-cost extension year, to refine question clarity and to identify types of new questions to fill gaps in content coverage. There have been 497 new faculty accounts created after June 30, 2018, and 3,035 unique students have answered these mechanics questions in the CW. We continue to analyze instructor interviews, focusing on 11 cases, all of whom participated in the CW Community of Practice (CoP). For six participants, we were able to compare use of the CW both before and after participating in professional development activities (workshops and/or a community or practice). Interview results have been coded and are currently being analyzed. To examine student learning, we recruited faculty to participate in deploying four common questions in both statics and dynamics. In statics, each instructor agreed to deploy the same four questions (one each for Rigid Body Equilibrium, Trusses, Frames, and Friction) among their overall deployments of the CW. In addition to answering the question, students were also asked to provide a written explanation to explain their reasoning, to rate the confidence of their answers, and to rate the degree to which the questions were clear and promoted deep thinking. The analysis to date has resulted in a Work-In-Progress paper presented at ASEE 2022, reporting a cross-case comparison of two instructors and a Work-In-Progress paper to be presented at ASEE 2023 analyzing students’ metacognitive reflections of concept questions. 

   more » « less
2. Author-Driven Computable Data and Ontology Production for Taxonomists

   https://doi.org/10.3897/biss.5.75741  

   Cui, Hong ; Ford, Bruce ; Starr, Julian ; Macklin, James ; Reznicek, Anton ; Giebink, Noah ; Longert, Dylan ; Léveillé-Bourret, Étienne ; Zhang, Limin ( September 2021 , Biodiversity Information Science and Standards)

   It takes great effort to manually or semi-automatically convert free-text phenotype narratives (e.g., morphological descriptions in taxonomic works) to a computable format before they can be used in large-scale analyses. We argue that neither a manual curation approach nor an information extraction approach based on machine learning is a sustainable solution to produce computable phenotypic data that are FAIR (Findable, Accessible, Interoperable, Reusable) (Wilkinson et al. 2016). This is because these approaches do not scale to all biodiversity, and they do not stop the publication of free-text phenotypes that would need post-publication curation. In addition, both manual and machine learning approaches face great challenges: the problem of inter-curator variation (curators interpret/convert a phenotype differently from each other) in manual curation, and keywords to ontology concept translation in automated information extraction, make it difficult for either approach to produce data that are truly FAIR. Our empirical studies show that inter-curator variation in translating phenotype characters to Entity-Quality statements (Mabee et al. 2007) is as high as 40% even within a single project. With this level of variation, curated data integrated from multiple curation projects may still not be FAIR. The key causes of this variation have been identified as semantic vagueness in original phenotype descriptions and difficulties in using standardized vocabularies (ontologies). We argue that the authors describing characters are the key to the solution. Given the right tools and appropriate attribution, the authors should be in charge of developing a project's semantics and ontology. This will speed up ontology development and improve the semantic clarity of the descriptions from the moment of publication. In this presentation, we will introduce the Platform for Author-Driven Computable Data and Ontology Production for Taxonomists, which consists of three components: a web-based, ontology-aware software application called 'Character Recorder,' which features a spreadsheet as the data entry platform and provides authors with the flexibility of using their preferred terminology in recording characters for a set of specimens (this application also facilitates semantic clarity and consistency across species descriptions); a set of services that produce RDF graph data, collects terms added by authors, detects potential conflicts between terms, dispatches conflicts to the third component and updates the ontology with resolutions; and an Android mobile application, 'Conflict Resolver,' which displays ontological conflicts and accepts solutions proposed by multiple experts. a web-based, ontology-aware software application called 'Character Recorder,' which features a spreadsheet as the data entry platform and provides authors with the flexibility of using their preferred terminology in recording characters for a set of specimens (this application also facilitates semantic clarity and consistency across species descriptions); a set of services that produce RDF graph data, collects terms added by authors, detects potential conflicts between terms, dispatches conflicts to the third component and updates the ontology with resolutions; and an Android mobile application, 'Conflict Resolver,' which displays ontological conflicts and accepts solutions proposed by multiple experts. Fig. 1 shows the system diagram of the platform. The presentation will consist of: a report on the findings from a recent survey of 90+ participants on the need for a tool like Character Recorder; a methods section that describes how we provide semantics to an existing vocabulary of quantitative characters through a set of properties that explain where and how a measurement (e.g., length of perigynium beak) is taken. We also report on how a custom color palette of RGB values obtained from real specimens or high-quality specimen images, can be used to help authors choose standardized color descriptions for plant specimens; and a software demonstration, where we show how Character Recorder and Conflict Resolver can work together to construct both human-readable descriptions and RDF graphs using morphological data derived from species in the plant genus Carex (sedges). The key difference of this system from other ontology-aware systems is that authors can directly add needed terms to the ontology as they wish and can update their data according to ontology updates. a report on the findings from a recent survey of 90+ participants on the need for a tool like Character Recorder; a methods section that describes how we provide semantics to an existing vocabulary of quantitative characters through a set of properties that explain where and how a measurement (e.g., length of perigynium beak) is taken. We also report on how a custom color palette of RGB values obtained from real specimens or high-quality specimen images, can be used to help authors choose standardized color descriptions for plant specimens; and a software demonstration, where we show how Character Recorder and Conflict Resolver can work together to construct both human-readable descriptions and RDF graphs using morphological data derived from species in the plant genus Carex (sedges). The key difference of this system from other ontology-aware systems is that authors can directly add needed terms to the ontology as they wish and can update their data according to ontology updates. The software modules currently incorporated in Character Recorder and Conflict Resolver have undergone formal usability studies. We are actively recruiting Carex experts to participate in a 3-day usability study of the entire system of the Platform for Author-Driven Computable Data and Ontology Production for Taxonomists. Participants will use the platform to record 100 characters about one Carex species. In addition to usability data, we will collect the terms that participants submit to the underlying ontology and the data related to conflict resolution. Such data allow us to examine the types and the quantities of logical conflicts that may result from the terms added by the users and to use Discrete Event Simulation models to understand if and how term additions and conflict resolutions converge. We look forward to a discussion on how the tools (Character Recorder is online at http://shark.sbs.arizona.edu/chrecorder/public) described in our presentation can contribute to producing and publishing FAIR data in taxonomic studies. 

   more » « less
3. Speech analysis of teaching assistant interventions in small group collaborative problem solving with undergraduate engineering students

   https://doi.org/10.1111/bjet.13449  

   D'Angelo, Cynthia M. ; Rajarathinam, Robin Jephthah ( March 2024 , British Journal of Educational Technology)

   This descriptive study focuses on using voice activity detection (VAD) algorithms to extract student speech data in order to better understand the collaboration of small group work and the impact of teaching assistant (TA) interventions in undergraduate engineering discussion sections. Audio data were recorded from individual students wearing head‐mounted noise‐cancelling microphones. Video data of each student group were manually coded for collaborative behaviours (eg, group task relatedness, group verbal interaction and group talk content) of students and TA–student interactions. The analysis includes information about the turn taking, overall speech duration patterns and amounts of overlapping speech observed both when TAs were intervening with groups and when they were not. We found that TAs very rarely provided explicit support regarding collaboration. Key speech metrics, such as amount of turn overlap and maximum turn duration, revealed important information about the nature of student small group discussions and TA interventions. TA interactions during small group collaboration are complex and require nuanced treatments when considering the design of supportive tools.

   What is already known about this topic

   Student turn taking can provide information about the nature of student discussions and collaboration.

   Real classroom audio data of small groups typically have lots of background noise and present challenges for audio analysis.

   TAs have little training in how to productively intervene with students about collaborative skills.

   What this paper adds

   TA interaction with groups primarily focused on task progress and understanding of concepts with negligible explicit support on building collaborative skills.

   TAs intervened with the groups often which gave the students little time for uptake of their suggestions or deeper discussion.

   Student turn overlap was higher without the presence of TAs.

   Maximum turn duration can be an important real‐time turn metric to identify the least verbally active student participant in a group.

   Implications for practice and/or policy

   TA training should include information about how to monitor groups for collaborative behaviours and when and how they should intervene to provide feedback and support.

   TA feedback systems should keep track of previous interventions by TAs (especially in contexts where there are multiple TAs facilitating) and the duration since previous intervention to ensure that TAs do not intervene with a group too frequently with little time for student uptake.

   Maximum turn duration could be used as a real‐time metric to identify the least verbally active student in a group so that support could be provided to them by the TAs.

    

   more » « less
4. Animation Analytics in an Interactive Textbook for Material and Energy Balances

   Stone, S. J. ; Crockett, B. ; Xu, Kevin S. ; Liberatore, M. W. ( August 2022 , ASEE Annual Conference proceedings)

   Interactive textbooks generate big data through student reading participation, including animations, question sets, and auto-graded homework. Animations are multi-step, dynamic visuals with text captions. By dividing new content into smaller chunks of information, student engagement is expected to be high, which aligns with tenets of cognitive load theory. Specifically, students’ clicks are recorded and measure usage, completion, and view time per step and for entire animations. Animation usage data from an interactive textbook for a chemical engineering course in Material and Energy Balances accounts for 60,000 animation views across 140+ unique animations. Data collected across five cohorts between 2016 and 2020 used various metrics to capture animation usage including watch and re-watch rates as well as the length of animation views. Variations in view rate and time were examined across content, parsed by book chapter, and five animation characterizations (Concept, Derivation, Figures and Plots, Physical World, and Spreadsheets). Important findings include: 1) Animation views were at or above 100% for all chapters and cohorts, 2) Median view time varies from 22 s (2-step) to 59 s (6-step) - a reasonable attention span for students and cognitive load, 3) Median view time for animations characterized as Derivation was the longest (40 s) compared to Physical World animations, which resulted in the shortest time (20 s). 

   more » « less
5. WIP Toward a Free-Body Diagram Mobile Application

   Sloboda, A. R. ; Wodin-Schwartz, S. ; Lechasseur, K. ; deWinter, J. ( June 2023 , ASEE annual conference exposition proceedings)

   No skill is more important for a student of mechanics than the ability to draw a complete and accurate free-body diagram (FBD). A good FBD facilitates proper accounting of forces when writing the balances that lead to governing equations in statics, solid mechanics, and dynamics. Because this skill is essential, educational approaches that improve the ability of students to draw correct FBDs are critical for maximizing the potential of the next generation of engineers. Traditionally, learning to draw FBDs involves classroom instruction followed by homework practice consisting of problems drawn from a textbook. Homework as practice does not serve all students well, because it does not scaffold the process of drawing FBDs in terms of distinct tasks (e.g., isolating the body, considering support reactions) nor does it offer immediate feedback, which students often need to avoid falling into the same error repeatedly. To address these shortcomings, we embarked on the design, implementation, and testing of a mobile application (app) that offers an alternative venue for FBD practice. The app provides students with asynchronous opportunities for training, varied tasks that target specific FBD issues, and several levels of immediate feedback. We hypothesize that the gamified environment and puzzle-based gameplay will improve student skill and self-efficacy in drawing FBDs, particularly for women, who may feel less confident in their spatial skills. Data collected to describe student experiences may also provide additional insight into how to improve FBD instruction generally. In this paper, we detail the process for designing and implementing the app and provide initial data regarding student impressions and use. The app was piloted in Fall 2022 in a large Introduction to Statics course as a non-graded study activity; all students except one (n=97) participated in an evaluation of its design features and user experiences. Approximately half (54%) of students indicated they had played half or more of the available games. When commenting about how the FBD app did, or did not, help their learning, 49% of respondents appreciated that the app allowed additional opportunities for practice. Students used these opportunities to further develop several skills, such as visualizing the system and setting up accurate diagrams, which strengthened their confidence and reviewed key concepts. While describing the value of practicing through the app, 21% of students called out how the app provided feedback. They specifically mentioned the positive experiences of receiving feedback that is immediate, that explains boundary connections, and that deepens learning after mistakes are made. These and other findings from the pilot study are discussed with corresponding next steps for development. 

   more » « less