skip to main content

Title: Work-in-Progress: A Review of the Type, Breadth, and Limitations of Publicly Available Educational Technology Products in 2022
One of the major changes in the higher education ecosystem over the last decade has been a rise in the availability of education-based software products, including education-based web-pages and web-services. Globally the investment in education-based startups in 2017 was $9.5B which surged to $18.7B in 2019 [1]. The COVID-19 pandemic further fueled record investment in this sector, with the US seeing $2.2B invested in 130 startups in 2020, up from $1.7B in 2019 and $1.4B in 2018 (see [2] and [3]). Early indicators show that 2021 will again see further increases [4]. While the majority (92%) of these investments are aimed at consumer and corporate sectors, there is potential for the innovations developed to diffuse into both the P-12 and higher education spaces [5]. What is evident from the investment numbers is that an integration of learning technologies specifically into higher education is progressing at a relatively slower pace [5]. It is the goal of this work-in-progress to identify some of the reasons for this slower progress. Our hypothesis is that, while some of these reasons may be obvious, there are also more subtle and/or counterintuitive reasons for the reduced interest in higher education. The motivation and need for the proposed study grew out of an ongoing NSF RED project where we endeavor to more » fuse the concept of convergence, loosely defined as “deep integration,” into our undergraduate engineering curriculum. Increasingly software and data systems at colleges and universities, and the affordances they do and do not offer, are integral to university structures. If the respective software systems do not support certain activities and functions then the programs are simply not useful to the faculty [6]. Additionally, any subset of systems needs to seamlessly integrate to form a coherent and usable learning support system that faculty, students, and staff can use without issue and/or barrier. The goal of the proposed activity within our grant is, thus, to build structures to collect, analyze, and display data in support of developing skills in addressing convergent problems. « less
Authors:
; ; ; ; ; ; ;
Award ID(s):
2022271
Publication Date:
NSF-PAR ID:
10356583
Journal Name:
American Society for Engineering Education Annual Conference and Exhibition
Page Range or eLocation-ID:
#37421
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    In 2016, 10 universities launched a Networked Improvement Community (NIC) aimed at increasing the number of scholars from Alliances for Graduate Education and the Professoriate (AGEP) populations entering science, technology, engineering, and mathematics (STEM) faculty careers. NICs bring together stakeholders focused on a common goal to accelerate innovation through structured, ongoing intervention development, implementation, and refinement. We theorized a NIC organizational structure would aid understandings of a complex problem in different contexts and accelerate opportunities to develop and improve interventions to address the problem. A distinctive feature of this NIC is its diverse institutional composition of public and private, predominantly white institutions, a historically Black university, a Hispanic-serving institution, and land grant institutions located across eight states and Washington, DC, United States. NIC members hold different positions within their institutions and have access to varied levers of change. Among the many lessons learned through this community case study, analyzing and addressing failed strategies is as equally important to a healthy NIC as is sharing learning from successful interventions. We initially relied on pre-existing relationships and assumptions about how we would work together, rather than making explicit how the NIC would develop, establish norms, understand common processes, and manage changing relationships.more »We had varied understandings of the depth of campus differences, sometimes resulting in frustrations about the disparate progress on goals. NIC structures require significant engagement with the group, often more intensive than traditional multi-institution organizational structures. They require time to develop and ongoing maintenance in order to advance the work. We continue to reevaluate our model for leadership, climate, diversity, conflict resolution, engagement, decision-making, roles, and data, leading to increased investment in the success of all NIC institutions. Our NIC has evolved from the traditional NIC model to become the Center for the Integration of Research, Teaching and Learning (CIRTL) AGEP NIC model with five key characteristics: (1) A well-specified aim, (2) An understanding of systems, including a variety of contexts and different organizations, (3) A culture and practice of shared leadership and inclusivity, (4) The use of data reflecting different institutional contexts, and (5) The ability to accelerate infrastructure and interventions. We conclude with recommendations for those considering developing a NIC to promote diversity, equity, and inclusion efforts.« less
  2. A 2019 report from the National Academies on Minority Serving Institutions (MSIs) concluded that MSIs need to change their culture to successfully serve students with marginalized racial and/or ethnic identities. The report recommends institutional responsiveness to meet students “where they are,” metaphorically, creating supportive campus environments and providing tailored academic and social support structures. In recent years, the faculty, staff, and administrators at California State University, Los Angeles have made significant efforts to enhance student success through multiple initiatives including a summer bridge program, first-year in engineering program, etc. However, it has become clear that more profound changes are needed to create a culture that meets students “where they are.” In 2020, we were awarded NSF support for Eco-STEM, an initiative designed to change a system that demands "college-ready" students into one that is "student-ready." Aimed at shifting the deficit mindset prevailing in engineering education, the Eco-STEM project embraces an asset-based ecosystem model that thinks of education as cultivation, and ideas as seeds we are planting, rather than a system of standards and quality checks. This significant paradigm and culture transformation is accomplished through: 1) The Eco-STEM Faculty Fellows’ Community of Practice (CoP), which employs critically reflective dialogue[ ][ ]more »to enhance the learning environment using asset-based learner-centered instructional approaches; 2) A Leadership CoP with department chairs and program directors that guides cultural change at the department/program level; 3) A Facilitators’ CoP that prepares facilitators to lead, sustain, update, and expand the Faculty and Leadership CoPs; 4) Reform of the teaching evaluation system to sustain the cultural changes. This paper presents the progress and preliminary findings of the Eco-STEM project. During the first project year, the project team formulated the curriculum for the Faculty CoP with a focus on inclusive pedagogy, community cultural wealth, and community building, developed a classroom peer observation tool to provide formative data for teaching reflection, and designed research inquiry tools. The latter investigates the following research questions: 1) To what extent do the Eco-STEM CoPs effectively shift the mental models of participants from a factory-like model to an ecosystem model of education? 2) To what extent does this shift support an emphasis on the assets of our students, faculty, and staff members and, in turn, allow for enhanced motivation, excellence and success? 3) To what extent do new faculty assessment tools designed to provide feedback that reflects ecosystem-centric principles and values allow for individuals within the system to thrive? In Fall 2021, the first cohort of Eco-STEM Faculty Fellows were recruited, and rich conversations and in-depth reflections in our CoP meetings indicated Fellows’ positive responses to both the CoP curriculum and facilitation practices. This paper offers a work-in-progress introduction to the Eco-STEM project, including the Faculty CoP, the classroom peer observation tool, and the proposed research instruments. We hope this work will cultivate broader conversations within the engineering education research community about cultural change in engineering education and methods towards its implementation.« less
  3. Obeid, Iyad ; Selesnick, Ivan ; Picone, Joseph (Ed.)
    The Temple University Hospital Seizure Detection Corpus (TUSZ) [1] has been in distribution since April 2017. It is a subset of the TUH EEG Corpus (TUEG) [2] and the most frequently requested corpus from our 3,000+ subscribers. It was recently featured as the challenge task in the Neureka 2020 Epilepsy Challenge [3]. A summary of the development of the corpus is shown below in Table 1. The TUSZ Corpus is a fully annotated corpus, which means every seizure event that occurs within its files has been annotated. The data is selected from TUEG using a screening process that identifies files most likely to contain seizures [1]. Approximately 7% of the TUEG data contains a seizure event, so it is important we triage TUEG for high yield data. One hour of EEG data requires approximately one hour of human labor to complete annotation using the pipeline described below, so it is important from a financial standpoint that we accurately triage data. A summary of the labels being used to annotate the data is shown in Table 2. Certain standards are put into place to optimize the annotation process while not sacrificing consistency. Due to the nature of EEG recordings, some recordsmore »start off with a segment of calibration. This portion of the EEG is instantly recognizable and transitions from what resembles lead artifact to a flat line on all the channels. For the sake of seizure annotation, the calibration is ignored, and no time is wasted on it. During the identification of seizure events, a hard “3 second rule” is used to determine whether two events should be combined into a single larger event. This greatly reduces the time that it takes to annotate a file with multiple events occurring in succession. In addition to the required minimum 3 second gap between seizures, part of our standard dictates that no seizure less than 3 seconds be annotated. Although there is no universally accepted definition for how long a seizure must be, we find that it is difficult to discern with confidence between burst suppression or other morphologically similar impressions when the event is only a couple seconds long. This is due to several reasons, the most notable being the lack of evolution which is oftentimes crucial for the determination of a seizure. After the EEG files have been triaged, a team of annotators at NEDC is provided with the files to begin data annotation. An example of an annotation is shown in Figure 1. A summary of the workflow for our annotation process is shown in Figure 2. Several passes are performed over the data to ensure the annotations are accurate. Each file undergoes three passes to ensure that no seizures were missed or misidentified. The first pass of TUSZ involves identifying which files contain seizures and annotating them using our annotation tool. The time it takes to fully annotate a file can vary drastically depending on the specific characteristics of each file; however, on average a file containing multiple seizures takes 7 minutes to fully annotate. This includes the time that it takes to read the patient report as well as traverse through the entire file. Once an event has been identified, the start and stop time for the seizure is stored in our annotation tool. This is done on a channel by channel basis resulting in an accurate representation of the seizure spreading across different parts of the brain. Files that do not contain any seizures take approximately 3 minutes to complete. Even though there is no annotation being made, the file is still carefully examined to make sure that nothing was overlooked. In addition to solely scrolling through a file from start to finish, a file is often examined through different lenses. Depending on the situation, low pass filters are used, as well as increasing the amplitude of certain channels. These techniques are never used in isolation and are meant to further increase our confidence that nothing was missed. Once each file in a given set has been looked at once, the annotators start the review process. The reviewer checks a file and comments any changes that they recommend. This takes about 3 minutes per seizure containing file, which is significantly less time than the first pass. After each file has been commented on, the third pass commences. This step takes about 5 minutes per seizure file and requires the reviewer to accept or reject the changes that the second reviewer suggested. Since tangible changes are made to the annotation using the annotation tool, this step takes a bit longer than the previous one. Assuming 18% of the files contain seizures, a set of 1,000 files takes roughly 127 work hours to annotate. Before an annotator contributes to the data interpretation pipeline, they are trained for several weeks on previous datasets. A new annotator is able to be trained using data that resembles what they would see under normal circumstances. An additional benefit of using released data to train is that it serves as a means of constantly checking our work. If a trainee stumbles across an event that was not previously annotated, it is promptly added, and the data release is updated. It takes about three months to train an annotator to a point where their annotations can be trusted. Even though we carefully screen potential annotators during the hiring process, only about 25% of the annotators we hire survive more than one year doing this work. To ensure that the annotators are consistent in their annotations, the team conducts an interrater agreement evaluation periodically to ensure that there is a consensus within the team. The annotation standards are discussed in Ochal et al. [4]. An extended discussion of interrater agreement can be found in Shah et al. [5]. The most recent release of TUSZ, v1.5.2, represents our efforts to review the quality of the annotations for two upcoming challenges we hosted: an internal deep learning challenge at IBM [6] and the Neureka 2020 Epilepsy Challenge [3]. One of the biggest changes that was made to the annotations was the imposition of a stricter standard for determining the start and stop time of a seizure. Although evolution is still included in the annotations, the start times were altered to start when the spike-wave pattern becomes distinct as opposed to merely when the signal starts to shift from background. This cuts down on background that was mislabeled as a seizure. For seizure end times, all post ictal slowing that was included was removed. The recent release of v1.5.2 did not include any additional data files. Two EEG files had been added because, originally, they were corrupted in v1.5.1 but were able to be retrieved and added for the latest release. The progression from v1.5.0 to v1.5.1 and later to v1.5.2, included the re-annotation of all of the EEG files in order to develop a confident dataset regarding seizure identification. Starting with v1.4.0, we have also developed a blind evaluation set that is withheld for use in competitions. The annotation team is currently working on the next release for TUSZ, v1.6.0, which is expected to occur in August 2020. It will include new data from 2016 to mid-2019. This release will contain 2,296 files from 2016 as well as several thousand files representing the remaining data through mid-2019. In addition to files that were obtained with our standard triaging process, a part of this release consists of EEG files that do not have associated patient reports. Since actual seizure events are in short supply, we are mining a large chunk of data for which we have EEG recordings but no reports. Some of this data contains interesting seizure events collected during long-term EEG sessions or data collected from patients with a history of frequent seizures. It is being mined to increase the number of files in the corpus that have at least one seizure event. We expect v1.6.0 to be released before IEEE SPMB 2020. The TUAR Corpus is an open-source database that is currently available for use by any registered member of our consortium. To register and receive access, please follow the instructions provided at this web page: https://www.isip.piconepress.com/projects/tuh_eeg/html/downloads.shtml. The data is located here: https://www.isip.piconepress.com/projects/tuh_eeg/downloads/tuh_eeg_artifact/v2.0.0/.« less
  4. In September 2019, the fourth and final workshop on the Future of Mechatronics and Robotics Education (FoMRE) was held at a Lawrence Technological University in Southfield, MI. This workshop was organized by faculty at several universities with financial support from industry partners and the National Science Foundation. The purpose of the workshops was to create a cohesive effort among mechatronics and robotics courses, minors and degree programs. Mechatronics and Robotics Engineering (MRE) is an integration of mechanics, controls, electronics, and software, which provides a unique opportunity for engineering students to function on multidisciplinary teams. Due to its multidisciplinary nature, it attracts diverse and innovative students, and graduates better-prepared professional engineers. In this fast growing field, there is a great need to standardize educational material and make MRE education more widely available and easier to adopt. This can only be accomplished if the community comes together to speak with one clear voice about not only the benefits, but also the best ways to teach it. These efforts would also aid in establishing more of these degree programs and integrating minors or majors into existing computer science, mechanical engineering, or electrical engineering departments. The final workshop was attended by approximately 50 practitionersmore »from industry and academia. Participants identified many practical skills required for students to succeed in an MRE curriculum and as practicing engineers after graduation. These skills were then organized into the following categories: professional, independent learning, controller design, numerical simulation and analysis, electronics, software development, and system design. For example, professional skills include technical reports, presentations, and documentation. Independent learning includes reading data sheets, performing internet searches, doing a literature review, and having a maker mindset. Numerical simulation skills include understanding data, presenting data graphically, solving and simulating in software such as MATLAB, Simulink and Excel. Controller design involves selecting a controller, tuning a controller, designing to meet specifications, and understanding when the results are good enough. Electronics skills include selecting sensors, interfacing sensors, interfacing actuators, creating printed circuit boards, wiring on a breadboard, soldering, installing drivers, using integrated circuits, and using microcontrollers. Software development of embedded systems includes agile program design, state machines, analyzing and evaluating code results, commenting code, troubleshooting, debugging, AI and machine learning. Finally, system design includes prototyping, creating CAD models, design for manufacturing, breaking a system down into subsystems, integrating and interfacing subcomponents, having a multidisciplinary perspective, robustness, evaluating tradeoffs, testing, validation, and verification, failure, effect, and mode analysis. A survey was prepared and sent out to the participants from all four workshops as well as other robotics faculty, researchers and industry personnel in order to elicit a broader community response. Because one of the biggest challenges in mechatronics and robotics education is the absence of standardized curricula, textbooks, platforms, syllabi, assignments, and learning outcomes, this was a vital part of the process to achieve some level of consensus. This paper presents an introduction to MRE education, related work on existing programs, methods, results of the practical skills survey, and then draws conclusions based upon these results. It aims to create the foundation for standardizing the development of student skills in mechatronics and robotics curricula across institutions, disciplines, majors and minors. The survey was completed by 94 participants and it was clear that there is a consensus that the primary skills students should have upon completion of MRE courses or a program is a broader multidisciplinary systems-level perspective, an ability to problem solve, and an ability to design a system to meet specifications.« less
  5. The development of tools that promote active learning in engineering disciplines is critical. It is widely understood that students engaged in active learning environments outperform those taught using passive methods. Previously, we reported on the development and implementation of hands-on Low-Cost Desktop Learning Modules (LCDLMs) that replicate real-world industrial equipment which serves to create active learning environments. Thus far, miniaturized venturi meter, hydraulic loss, and double-pipe and shell & tube heat exchanger DLMs have been utilized by hundreds of students across the country. It was demonstrated that the use of DLMs in face-to-face classrooms results in statistically significant improvements in student performance as well as increases in student motivation compared to students taught in a traditional lecture-only style classroom. Last year, participants in the project conducted 45 implementations including over 600 DLMs at 24 universities across the country reaching more than 1,000 students. In this project, we report on the significant progress made in broad dissemination of DLMs and accompanying pedagogy. We demonstrate that DLMs serve to increase student learning gains not only in face-to-face environments but also in virtual learning environments. Instructional videos were developed to aid in DLM-based learning during the COVID-19 pandemic when instructors were limited tomore »virtual instruction. Preliminary results from this work show that students working with DLMs even in a virtual setting significantly outperform those taught without DLM-associated materials. Significant progress has also been made on the development of a new DLM cartridge: a see-through 3D-printed miniature fluidized bed. The new 3D printing methodology will allow for rapid prototyping and streamlined development of DLMs. A 3D-printed evaporative cooling tower DLM will also be developed in the coming year. In October 2020, the team held a virtual implementers workshop to train new participating faculty in DLM use and implementation. In total, 13 new faculty participants from 10 universities attended the 6-hour, 2-day workshop and plan to implement DLMs in their classrooms during this academic year. In the last year, this project was disseminated in 8 presentations at the American Society for Engineering Education (ASEE) Virtual Conference (June 2020) and American Institute of Chemical Engineers Annual Conference (November 2019) as well as the AIChE virtual Community of Practice Labs Group and a seminar at a major university, ultimately disseminating DLM pedagogy to approximately 200 individuals including approximately 120 university faculty. Further, the former group postdoc has accepted an instructor faculty position at University of Wisconsin Madison where she will teach unit operations among other subjects; she and the remainder of the team believe the LCDLM project has prepared her well for that position. In the remaining 2.5 years of the project, we will continue to evaluate the effectiveness of DLMs in teaching key heat transfer and fluid dynamics concepts thru implementations in the rapidly expanding pool of participating universities. Further, we continue our ongoing efforts in creating the robust support structure necessary for large-scale adoption of hands-on educational tools for promotion of hands-on interactive student learning.« less