skip to main content


This content will become publicly available on October 23, 2024

Title: Opportunities for enhancing MLCommons efforts while leveraging insights from educational MLCommons earthquake benchmarks efforts

MLCommons is an effort to develop and improve the artificial intelligence (AI) ecosystem through benchmarks, public data sets, and research. It consists of members from start-ups, leading companies, academics, and non-profits from around the world. The goal is to make machine learning better for everyone. In order to increase participation by others, educational institutions provide valuable opportunities for engagement. In this article, we identify numerous insights obtained from different viewpoints as part of efforts to utilize high-performance computing (HPC) big data systems in existing education while developing and conducting science benchmarks for earthquake prediction. As this activity was conducted across multiple educational efforts, we project if and how it is possible to make such efforts available on a wider scale. This includes the integration of sophisticated benchmarks into courses and research activities at universities, exposing the students and researchers to topics that are otherwise typically not sufficiently covered in current course curricula as we witnessed from our practical experience across multiple organizations. As such, we have outlined the many lessons we learned throughout these efforts, culminating in the need forbenchmark carpentryfor scientists using advanced computational resources. The article also presents the analysis of an earthquake prediction code benchmark while focusing on the accuracy of the results and not only on the runtime; notedly, this benchmark was created as a result of our lessons learned. Energy traces were produced throughout these benchmarks, which are vital to analyzing the power expenditure within HPC environments. Additionally, one of the insights is that in the short time of the project with limited student availability, the activity was only possible by utilizing a benchmark runtime pipeline while developing and using software to generate jobs from the permutation of hyperparameters automatically. It integrates a templated job management framework for executing tasks and experiments based on hyperparameters while leveraging hybrid compute resources available at different institutions. The software is part of a collection calledcloudmeshwith its newly developed components, cloudmesh-ee (experiment executor) and cloudmesh-cc (compute coordinator).

 
more » « less
Award ID(s):
2210266
NSF-PAR ID:
10473591
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
Frontiers
Date Published:
Journal Name:
Frontiers in High Performance Computing
Volume:
1
ISSN:
2813-7337
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. There have been numerous demands for enhancements in the way undergraduate learning occurs today, especially at a time when the value of higher education continues to be called into question (The Boyer 2030 Commission, 2022). One type of demand has been for the increased integration of subjects/disciplines around relevant issues/topics—with a more recent trend of seeking transdisciplinary learning experiences for students (Sheets, 2016; American Association for the Advancement of Science, 2019). Transdisciplinary learning can be viewed as the holistic way of working equally across disciplines to transcend their own disciplinary boundaries to form new conceptual understandings as well as develop new ways in which to address complex topics or challenges (Ertas, Maxwell, Rainey, & Tanik, 2003; Park & Son, 2010). This transdisciplinary approach can be important as humanity’s problems are not typically discipline specific and require the convergence of competencies to lead to innovative thinking across fields of study. However, higher education continues to be siloed which makes the authentic teaching of converging topics, such as innovation, human-technology interactions, climate concerns, or harnessing the data revolution, organizationally difficult (Birx, 2019; Serdyukov, 2017). For example, working across a university’s academic units to collaboratively teach, or co-teach, around topics of convergence are likely to be rejected by the university systems that have been built upon longstanding traditions. While disciplinary expertise is necessary and one of higher education’s strengths, the structures and academic rigidity that come along with the disciplinary silos can prevent modifications/improvements to the roles of academic units/disciplines that could better prepare students for the future of both work and learning. The balancing of disciplinary structure with transdisciplinary approaches to solving problems and learning is a challenge that must be persistently addressed. These institutional challenges will only continue to limit universities seeking toward scaling transdisciplinary programs and experimenting with novel ways to enhance the value of higher education for students and society. This then restricts innovations to teaching and also hinders the sharing of important practices across disciplines. To address these concerns, a National Science Foundation Improving Undergraduate STEM Education project team, which is the topic of this paper, has set the goal of developing/implementing/testing an authentically transdisciplinary, and scalable educational model in an effort to help guide the transformation of traditional undergraduate learning to span academics silos. This educational model, referred to as the Mission, Meaning, Making (M3) program, is specifically focused on teaching the crosscutting practices of innovation by a) implementing co-teaching and co-learning from faculty and students across different academic units/colleges as well as b) offering learning experiences spanning multiple semesters that immerse students in a community that can nourish both their learning and innovative ideas. As a collaborative initiative, the M3 program is designed to synergize key strengths of an institution’s engineering/technology, liberal arts, and business colleges/units to create a transformative undergraduate experience focused on the pursuit of innovation—one that reaches the broader campus community, regardless of students’ backgrounds or majors. Throughout the development of this model, research was conducted to help identify institutional barriers toward creating such a cross-college program at a research-intensive public university along with uncovering ways in which to address these barriers. While data can show how students value and enjoy transdisciplinary experiences, universities are not likely to be structured in a way to support these educational initiatives and they will face challenges throughout their lifespan. These challenges can result from administration turnover whereas mutual agreements across colleges may then vanish, continued disputes over academic territory, and challenges over resource allotments. Essentially, there may be little to no incentives for academic departments to engage in transdisciplinary programming within the existing structures of higher education. However, some insights and practices have emerged from this research project that can be useful in moving toward transdisciplinary learning around topics of convergence. Accordingly, the paper will highlight features of an educational model that spans disciplines along with the workarounds to current institutional barriers. This paper will also provide lessons learned related to 1) the potential pitfalls with educational programming becoming “un-disciplinary” rather than transdisciplinary, 2) ways in which to incentivize departments/faculty to engage in transdisciplinary efforts, and 3) new structures within higher education that can be used to help faculty/students/staff to more easily converge to increase access to learning across academic boundaries. 
    more » « less
  2. Intelligent Autonomous Systems, including Intelligent Manufacturing & Automation and Industry 4.0, have immense potential to improve human health, safety, and welfare. Engineering these systems requires an interdisciplinary knowledge of mechanical, electrical, computer, software, and systems engineering throughout the design and development process. Mechatronics and Robotics Engineering (MRE) is emerging as a discipline that can provide the broad inter-disciplinary technical and professional skill sets that are critical to fulfill the research and development needs for these advanced systems. Despite experiencing tremendous, dynamic growth, MRE lacks a settled-on and agreed-upon body-of-knowledge, leading to unmet needs for standardized curricula, courses, laboratory platforms, and accreditation criteria, resulting in missed career opportunities for individuals and missed economic opportunities for industry. There have been many educational efforts around MRE, including courses, minors, and degree programs, but they have not been well integrated or widely adopted, especially in USA. To enable MRE to coalesce as a distinct and identifiable engineering field, the authors conducted four workshops on the Future of Mechatronics and Robotics Engineering (FoMRE) education at the bachelor’s degree level. The overall goal of the workshops was to improve the quality of undergraduate MRE education and to ease the adoption of teaching materials to prepare graduates with a blend of theoretical knowledge and practical hands-on skills. To realize this goal, the specific objectives were to generate enthusiasm and a sense of community among current and future MRE educators, promote diversity and inclusivity within the MRE community, identify thought leaders, and seek feedback from the community to serve as a foundation for future activities. The workshops were intended to benefit a wide range of participants including educators currently teaching or developing programs in MRE, PhD students seeking academic careers in MRE, and industry professionals desiring to shape the future workforce. Workshop activities included short presentations on sample MRE programs, breakout sessions on specific topics, and open discussion sessions. As a result of these workshops, the MRE educational community has been enlarged and engaged, with members actively contributing to the scholarship of teaching and learning. This paper presents the workshops’ formats, outcomes, results of participant surveys, and their analyses. A major outcome was identifying concept, skill, and experience inventories organized around the dimensions of foundational/practical/applications and student preparation/MRE knowledgebase. Particular attention is given to the extent to which the workshops realized the project goals, including attendee demographics, changes in participant attitudes, and development of the MRE community. The paper concludes with a summary of lessons learned and a call for future activities to shape the field. 
    more » « less
  3. Nicewonger, Todd E. ; McNair, Lisa D. ; Fritz, Stacey (Ed.)
    https://pressbooks.lib.vt.edu/alaskanative/ At the start of the pandemic, the editors of this annotated bibliography initiated a remote (i.e., largely virtual) ethnographic research project that investigated how COVID-19 was impacting off-site modular construction practices in Alaska Native communities. Many of these communities are located off the road system and thus face not only dramatically higher costs but multiple logistical challenges in securing licensed tradesmen and construction crews and in shipping building supplies and equipment to their communities. These barriers, as well as the region’s long winters and short building seasons, complicate the construction of homes and related infrastructure projects. Historically, these communities have also grappled with inadequate housing, including severe overcrowding and poor-quality building stock that is rarely designed for northern Alaska’s climate (Marino 2015). Moreover, state and federal bureaucracies and their associated funding opportunities often further complicate home building by failing to accommodate the digital divide in rural Alaska and the cultural values and practices of Native communities.[1] It is not surprising, then, that as we were conducting fieldwork for this project, we began hearing stories about these issues and about how the restrictions caused by the pandemic were further exacerbating them. Amidst these stories, we learned about how modular home construction was being imagined as a possible means for addressing both the complications caused by the pandemic and the need for housing in the region (McKinstry 2021). As a result, we began to investigate how modular construction practices were figuring into emergent responses to housing needs in Alaska communities. We soon realized that we needed to broaden our focus to capture a variety of prefabricated building methods that are often colloquially or idiomatically referred to as “modular.” This included a range of prefabricated building systems (e.g., manufactured, volumetric modular, system-built, and Quonset huts and other reused military buildings[2]). Our further questions about prefabricated housing in the region became the basis for this annotated bibliography. Thus, while this bibliography is one of multiple methods used to investigate these issues, it played a significant role in guiding our research and helped us bring together the diverse perspectives we were hearing from our interviews with building experts in the region and the wider debates that were circulating in the media and, to a lesser degree, in academia. The actual research for each of three sections was carried out by graduate students Lauren Criss-Carboy and Laura Supple.[3] They worked with us to identify source materials and their hard work led to the team identifying three themes that cover intersecting topics related to housing security in Alaska during the pandemic. The source materials collected in these sections can be used in a variety of ways depending on what readers are interested in exploring, including insights into debates on housing security in the region as the pandemic was unfolding (2021-2022). The bibliography can also be used as a tool for thinking about the relational aspects of these themes or the diversity of ways in which information on housing was circulating during the pandemic (and the implications that may have had on community well-being and preparedness). That said, this bibliography is not a comprehensive analysis. Instead, by bringing these three sections together with one another to provide a snapshot of what was happening at that time, it provides a critical jumping off point for scholars working on these issues. The first section focuses on how modular housing figured into pandemic responses to housing needs. In exploring this issue, author Laura Supple attends to both state and national perspectives as part of a broader effort to situate Alaska issues with modular housing in relation to wider national trends. This led to the identification of multiple kinds of literature, ranging from published articles to publicly circulated memos, blog posts, and presentations. These materials are important source materials that will likely fade in the vastness of the Internet and thus may help provide researchers with specific insights into how off-site modular construction was used – and perhaps hyped – to address pandemic concerns over housing, which in turn may raise wider questions about how networks, institutions, and historical experiences with modular construction are organized and positioned to respond to major societal disruptions like the pandemic. As Supple pointed out, most of the material identified in this review speaks to national issues and only a scattering of examples was identified that reflect on the Alaskan context. The second section gathers a diverse set of communications exploring housing security and homelessness in the region. The lack of adequate, healthy housing in remote Alaska communities, often referred to as Alaska’s housing crisis, is well-documented and preceded the pandemic (Guy 2020). As the pandemic unfolded, journalists and other writers reported on the immense stress that was placed on already taxed housing resources in these communities (Smith 2020; Lerner 2021). The resulting picture led the editors to describe in their work how housing security in the region exists along a spectrum that includes poor quality housing as well as various forms of houselessness including, particularly relevant for the context, “hidden homelessness” (Hope 2020; Rogers 2020). The term houseless is a revised notion of homelessness because it captures a richer array of both permanent and temporary forms of housing precarity that people may experience in a region (Christensen et al. 2107). By identifying sources that reflect on the multiple forms of housing insecurity that people were facing, this section highlights the forms of disparity that complicated pandemic responses. Moreover, this section underscores ingenuity (Graham 2019; Smith 2020; Jason and Fashant 2021) that people on the ground used to address the needs of their communities. The third section provides a snapshot from the first year of the pandemic into how CARES Act funds were allocated to Native Alaska communities and used to address housing security. This subject was extremely complicated in Alaska due to the existence of for-profit Alaska Native Corporations and disputes over eligibility for the funds impacted disbursements nationwide. The resources in this section cover that dispute, impacts of the pandemic on housing security, and efforts to use the funds for housing as well as barriers Alaska communities faced trying to secure and use the funds. In summary, this annotated bibliography provides an overview of what was happening, in real time, during the pandemic around a specific topic: housing security in largely remote Alaska Native communities. The media used by housing specialists to communicate the issues discussed here are diverse, ranging from news reports to podcasts and from blogs to journal articles. This diversity speaks to the multiple ways in which information was circulating on housing at a time when the nightly news and radio broadcasts focused heavily on national and state health updates and policy developments. Finding these materials took time, and we share them here because they illustrate why attention to housing security issues is critical for addressing crises like the pandemic. For instance, one theme that emerged out of a recent National Science Foundation workshop on COVID research in the North NSF Conference[4] was that Indigenous communities are not only recovering from the pandemic but also evaluating lessons learned to better prepare for the next one, and resilience will depend significantly on more—and more adaptable—infrastructure and greater housing security. 
    more » « less
  4. null (Ed.)
    To enable the sustainable use of their ocean resources, capacity for ocean science and observations is important for every coastal nation. In many developing areas of the world, capability for ocean science and observations is not yet adequate to meet management needs. International organizations have employed a variety of capacity development approaches to assist developing countries in building self-sustaining ocean science and observational communities. This article describes the lessons learned from visiting scientist programs conducted for more than a decade by the Partnership for Observation of the Global Ocean (POGO) and the Scientific Committee on Oceanic Research (SCOR) that dispatched ocean scientists to developing countries to train hundreds of individuals in a variety of ocean science and observation topics and techniques. From these programs, SCOR and POGO have learned that training in-country has multiple benefits to trainees, host institutions, and trainers, benefits that are not achievable when students leave their countries. These benefits include more cost-effective training on issues relevant to the host institutions using locally available technology, as well as the ability to reach a large number of trainees. Lessons learned from the POGO and SCOR programs can be used to inform the future capacity-development activities of POGO and SCOR, as well as other organizations, to improve, enhance, and expand the use of in-country training and mentoring. Such approaches could contribute to the capacity development efforts of the UN Decade of Ocean Science for Sustainable Development. 
    more » « less
  5. Foundational engineering courses are critical to student success in engineering programs. The conceptually challenging content of these courses establishes the requisite knowledge for future classes. Thus, it is no surprise that such courses can serve as barriers or gatekeepers to successful student progress through the undergraduate curriculum. Although the difficulty of the courses may be necessary, often other features of the course delivery such as large class environments or a few very high-stakes assessments can further exacerbate these challenges. And especially problematic, past studies have shown that grade penalties associated with these courses and environments may disproportionately impact women. On the faculty side, institutions often turn to non-tenure track instructional faculty to teach multiple sections of foundational courses each semester. Although having faculty whose sole role is dedicated to quality teaching is an asset, benefits would likely be maximized when such faculty have clear metrics for paths to promotion, some autonomy and ownership regarding the curriculum, and overall job satisfaction. However, literature suggests that faculty, like students, note ill effects from large classes, such as challenges connecting and building rapport with students and having time to offer individualized feedback to students. Our NSF IUSE project focuses on instructors of large foundational engineering students with the belief that by better understanding the educational environment from their perspective we can improve the quality of the teaching and learning environment for all engineering students. Our project regularly convenes faculty teaching an array of core courses (e.g,. Mathematics, Chemistry, Mechanics, Physics) and uses insights from these meetings and individual interviews to identify possible leverage points where our project or the institution more broadly might affect change. Parallel to this effort, we have been working with data stewards on campus to gain access to institutional data (e.g., student course and grade histories, student evaluations of faculty teaching) to link and provide aggregate deidentified results to faculty to feed more information in to their decision-making. We are demonstrating that regular engagement between faculty and institutional leaders around analyzed and curated data is essential to continuous and systematic improvement. Efforts to date have included building an institutional data explorer dashboard (e.g., influences of pre-requisite courses on future courses) and drafting reports to be sent to department heads and associate deans which gather priorities identified in the first year of our research. For example, participating instructors identified that clarity of promotion paths across non-tenure track teaching faculty from different departments varied greatly, and the institution as a whole could benefit from clarified university-wide guidance. While some findings may be institution-specific (NSF IUSE Institutional Transformation track), as a large public research institution, peer-institutions with high engineering enrollments often face similar challenges and so findings from our change efforts potentially have broad applicability. 
    more » « less