skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Jetstream2: Accelerating cloud computing via Jetstream
Jetstream2 will be a category I production cloud resource that is part of the National Science Foundation’s Innovative HPC Program. The project’s aim is to accelerate science and engineering by providing “on-demand” programmable infrastructure built around a core system at Indiana University and four regional sites. Jetstream2 is an evolution of the Jetstream platform, which functions primarily as an Infrastructure-as-a-Service cloud. The lessons learned in cloud architecture, distributed storage, and container orchestration have inspired changes in both hardware and software for Jetstream2. These lessons have wide implications as institutions converge HPC and cloud technology while building on prior work when deploying their own cloud environments. Jetstream2’s next-generation hardware, robust open-source software, and enhanced virtualization will provide a significant platform to further cloud adoption within the US research and education communities.  more » « less
Award ID(s):
2005506
PAR ID:
10296117
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ; ;
Date Published:
Journal Name:
Proceedings of the 2021 Practice & Experience in Advanced Research Computing (PEARC) Conference
Page Range / eLocation ID:
1 to 8
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, Computing in Science & Engineering)
    Over the past decade, the convergence of Cloud and High-Performance Computing (HPC) has undergone significant movement. We explore the evolution, motivations, and practicalities of establishing on-premise research cloud infrastructure and the complementary nature with HPC and commercial resources; under the belief that research clouds serve a unique role within research and education as a convergence accelerator. This role is highlighted through exploring the design tradeoffs in architecting research clouds versus HPC resources, focusing on the balance between utility, availability, and hardware utilization. The discussion provides insights from experiences with the National Science Foundation-supported Jetstream and Jetstream2 systems, showcasing convergence technologies and challenges. A variety of real-world use cases are provided that show the interplay between these computing paradigms; exploring use in research and education for interactive and iterative development, as an on-ramp to large-scale resources, as a powerful tool for education and workforce development, and for domain specific science gateways. 
    more » « less
  2. ; (, F1000Research)
    Background The digitization of biological specimens has revolutionized morphology, generating massive 3D datasets such as microCT scans. While open-source platforms like 3D Slicer and SlicerMorph have democratized access to advanced visualization and analysis software, a significant “compute gap” persists. Processing high-resolution 3D data requires high-end GPUs and substantial RAM, resources that are frequently unavailable at Primarily Undergraduate Institutions (PUIs) and other educational settings. This “digital divide” prevents many researchers and students from utilizing the very data and software that have been made open to them. Methods We present MorphoCloud, a platform designed to bridge this hardware barrier by providing on-demand, research-grade computing environments via a web browser. MorphoCloud utilizes an “IssuesOps” architecture, where users manage their remote workstations entirely through GitHub Issues using natural-language commands (e.g., /create, /unshelve). The technology stack leverages GitHub Issues and Actions for front-end and orchestration respectively, JetStream2 for backend compute, and Apache Guacamole to deliver a high-performance, GPU-accelerated desktop experience to any modern browser. Results The platform enables a streamlined lifecycle for remote instances, which come pre-configured with the SlicerMorph ecosystem, R/RStudio, and AI-assisted segmentation tools like NNInteractive and MEMOs. Users have access to a persistent storage volume that is decoupled from the instance. For educational purposes, MorphoCloud supports “Workshop” instances that allow for bulk provisioning and stay online continuously for short-term events. This identical environment ensures that instructors can conduct complex 3D workflows without the typical troubleshooting delays caused by heterogeneous student hardware. Conclusion MorphoCloud demonstrates that true scientific accessibility requires not just open data and software, but also open infrastructure. By abstracting the complexities of cloud administration into a simple, command-driven interface, MorphoCloud empowers researchers at under-resourced institutions to engage in high-performance morphological analysis and AI-assisted segmentation. 
    more » « less
  3. ; ; ; (, Zenodo)
    To improve accessibility and community knowledge of applications in the Lidar Radar Open Software Environment (LROSE), a team from the National Science Foundation (NSF) National Center for Atmospheric Research, Colorado State University, and NSF Unidata has developed a lidar and radar meteorology science gateway deployed on the NSF Jetstream2 cloud. Utilizing the “Zero to JupyterHub with Kubernetes” workflow, the science gateway integrates LROSE with other lidar and radar meteorology software packages. This integration allows users to execute applications directly from the JupyterLab terminal, streamlining the creation of datasets for further analysis and visualization within Jupyter notebooks. By combining traditional command-line operations with modern Python-based tools for data analysis and visualization, this gateway provides a robust end-to-end solution that caters to both educational and research needs. The gateway has already facilitated LROSE instructional workshops and classroom exercises. Our work demonstrates the significant potential of merging established scientific computing techniques with advanced Python environments, opening new avenues for computational science education and research. The LROSE team has acquired successive allocations on the NSF Jetstream2 cloud at Indiana University through ACCESS. To develop the LROSE Science Gateway, we employed the “Zero to JupyterHub with Kubernetes” workflow ported to the NSF Jetstream2 cloud, enabling rapid and scalable deployment to accommodate a variable number of users. Authentication is managed through either GitHub OAuth or temporary credentials, depending on the situation. Since LROSE is a collection of C/C++ applications, we configured Docker containers based on the Jupyter Docker Stack to integrate the LROSE software, available via the JupyterLab terminal. These containers also include Conda package manager environments equipped with Python packages like Py-ART, CSU RadarTools, and Metpy for further data analysis. A shared drive accessible to all participants contains instructional datasets for lidar and radar data analysis. Tutorials take the form of Jupyter notebooks for use by individuals, in classroom exercises, or at instructional workshops. Some tutorials are complete with pre-loaded examples to quickly visualize workflows and results. Other tutorials guide students how to run the applications independently. All tutorials are hosted on the LROSE Science Gateway GitHub repository, which is open to contributions from colleagues and community members. Future plans include an "intermediate" level workshop on SAMURAI, one of the multi-Doppler wind applications of the LROSE suite. Additionally, work is currently underway to run GUI applications in the same browser-based JupyterLab environment. GUI applications for radar and lidar data visualization utilize the QT framework and present unique technical challenges. The techniques to accomplish GUI access have immediate applications for other GUI programs, such as NSF Unidata's IDV and their version of the AWIPS CAVE data visualization tools. Lastly, as demand for the resources found on the gateway increases, it becomes increasingly important to efficiently manage the Jetstream2 resources allocated by the ACCESS program. LROSE, NSF Unidata, San Diego Supercomputing Center (SDSC), and Indiana University staff are working together to deploy and evaluate Kubernetes cluster auto-scaling. With auto-scaling, resources will no longer sit idle while awaiting new logins and will instead be provisioned on-demand. 
    more » « less
  4. ; ; ; (, IEEE 5G World Forum 2018)
    Practical testing of the latest wireless communications standards requires the availability of flexible radio frequency hardware, net-working and computing resources. We are providing a Cloud-based infrastructure which offers the necessary resources to carry out tests of the latest 5G standards. The testbed provides a Cloud-based Infrastructure as a Service. The research community can access hardware and software resources through a virtual platform that enables isolation and customization of experiments. In other words, researchers have control over the preferred experimental architecture and can run concurrent experiments on the same testbed. This paper introduces the resources that can be used to develop 5G testbeds and experiments. 
    more » « less
  5. ; ; ; ; ; (, Zenodo)
    This paper introduces a lidar and radar meteorology science gateway deployed on the NSF Jetstream2 cloud, designed to enhance educational and research activities in atmospheric science. Utilizing the "Zero to JupyterHub with Kubernetes" workflow, we have created a science gateway that integrates lidar and radar meteorology software packages, notably the Lidar Radar Open Software Environment (LROSE). This integration allows users to execute applications directly from the JupyterLab terminal, streamlining the creation of datasets for further anal- ysis and visualization within Jupyter notebooks. By combining traditional command-line operations with modern Python-based tools for data analysis and visualization, this gateway provides a robust end-to-end solution that caters to both educational and research needs. The gateway has already been vital in facilitating LROSE instructional workshops and will see future enhancements such as GPU acceleration to boost performance. Our work demonstrates the significant potential of merging established scientific computing techniques with advanced Python environments, opening new avenues for computational science education and research. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email