skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Deploying an Educational JupyterHub for Exploratory Data Analysis, Visualization, and Running Idealized Weather Models on the Jetstream2 Cloud
Title: Deploying an Educational JupyterHub for Exploratory Data Analysis, Visualization, and Running Idealized Weather Models on the Jetstream2 Cloud
The Unidata Program Center dedicates two software engineers to the development and maintenance of a science gateway meant to serve the members of the Earth Systems Science community. Unidata collaborated with one such community member, Dr. Greg Blumberg of the Department of Earth Sciences at Millerville University, in order to provide three undergraduate courses in atmospheric science with access to a custom JupyterHub cluster on the Jetstream2 Cloud boasting preconfigured environments, a shared network drive, and the capability to enable machine learning education and the execution of the Weather Research and Forecasting (WRF) model. The implementation of these features through the Kubernetes orchestration engine is discussed in detail, including initial failures of the Unidata Science Gateway team and the resolution of the issues that arose as a result. The performance of WRF executed at scale using JupyterHub is discussed at a surface level, with more study necessary to make further conclusions. Finally, feedback from Dr. Blumberg, both positive and constructive, is discussed along with specific use cases for the cyberinfrastructure.  more » « less
Award ID(s):
2005506
PAR ID:
10546271
Author(s) / Creator(s):
; ;
Publisher / Repository:
Zenodo
Date Published:
Subject(s) / Keyword(s):
Science Gateways Cyberinfrastructure Meteorology Atmospheric Science Education Jupyterhub Kubernetes Docker
Format(s):
Medium: X
Right(s):
Creative Commons Attribution 4.0 International
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Zenodo)
    To improve accessibility and community knowledge of applications in the Lidar Radar Open Software Environment (LROSE), a team from the National Science Foundation (NSF) National Center for Atmospheric Research, Colorado State University, and NSF Unidata has developed a lidar and radar meteorology science gateway deployed on the NSF Jetstream2 cloud. Utilizing the “Zero to JupyterHub with Kubernetes” workflow, the science gateway integrates LROSE with other lidar and radar meteorology software packages. This integration allows users to execute applications directly from the JupyterLab terminal, streamlining the creation of datasets for further analysis and visualization within Jupyter notebooks. By combining traditional command-line operations with modern Python-based tools for data analysis and visualization, this gateway provides a robust end-to-end solution that caters to both educational and research needs. The gateway has already facilitated LROSE instructional workshops and classroom exercises. Our work demonstrates the significant potential of merging established scientific computing techniques with advanced Python environments, opening new avenues for computational science education and research. The LROSE team has acquired successive allocations on the NSF Jetstream2 cloud at Indiana University through ACCESS. To develop the LROSE Science Gateway, we employed the “Zero to JupyterHub with Kubernetes” workflow ported to the NSF Jetstream2 cloud, enabling rapid and scalable deployment to accommodate a variable number of users. Authentication is managed through either GitHub OAuth or temporary credentials, depending on the situation. Since LROSE is a collection of C/C++ applications, we configured Docker containers based on the Jupyter Docker Stack to integrate the LROSE software, available via the JupyterLab terminal. These containers also include Conda package manager environments equipped with Python packages like Py-ART, CSU RadarTools, and Metpy for further data analysis. A shared drive accessible to all participants contains instructional datasets for lidar and radar data analysis. Tutorials take the form of Jupyter notebooks for use by individuals, in classroom exercises, or at instructional workshops. Some tutorials are complete with pre-loaded examples to quickly visualize workflows and results. Other tutorials guide students how to run the applications independently. All tutorials are hosted on the LROSE Science Gateway GitHub repository, which is open to contributions from colleagues and community members. Future plans include an "intermediate" level workshop on SAMURAI, one of the multi-Doppler wind applications of the LROSE suite. Additionally, work is currently underway to run GUI applications in the same browser-based JupyterLab environment. GUI applications for radar and lidar data visualization utilize the QT framework and present unique technical challenges. The techniques to accomplish GUI access have immediate applications for other GUI programs, such as NSF Unidata's IDV and their version of the AWIPS CAVE data visualization tools. Lastly, as demand for the resources found on the gateway increases, it becomes increasingly important to efficiently manage the Jetstream2 resources allocated by the ACCESS program. LROSE, NSF Unidata, San Diego Supercomputing Center (SDSC), and Indiana University staff are working together to deploy and evaluate Kubernetes cluster auto-scaling. With auto-scaling, resources will no longer sit idle while awaiting new logins and will instead be provisioned on-demand. 
    more » « less
  2. ; ; ; ; ; (, Zenodo)
    This paper introduces a lidar and radar meteorology science gateway deployed on the NSF Jetstream2 cloud, designed to enhance educational and research activities in atmospheric science. Utilizing the "Zero to JupyterHub with Kubernetes" workflow, we have created a science gateway that integrates lidar and radar meteorology software packages, notably the Lidar Radar Open Software Environment (LROSE). This integration allows users to execute applications directly from the JupyterLab terminal, streamlining the creation of datasets for further anal- ysis and visualization within Jupyter notebooks. By combining traditional command-line operations with modern Python-based tools for data analysis and visualization, this gateway provides a robust end-to-end solution that caters to both educational and research needs. The gateway has already been vital in facilitating LROSE instructional workshops and will see future enhancements such as GPU acceleration to boost performance. Our work demonstrates the significant potential of merging established scientific computing techniques with advanced Python environments, opening new avenues for computational science education and research. 
    more » « less
  3. ; ; ; ; ; ; ; ; ; ; et al (, Data)
    Planetary Defense (PD) has become a critical effort of protecting our home planet by discovering potentially hazardous objects (PHOs), simulating the potential impact, and mitigating the threats. Due to the lack of structured architecture and framework, pertinent information about detecting and mitigating near earth object (NEO) threats are still dispersed throughout numerous organizations. Scattered and unorganized information can have a significant impact at the time of crisis, resulting in inefficient processes, and decisions made on incomplete data. This PD Mitigation Gateway (pd.cloud.gmu.edu) is developed and embedded within a framework to integrate the dispersed, diverse information residing at different organizations across the world. The gateway offers a home to pertinent PD-related contents and knowledge produced by the NEO mitigation team and the community through (1) a state-of-the-art smart-search discovery engine based on PD knowledge base; (2) a document archiving and understanding mechanism for managing and utilizing the results produced by the PD science community; (3) an evolving PD knowledge base accumulated from existing literature, using natural language processing and machine learning; and (4) a 4D visualization tool that allows the viewers to analyze near-Earth approaches in a three-dimensional environment using dynamic, adjustable PHO parameters to mimic point-of-impact asteroid deflections via space vehicles and particle system simulations. Along with the benefit of accessing dispersed data from a single port, this framework is built to advance discovery, collaboration, innovation, and education across the PD field-of-study, and ultimately decision support. 
    more » « less
  4. ; ; ; ; (, Zenodo)
    Community growth is one of the cornerstones contributing to the sustainability of a science gateway. Achieving community growth requires careful planning and a multifaceted approach. The Science Gateways Community Institute (SGCI) and the Center of Excellence for Science Gateways (SGX3) offer services such as UX advice, sustainability training via the Focus Week, and an annual conference to support the science gateway community with developers and users. This panel will discuss four successful use cases – QUBES, MyGeoHub, CHEESE, and the Hawaii Behavioral Health Dashboard – where the teams utilized various SGCI/SGX3 services, which significantly contributed to their community growth. The discussion will highlight specific strategies and outcomes from these use cases, providing valuable insights into the effective practices that drive community engagement and sustainability in science gateways. Additionally, panelists will share lessons learned and good practices that can be applied to other science gateways seeking to enhance their community presence and impact. 
    more » « less
  5. ; ; ; (, Gateway Computing Environments Workshop)
    SCAIGATE is an ambitious project to design the first AI-centric science gateway based on field-programmable gate arrays (FPGAs). The goal is to democratize access to FPGAs and AI in scientific computing and related applications. When completed, the project will enable the large-scale deployment and use of machine learning models on AI-centric FPGA platforms, allowing increased performance-efficiency, reduced development effort, and customization at unprecedented scale, all while simplifying ease-of-use in science domains which were previously AI-lagging. SCAIGATE was an incubation project at the Science Gateway Community Institute (SGCI) bootcamp held in Austin, Texas in 2018. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email