skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


This content will become publicly available on August 28, 2026

Title: Improved Accessibility and Community Knowledge of Lidar and Radar Data Analysis
To improve accessibility and community knowledge of applications in the Lidar Radar Open Software Environment (LROSE), a team from the National Science Foundation (NSF) National Center for Atmospheric Research, Colorado State University, and NSF Unidata has developed a lidar and radar meteorology science gateway deployed on the NSF Jetstream2 cloud. Utilizing the “Zero to JupyterHub with Kubernetes” workflow, the science gateway integrates LROSE with other lidar and radar meteorology software packages. This integration allows users to execute applications directly from the JupyterLab terminal, streamlining the creation of datasets for further analysis and visualization within Jupyter notebooks. By combining traditional command-line operations with modern Python-based tools for data analysis and visualization, this gateway provides a robust end-to-end solution that caters to both educational and research needs. The gateway has already facilitated LROSE instructional workshops and classroom exercises. Our work demonstrates the significant potential of merging established scientific computing techniques with advanced Python environments, opening new avenues for computational science education and research. The LROSE team has acquired successive allocations on the NSF Jetstream2 cloud at Indiana University through ACCESS. To develop the LROSE Science Gateway, we employed the “Zero to JupyterHub with Kubernetes” workflow ported to the NSF Jetstream2 cloud, enabling rapid and scalable deployment to accommodate a variable number of users. Authentication is managed through either GitHub OAuth or temporary credentials, depending on the situation. Since LROSE is a collection of C/C++ applications, we configured Docker containers based on the Jupyter Docker Stack to integrate the LROSE software, available via the JupyterLab terminal. These containers also include Conda package manager environments equipped with Python packages like Py-ART, CSU RadarTools, and Metpy for further data analysis. A shared drive accessible to all participants contains instructional datasets for lidar and radar data analysis. Tutorials take the form of Jupyter notebooks for use by individuals, in classroom exercises, or at instructional workshops. Some tutorials are complete with pre-loaded examples to quickly visualize workflows and results. Other tutorials guide students how to run the applications independently. All tutorials are hosted on the LROSE Science Gateway GitHub repository, which is open to contributions from colleagues and community members. Future plans include an "intermediate" level workshop on SAMURAI, one of the multi-Doppler wind applications of the LROSE suite. Additionally, work is currently underway to run GUI applications in the same browser-based JupyterLab environment. GUI applications for radar and lidar data visualization utilize the QT framework and present unique technical challenges. The techniques to accomplish GUI access have immediate applications for other GUI programs, such as NSF Unidata's IDV and their version of the AWIPS CAVE data visualization tools. Lastly, as demand for the resources found on the gateway increases, it becomes increasingly important to efficiently manage the Jetstream2 resources allocated by the ACCESS program. LROSE, NSF Unidata, San Diego Supercomputing Center (SDSC), and Indiana University staff are working together to deploy and evaluate Kubernetes cluster auto-scaling. With auto-scaling, resources will no longer sit idle while awaiting new logins and will instead be provisioned on-demand.  more » « less
Award ID(s):
2103776
PAR ID:
10649240
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Zenodo
Date Published:
Edition / Version:
v1.2.0
Subject(s) / Keyword(s):
open source radar lidar science gateway LROSE
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, Zenodo)
    This paper introduces a lidar and radar meteorology science gateway deployed on the NSF Jetstream2 cloud, designed to enhance educational and research activities in atmospheric science. Utilizing the "Zero to JupyterHub with Kubernetes" workflow, we have created a science gateway that integrates lidar and radar meteorology software packages, notably the Lidar Radar Open Software Environment (LROSE). This integration allows users to execute applications directly from the JupyterLab terminal, streamlining the creation of datasets for further anal- ysis and visualization within Jupyter notebooks. By combining traditional command-line operations with modern Python-based tools for data analysis and visualization, this gateway provides a robust end-to-end solution that caters to both educational and research needs. The gateway has already been vital in facilitating LROSE instructional workshops and will see future enhancements such as GPU acceleration to boost performance. Our work demonstrates the significant potential of merging established scientific computing techniques with advanced Python environments, opening new avenues for computational science education and research. 
    more » « less
  2. ; ; (, Zenodo)
    The Unidata Program Center dedicates two software engineers to the development and maintenance of a science gateway meant to serve the members of the Earth Systems Science community. Unidata collaborated with one such community member, Dr. Greg Blumberg of the Department of Earth Sciences at Millerville University, in order to provide three undergraduate courses in atmospheric science with access to a custom JupyterHub cluster on the Jetstream2 Cloud boasting preconfigured environments, a shared network drive, and the capability to enable machine learning education and the execution of the Weather Research and Forecasting (WRF) model. The implementation of these features through the Kubernetes orchestration engine is discussed in detail, including initial failures of the Unidata Science Gateway team and the resolution of the issues that arose as a result. The performance of WRF executed at scale using JupyterHub is discussed at a surface level, with more study necessary to make further conclusions. Finally, feedback from Dr. Blumberg, both positive and constructive, is discussed along with specific use cases for the cyberinfrastructure. 
    more » « less
  3. ; ; ; ; ; ; ; (, 2020 IEEE/ACM 9th Workshop on Python for High-Performance and Scientific Computing (PyHPC))
    null (Ed.)
    We describe JetLag, a Python-based environment that provides access to a distributed, interactive, asynchronous many-task (AMT) computing framework called Phylanx. This environment encompasses the entire computing process, from a Jupyter front-end for managing code and results to the collection and visualization of performance data.We use a Python decorator to access the abstract syntax tree of Python functions and transpile them into a set of C++ data structures which are then executed by the HPX runtime. The environment includes services for sending functions and their arguments to run as jobs on remote resources.A set of Docker and Singularity containers are used to simplify the setup of the JetLag environment. The JetLag system is suitable for a variety of array computational tasks, including machine learning and exploratory data analysis. 
    more » « less
  4. ; ; (, Computer Graphics Forum)
    Computational notebooks, such as Jupyter, support rich data visualization. However, even when visualizations in notebooks are interactive, they are a dead end: Interactive data manipulations, such as selections, applying labels, filters, categorizations, or fixes to column or cell values, could be efficiently applied in interactive visual components, but interactive components typically cannot manipulate Python data structures. Furthermore, actions performed in interactive plots are lost as soon as the cell is re‐run, prohibiting reusability and reproducibility. To remedy this problem, we introduce Persist, a family of techniques to (a) capture interaction provenance, enabling the persistence of interactions, and (b) map interactions to data manipulations that can be applied to dataframes. We implement our approach as a JupyterLab extension that supports tracking interactions in Vega‐Altair plots and in a data table view. Persist can re‐execute interaction provenance when a notebook or a cell is re‐executed, enabling reproducibility and re‐use. We evaluate Persist in a user study targeting data manipulations with 11 participants skilled in Python and Pandas, comparing it to traditional code‐based approaches. Participants were consistently faster and were able to correctly complete more tasks with Persist. 
    more » « less
  5. ; ; ; ; ; ; ; ; ; ; et al (, Bioinformatics)
    Marschall, Tobias (Ed.)
    Abstract MotivationJBrowse Jupyter is a package that aims to close the gap between Python programming and genomic visualization. Web-based genome browsers are routinely used for publishing and inspecting genome annotations. Historically they have been deployed at the end of bioinformatics pipelines, typically decoupled from the analysis itself. However, emerging technologies such as Jupyter notebooks enable a more rapid iterative cycle of development, analysis and visualization. ResultsWe have developed a package that provides a Python interface to JBrowse 2’s suite of embeddable components, including the primary Linear Genome View. The package enables users to quickly set up, launch and customize JBrowse views from Jupyter notebooks. In addition, users can share their data via Google’s Colab notebooks, providing reproducible interactive views. Availability and implementationJBrowse Jupyter is released under the Apache License and is available for download on PyPI. Source code and demos are available on GitHub at https://github.com/GMOD/jbrowse-jupyter. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email