Experience Migrating a Pipeline for the C-MĀIKI gateway from Tapis v2 to Tapis v3
- Award ID(s):
- 1931439
- PAR ID:
- 10442807
- Date Published:
- Journal Name:
- Practice and Experience in Advanced Research Computing (2022)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
The C-MĀIKI gateway is a science gateway that leverages a computational workload management API called Tapis to support modern, interoperable, and scalable microbiome data analysis. This project is focused on migrating an existing C-MĀIKI gateway pipeline from Tapis v2 to Tapis v3 so that it can take advantage of the new robust Tapis v3 features and stay modern. This requires three major steps: 1) Containerization of each existing microbiome workflow. 2) Create a new app definition for each of the workflows. 3) Enabling the ability to submit jobs to a SLURM scheduler inside of a singularity container to support the Nextflow workflow manager. This work presents the experience and challenges in upgrading the pipeline.more » « less
-
The Tapis Pods service is a novel open-source API within the Tapis platform which enables researchers to seamlessly manage Kubernetes containers, volumes, networking, and security at the Texas Advanced Computing Center (TACC). This paper explores the underlying operations, technologies, and workflows of the Tapis Pods service, showcasing its implementation and effectiveness. Additionally, we discuss current and potential use cases, highlighting the service's unique features, such as management capabilities, persistent storage, sharing, and automatically encrypted networking. Initial performance measurements against local Docker containers and alternative cloud solutions demonstrate the Tapis Pods service's competitive performance, emphasizing its value as a general interface for deploying user-defined containers.more » « less
-
The adaptation of machine learning (ML) in scientific and medical research in recent years has heralded a new era of innovation, catalyzing breakthroughs that were once deemed unattainable. In this paper, we present the Machine Learning Hub (ML Hub) – a web application offering a single point of access to pre-trained ML models and datasets, catering to users across varying expertise levels. Built upon the NSF-funded Tapis v3 Application Programming Interface (API) and Tapis User Interface (TapisUI), the platform offers a user-friendly interface for model discovery, dataset exploration, and inference server deployment.more » « less
An official website of the United States government

