An official website of the United States government
Kubernetes, an open-source container orchestration platform, has been widely adopted by cloud service providers (CSPs) for its advantages in simplifying container deployment, scalability and scheduling. Networking is one of the central components of Kubernetes, providing connectivity between different pods (group of containers) both within the same host and across hosts. To bootstrap Kubernetes networking, the Container Network Interface (CNI) provides a unified interface for the interaction between container runtimes. There are several CNI implementations, available as open-source ‘CNI plugins’. While they differ in functionality and performance, it is a challenge for a cloud provider to differentiate and choose the appropriate plugin for their environment. In this paper, we compare the various open source CNI plugins available from the community, qualitatively and through detailed quantitative measurements. With our experimental evaluation, we analyze the overheads and bottlenecks for each CNI plugin, as a result of the network model it implements, interaction with the host network protocol stack and the network policies implemented in iptables rules. The choice of the CNI plugin may also be based on whether intra-host or inter-host communication dominates.
more »
« less
Tapis Pods Service Exploration and Initial Performance Analysis
The Tapis Pods service is a novel open-source API within the Tapis platform which enables researchers to seamlessly manage Kubernetes containers, volumes, networking, and security at the Texas Advanced Computing Center (TACC). This paper explores the underlying operations, technologies, and workflows of the Tapis Pods service, showcasing its implementation and effectiveness. Additionally, we discuss current and potential use cases, highlighting the service's unique features, such as management capabilities, persistent storage, sharing, and automatically encrypted networking. Initial performance measurements against local Docker containers and alternative cloud solutions demonstrate the Tapis Pods service's competitive performance, emphasizing its value as a general interface for deploying user-defined containers.
more »
« less
- Award ID(s):
- 1931439
- PAR ID:
- 10649772
- Publisher / Repository:
- Zenodo
- Date Published:
- Subject(s) / Keyword(s):
- containers-as-a-service cloud infrastructure open-source API Kubernetes
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The Change Hawaii (Change(HI)) project is fundamentally addressing the existential threat of climate change in Hawaii by integrating data and climate science to foster statewide resilience, enhance decision science, and support workforce development in critical fields. A cornerstone of this initiative is the \textbf{Hawaii Climate Data Portal (HCDP)}, which operates as a vital science gateway and data hub \cite. The HCDP's primary objective is to build capacity through advanced data science and artificial intelligence (AI), serving as a robust resource for monitoring, visualizing, and communicating environmental change \cite{longman_hawaii_2024}. Its critical role is highlighted by its extensive provision of climate data and its Application Programming Interface (API), which is instrumental in the development and functionality of diverse decision support tools tailored for various stakeholders across the state. This paper details the HCDP's integration with the Tapis API platform, and its successful application in developing actionable climate science outcomes for Hawaii.more » « less
-
Summary The explosion of IoT devices and sensors in recent years has led to a demand for efficiently storing, processing and analyzing time‐series data. Geoscience researchers use time‐series data stores such as Hydroserver, Virtual Observatory and Ecological Informatics System (VOEIS), and Cloud‐Hosted Real‐time Data Service (CHORDS). Many of these tools require a great deal of infrastructure to deploy and expertise to manage and scale. The Tapis framework, an NSF funded project, provides science as a service APIs to allow researchers to achieve faster scientific results, by eliminating the need to set up a complex infrastructure stack. The University of Hawai'i (UH) and Texas Advanced Computing Center (TACC) have collaborated to develop an open source Tapis Streams API that builds on the concepts of the CHORDS time series data service to support research. This new hosted service allows storing, processing, annotating, archiving, and querying time‐series data in the Tapis multi‐user and multi‐tenant collaborative platform. The Streams API provides a hosted production level middleware service that enables new data‐driven event workflows capabilities that may be leveraged by researchers and Tapis powered science gateways for handling spatially indexed time‐series datasets.more » « less
-
This paper describes EdgeNet, a lightweight cloud infrastructure for the edge. We aim to bring as much of the flexibility of open cloud computing as possible to a very lightweight, easily-deployed, software-only edge infrastructure. EdgeNet has been informed by the advances of cloud computing and the successes of such distributed systems as PlanetLab, GENI, G-Lab, SAVI, and V-Node: a large number of small points-of-presence, designed for the deployment of highly distributed experiments and applications. EdgeNet differs from its predecessors in two significant areas: first, it is a software-only infrastructure, where each worker node is designed to run part- or full-time on existing hardware at the local site; and, second, it uses modern, industry-standard software both as the node agent and the control framework. The first innovation permits rapid and unlimited scaling: whereas GENI and PlanetLab required the installation and maintenance of dedicated hardware at each site, EdgeNet requires only a software download, and a node can be added to the EdgeNet infrastructure in 15 minutes. The second offers performance, maintenance, and training benefits; rather than maintaining bespoke kernels and control frameworks, and developing training materials on using the latter, we are able to ride the wave of open-source and industry development, and the plethora of industry and community tutorial materials developed for industry standard control frameworks. The result is a global Kubernetes cluster, where pods of Docker containers form the service instances at each point of presence.more » « less
-
Constructing and executing reproducible workflows is fundamental to performing research in a variety of scientific domains. Many of the current commercial and open source solutions for workflow en- gineering impose constraints—either technical or budgetary—upon researchers, requiring them to use their limited funding on expensive cloud platforms or spend valuable time acquiring knowledge of software systems and processes outside of their domain expertise. Even though many commercial solutions offer free-tier services, they often do not meet the resource and architectural requirements (memory, data storage, compute time, networking, etc) for researchers to run their workflows effectively at scale. Tapis Workflows abstracts away the complexities of workflow creation and execution behind a web-based API with a simplified workflow model comprised of only pipelines and tasks. This paper will de- tail how Tapis Workflows approaches workflow management by exploring its domain model, the technologies used, application architecture, design patterns, how organizations are leveraging Tapis Workflows to solve unique problems in their scientific workflows, and this projects’s vision for a simple, open source, extensible, and easily deployable workflow engine.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.5281/zenodo.10034632
