An official website of the United States government
The lack of a readily accessible, tightly integrated data fabric connecting high-speed networking, storage, and computing services remains a critical barrier to the democratization of scientific discovery. To address this challenge, we are building National Science Data Fabric (NSDF), a holistic ecosystem to facilitate domain scientists in their daily research. NSDF comprises networking, storage, and computing services, as well as outreach initiatives. In this paper, we present a testbed integrating three services (i.e., networking, storage, and computing). We evaluate their performance. Specifically, we study the networking services and their throughput and latency with a focus on academic cloud providers; the storage services and their performance with a focus on data movement using file system mappers for both academic and commercial clouds; and computing orchestration services focusing on commercial cloud providers. We discuss NSDF's potential to increase scalability and usability as it decreases time-to-discovery across scientific domains.
more »
« less
COMET: Distributed Metadata Service for Multi-cloud Experiments
A majority of today's cloud services are independently operated by individual cloud service providers. In this approach, the locations of cloud resources are strictly constrained by the distribution of cloud service providers' sites. As the popularity and scale of cloud services increase, we believe this traditional paradigm is about to change toward further federated services, a.k.a., multi-cloud, due to the improved performance, reduced cost of compute, storage and network resources, as well as increased user demands. In this paper, we present COMET, a lightweight, distributed storage system for managing metadata on large scale, federated cloud infrastructure providers, end users, and their applications (e.g. HTCondor Cluster or Hadoop Cluster). We showcase use case from NSF's, Chameleon, ExoGENI and JetStream research cloud testbeds to show the effectiveness of COMET design and deployment.
more »
« less
- Award ID(s):
- 1826997
- PAR ID:
- 10158250
- Date Published:
- Journal Name:
- 2019 IEEE 27th International Conference on Network Protocols (ICNP)
- Page Range / eLocation ID:
- 1 to 2
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Transient computing has become popular in public cloud environments for running delay-insensitive batch and data processing applications at low cost. Since transient cloud servers can be revoked at any time by the cloud provider, they are considered unsuitable for running interactive application such as web services. In this paper, we present VM deflation as an alternative mechanism to server preemption for reclaiming resources from transient cloud servers under resource pressure. Using real traces from top-tier cloud providers, we show the feasibility of using VM deflation as a resource reclamation mechanism for interactive applications in public clouds. We show how current hypervisor mechanisms can be used to implement VM deflation and present cluster deflation policies for resource management of transient and on-demand cloud VMs. Experimental evaluation of our deflation system on a Linux cluster shows that microservice-based applications can be deflated by up to 50% with negligible performance overhead. Our cluster-level deflation policies allow overcommitment levels as high as 50%, with less than a 1% decrease in application throughput, and can enable cloud platforms to increase revenue by 30%more » « less
-
The growing demand of industrial, automotive and service robots presents a challenge to the centralized Cloud Robotics model in terms of privacy, security, latency, bandwidth, and reliability. In this paper, we present a ‘Fog Robotics’ approach to deep robot learning that distributes compute, storage and networking resources between the Cloud and the Edge in a federated manner. Deep models are trained on non-private (public) synthetic images in the Cloud; the models are adapted to the private real images of the environment at the Edge within a trusted network and subsequently, deployed as a service for low-latency and secure inference/prediction for other robots in the network. We apply this approach to surface decluttering, where a mobile robot picks and sorts objects from a cluttered floor by learning a deep object recognition and a grasp planning model. Experiments suggest that Fog Robotics can improve performance by sim-to-real domain adaptation in comparison to exclusively using Cloud or Edge resources, while reducing the inference cycle time by 4 to successfully declutter 86% of objects over 213 attempts.more » « less
-
null (Ed.)Cloud Legal documents, like Privacy Policies and Terms of Services (ToS), include key terms and rules that enable consumers to continuously monitor the performance of the cloud services used in their organization. To ensure high consumer confidence in the cloud service, it is necessary that these documents are clear and comprehensible to the average consumer. However, in practice, service providers often use legalese and ambiguous language in cloud legal documents resulting in consumers consenting or rejecting the terms without understanding the details. A measure capturing ambiguity in the texts of cloud service documents will enable consumers to decide if they understand what they are agreeing to, and deciding whether that service will meet their organizational requirements. It will also allow them to compare the service policies across various vendors. We have developed a novel model, ViCLOUD, that defines a scoring method based on linguistic cues to measure ambiguity in cloud legal documents and compare them to other peer websites. In this paper, we describe the ViCLOUD model in detail along with the validation results when applying it to 112 privacy policies and 108 Terms of Service documents of 115 cloud service vendors. The score distribution gives us a landscape of current trends in cloud services and a scale of comparison for new documentation. Our model will be very useful to organizations in making judicious decisions when selecting their cloud service.more » « less
-
While cloud storage has become a common practice for more and more organizations, many severe cloud data breaches in recent years show that protecting sensitive data in the cloud is still a challenging problem. Although various mitigation techniques have been proposed, they are not scalable for large scale enterprise users with strict security requirements or often depend on error-prone human interventions. To address these issues, we propose FileCrypt, a generic proxy-based technique for enterprise users to automatically secure sensitive files in browser-based cloud storage. To the best of our knowledge, FileCrypt is the first attempt towards transparent and fully automated file encryption for browser-based cloud storage services. More importantly, it does not require active cooperations from cloud providers or modifications of existing cloud applications. By instrumenting mandatory file-related JavaScript APIs in browsers, FileCrypt can naturally support new cloud storage services and guarantee the file encryption cannot be bypassed. We have evaluated the efficacy of FileCrypt on a number of popular realworld cloud storage services. The results show that it can protect files on the public cloud with relatively low overheads.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICNP.2019.8888138
