More Like this

1. FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices

   Qiu, Haoran ; Banerjee, Subho S. ; Jha, Saurabh ; Kalbarczyk, Zbigniew T. ; Iyer, Ravishankar K. ( November 2020 , Proceedings of The 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI ‘20))

   null (Ed.)

   User-facing latency-sensitive web services include numerous distributed, intercommunicating microservices that promise to simplify software development and operation. However, multiplexing of compute resources across microservices is still challenging in production because contention for shared resources can cause latency spikes that violate the service level objectives (SLOs) of user requests. This paper presents FIRM, an intelligent fine-grained resource management framework for predictable sharing of resources across microservices to drive up overall utilization. FIRM leverages online telemetry data and machine-learning methods to adaptively (a) detect/localize microservices that cause SLO violations, (b) identify low-level resources in contention, and (c) take actions to mitigate SLO violations via dynamic reprovisioning. Experiments across four microservice benchmarks demonstrate that FIRM reduces SLO violations by up to 16Å~ while reducing the overall requested CPU limit by up to 62%. Moreover, FIRM improves performance predictability by reducing tail latencies by up to 11Å~. 

   more » « less
2. E3: Energy-Efficient Microservices on SmartNIC-Accelerated Servers

   Liu, Ming ; Peter, Simon ; Krishnamurthy, Arvind ; Phothilimthana, Phitchaya M ( July 2019 , 2019 USENIX Annual Technical Conference (USENIX ATC 19))

   We investigate the use of SmartNIC-accelerated servers to execute microservice-based applications in the data center. By offloading suitable microservices to the SmartNIC’s low-power processor, we can improve server energy-efficiency without latency loss. However, as a heterogeneous computing substrate in the data path of the host, SmartNICs bring several challenges to a microservice platform: network traffic routing and load balancing, microservice placement on heterogeneous hardware, and contention on shared SmartNIC resources. We present E3, a microservice execution platform for SmartNIC-accelerated servers. E3 follows the design philosophies of the Azure Service Fabric microservice platform and extends key system components to a SmartNIC to address the above-mentioned challenges. E3 employs three key techniques: ECMP-based load balancing via SmartNICs to the host, network topology-aware microservice placement, and a data-plane orchestrator that can detect SmartNIC overload. Our E3 prototype using Cavium LiquidIO SmartNICs shows that SmartNIC offload can improve cluster energy-efficiency up to 3× and cost efficiency up to 1.9× at up to 4% latency cost for common microservices, including real-time analytics, an IoT hub, and virtual network functions. 

   more » « less
3. Nightcore: efficient and scalable serverless computing for latency-sensitive, interactive microservices

   https://doi.org/10.1145/3445814.3446701  

   Jia, Zhipeng ; Witchel, Emmett ( April 2021 , Architectrual Support for Programming Languages and Operating Systems)

   null (Ed.)

   The microservice architecture is a popular software engineering approach for building flexible, large-scale online services. Serverless functions, or function as a service (FaaS), provide a simple programming model of stateless functions which are a natural substrate for implementing the stateless RPC handlers of microservices, as an alternative to containerized RPC servers. However, current serverless platforms have millisecond-scale runtime overheads, making them unable to meet the strict sub-millisecond latency targets required by existing interactive microservices. We present Nightcore, a serverless function runtime with microsecond-scale overheads that provides container-based isolation between functions. Nightcore’s design carefully considers various factors having microsecond-scale overheads, including scheduling of function requests, communication primitives, threading models for I/O, and concurrent function executions. Nightcore currently supports serverless functions written in C/C++, Go, Node.js, and Python. Our evaluation shows that when running latency-sensitive interactive microservices, Nightcore achieves 1.36×–2.93× higher throughput and up to 69% reduction in tail latency. 

   more » « less
4. uDiscover: User-Driven Service Discovery in Pervasive Edge Computing using NDN

   https://doi.org/10.1109/EDGE55608.2022.00022  

   Torres, George ; Tourani, Reza ; Mtibaa, Abderrahmen ; Stelmakh, Diana ; Misra, Satyajayant ; Srikanteswara, Srikathyayani ; Zhang, Yi ; Hoque, Sanzida ( July 2022 , 2022 IEEE International Conference on Edge Computing and Communications (EDGE))

   New breed of applications, such as autonomous driving and their need for computation-aided quick decision making has motivated the delegation of compute-intensive services (e.g., video analytic) to the more powerful surrogate machines at the network edge–edge computing (EC). Recently, the notion of pervasive edge computing (PEC) has emerged, in which users’ devices can join the pool of the computing resources that perform edge computing. Inclusion of users’ devices increases the computing capability at the edge (adding to the infrastructure servers), but in comparison to the conventional edge ecosystems, it also introduces new challenges, such as service orchestration (i.e., service placement, discovery, and migration). We propose uDiscover, a novel user-driven service discovery and utilization framework for the PEC ecosystem. In designing uDiscover, we considered the Named-Data Networking architecture for balancing users workloads and reducing user-perceived latency. We propose proactive and reactive service discovery approaches and assess their performance in PEC and infrastructure-only ecosystems. Our simulation results show that (i) the PEC ecosystem reduces the user-perceived delays by up to 70%, and (ii) uDiscover selects the most suitable server–"accurate" delay estimates with less than 10% error–to execute any given task. 

   more » « less
5. Joint Service Placement and Request Routing in Multi-cell Mobile Edge Computing Networks

   Poularakis, K. ; Llorca, J. ; Tulino, A. ; Taylor, I. ; Tassiulas, L. ( April 2019 , IEEE International Conference on Computer Communications)

   The proliferation of innovative mobile services such as augmented reality, networked gaming, and autonomous driving has spurred a growing need for low-latency access to computing resources that cannot be met solely by existing centralized cloud systems. Mobile Edge Computing (MEC) is expected to be an effective solution to meet the demand for low-latency services by enabling the execution of computing tasks at the network-periphery, in proximity to end-users. While a number of recent studies have addressed the problem of determining the execution of service tasks and the routing of user requests to corresponding edge servers, the focus has primarily been on the efficient utilization of computing resources, neglecting the fact that non-trivial amounts of data need to be stored to enable service execution, and that many emerging services exhibit asymmetric bandwidth requirements. To fill this gap, we study the joint optimization of service placement and request routing in MEC-enabled multi-cell networks with multidimensional (storage-computation-communication) constraints. We show that this problem generalizes several problems in literature and propose an algorithm that achieves close-to-optimal performance using randomized rounding. Evaluation results demonstrate that our approach can effectively utilize the available resources to maximize the number of requests served by low-latency edge cloud servers. 

   more » « less