More Like this

1. Enhancing Observability of Serverless Computing with the Serverless Application Analytics Framework

   https://doi.org/10.1145/3447545.3451173  

   Cordingly, Robert ; Heydari, Navid ; Yu, Hanfei ; Hoang, Varik ; Sadeghi, Zohreh ; Lloyd, Wes ( April 2021 , 12th ACM/SPEC International Conference on Performance Engineering)

   To improve the observability of workload performance, resource utilization, and infrastructure underlying serverless Function-as-a-Service (FaaS) platforms, we have developed the Serverless Application Analytics Framework (SAAF). SAAF provides a reusable framework supporting multiple programming languages that developers can leverage to inspect performance, resource utilization, scalability, and infrastructure metrics of function deployments to commercial and open-source FaaS platforms. To automate reproducible FaaS performance experiments, we provide the FaaS Runner as a multithreaded FaaS client. FaaS Runner provides a programmable client that can orchestrate over one thousand concurrent FaaS function calls. The ReportGenerator is then used to aggregate experiment output into CSV files for consumption by popular data analytics tools. SAAF and its supporting tools combined can assess forty-eight distinct metrics to enhance observability of serverless software deployments. In this tutorial paper, we describe SAAF and its supporting tools and provide examples of observability insights that can be derived. 

   more » « less
2. Memory deduplication for serverless computing with Medes

   https://doi.org/10.1145/3492321.3524272  

   Saxena, Divyanshu ; Ji, Tao ; Singhvi, Arjun ; Khalid, Junaid ; Akella, Aditya ( January 2022 , EuroSys)
3. Predicting Performance and Cost of Serverless Computing Functions with SAAF

   https://doi.org/10.1109/DASC-PICom-CBDCom-CyberSciTech49142.2020.00111  

   Cordingly, Robert ; Shu, Wen ; Lloyd, Wes J. ( August 2020 , 2020 IEEE International Conference on Cloud and Big Data Computing)
4. Optimizing inference serving on serverless platforms

   https://doi.org/10.14778/3547305.3547313  

   Ali, Ahsan ; Pinciroli, Riccardo ; Yan, Feng ; Smirni, Evgenia ( June 2022 , Proceedings of the VLDB Endowment)

   Serverless computing is gaining popularity for machine learning (ML) serving workload due to its autonomous resource scaling, easy to use and pay-per-use cost model. Existing serverless platforms work well for image-based ML inference, where requests are homogeneous in service demands. That said, recent advances in natural language processing could not fully benefit from existing serverless platforms as their requests are intrinsically heterogeneous. Batching requests for processing can significantly increase ML serving efficiency while reducing monetary cost, thanks to the pay-per-use pricing model adopted by serverless platforms. Yet, batching heterogeneous ML requests leads to additional computation overhead as small requests need to be "padded" to the same size as large requests within the same batch. Reaching effective batching decisions (i.e., which requests should be batched together and why) is non-trivial: the padding overhead coupled with the serverless auto-scaling forms a complex optimization problem. To address this, we develop Multi-Buffer Serving (MBS), a framework that optimizes the batching of heterogeneous ML inference serving requests to minimize their monetary cost while meeting their service level objectives (SLOs). The core of MBS is a performance and cost estimator driven by analytical models supercharged by a Bayesian optimizer. MBS is prototyped and evaluated on AWS using bursty workloads. Experimental results show that MBS preserves SLOs while outperforming the state-of-the-art by up to 8 x in terms of cost savings while minimizing the padding overhead by up to 37 x with 3 x less number of serverless function invocations. 

   more » « less
5. Particle: ephemeral endpoints for serverless networking

   https://doi.org/10.1145/3419111.3421275  

   Thomas, Shelby ; Ao, Lixiang ; Voelker, Geoffrey M. ; Porter, George ( October 2020 , 11th ACM Symposium on Cloud Computing (SoCC))

   Burst-parallel serverless applications invoke thousands of short-lived distributed functions to complete complex jobs such as data analytics, video encoding, or compilation. While these tasks execute in seconds, starting and configuring the virtual network they rely on is a major bottleneck that can consume up to 84% of total startup time. In this paper we characterize the magnitude of this network cold start problem in three popular overlay networks, Docker Swarm, Weave, and Linux Overlay. We focus on end-to-end startup time that encompasses both the time to boot a group of containers as well as interconnecting them. Our primary observation is that existing overlay approaches for serverless networking scale poorly in short-lived serverless environments. Based on our findings we develop Particle, a network stack tailored for multi-node serverless overlay networks that optimizes network creation without sacrificing multi-tenancy, generality, or throughput. When integrated into a serverless burst-parallel video processing pipeline, Particle improves application runtime by 2.4--3X over existing overlays. 

   more » « less