More Like this

1. Database Benchmarking for Supporting Real-Time Interactive Querying of Large Data

   https://doi.org/10.1145/3318464.3389732  

   Battle, Leilani; Eichmann, Philipp; Angelini, Marco; Catarci, Tiziana; Santucci, Giuseppe; Zheng, Yukun; Binnig, Carsten; Fekete, Jean-Daniel; Moritz, Dominik (June 2021, Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data)

   null (Ed.)

   In this paper, we present a new benchmark to validate the suitability of database systems for interactive visualization workloads. While there exist proposals for evaluating database systems on interactive data exploration workloads, none rely on real user traces for database benchmarking. To this end, our long term goal is to collect user traces that represent workloads with different exploration characteristics. In this paper, we present an initial benchmark that focuses on "crossfilter"-style applications, which are a popular interaction type for data exploration and a particularly demanding scenario for testing database system performance. We make our benchmark materials, including input datasets, interaction sequences, corresponding SQL queries, and analysis code, freely available as a community resource, to foster further research in this area: https://osf.io/9xerb/?view_only=81de1a3f99d04529b6b173a3bd5b4d23. 

   more » « less
2. CENSUS: Counting Interleaved Workloads on Shared Storage

   Chen, Si; Liu, Jianqiao; Wildani, Avani (October 2020, 36th International Conference on Massive Storage Systems and Technology (MSST 2020))

   Understanding the different workloaddependent factors that impact the latency or reliability of a storage system is essential for SLA satisfaction and fair resource provisioning. However, due to the volatility of system behavior under multiple workloads, determining even the number of concurrent types of workload functions, a necessary precursor to workload separation, is an unsolved problem in the general case. We introduce CENSUS, a novel classification framework that combines time-series analysis with gradient boosting to identify the number of functional workloads in a shared storage system by projecting workload traces into a high-dimensional feature representation space. We show that CENSUS can distinguish the number of interleaved workloads in a real-world trace segment with up to 95% accuracy, leading to a decrement of the mean square error to as little as 5% compared to the 

   more » « less
3. Beating OPT with Statistical Clairvoyance and Variable Size Caching

   https://doi.org/10.1145/3297858.3304067  

   Li, Pengcheng; Pronovost, Colin; Wilson, William; Tait, Benjamin; Zhou, Jie; Ding, Chen; Criswell, John (January 2019, Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2019)

   Caching techniques are widely used in today’s computing infrastructure from virtual memory management to server cache and memory cache. This paper builds on two observa- tions. First, the space utilization in cache can be improved by varying the cache size based on dynamic application demand. Second, it is easier to predict application behavior statistically than precisely. This paper presents a new variable-size cache that uses statistical knowledge of program behavior to maximize the cache performance. We measure performance using data access traces from real-world workloads, including Memcached traces from Facebook and storage traces from Microsoft Research. In an offline setting, the new cache is demonstrated to outperform even OPT, the optimal fixed- size cache which makes use of precise knowledge of program behavior. 

   more » « less
4. Optimal Column Layout for Hybrid Workloads

   https://doi.org/10.14778/3358701.3358707  

   Athanassoulis, Manos; Bøgh, Kenneth; Idreos, Stratos (September 2019, Proceedings of the VLDB Endowment)

   Data-intensive analytical applications need to support both efficient reads and writes. However, what is usually a good data layout for an update-heavy workload, is not well-suited for a read-mostly one and vice versa. Modern analytical data systems rely on columnar layouts and employ delta stores to inject new data and updates. We show that for hybrid workloads we can achieve close to one order of magnitude better performance by tailoring the column layout design to the data and query workload. Our approach navigates the possible design space of the physical layout: it organizes each column’s data by determining the number of partitions, their corresponding sizes and ranges, and the amount of buffer space and how it is allocated. We frame these design decisions as an optimization problem that, given workload knowledge and performance requirements, provides an optimal physical layout for the workload at hand. To evaluate this work, we build an in-memory storage engine, Casper, and we show that it outperforms state-of-the-art data layouts of analytical systems for hybrid workloads. Casper delivers up to 2.32x higher throughput for update-intensive workloads and up to 2.14x higher throughput for hybrid workloads. We further show how to make data layout decisions robust to workload variation by carefully selecting the input of the optimization. 

   more » « less
5. Is Big Data Performance Reproducible in Modern Cloud Networks?

   Uta, Alexandru; Custura, Alexandru; Duplyakin, Dmitry; Jimenez, Ivo; Rellermeyer, Jan; Maltzahn, Carlos; Ricci, Robert; Iosup, Alexandru (February 2029, Proceedings of the Seventeenth USENIX Symposium on Networked Systems Design and Implementation (NSDI))

   Performance variability has been acknowledged as a problem for over a decade by cloud practitioners and performance engineers. Yet, our survey of top systems conferences reveals that the research community regularly disregards variability when running experiments in the cloud. Focusing on networks, we assess the impact of variability on cloud-based big-data workloads by gathering traces from mainstream commercial clouds and private research clouds. Our data collection consists of millions of datapoints gathered while transferring over 9 petabytes of data. We characterize the network variability present in our data and show that, even though commercial cloud providers implement mechanisms for quality-of-service enforcement, variability still occurs, and is even exacerbated by such mechanisms and service provider policies. We show how big-data workloads suffer from significant slowdowns and lack predictability and replicability, even when state-of-the-art experimentation techniques are used. We provide guidelines for practitioners to reduce the volatility of big data performance, making experiments more repeatable. 

   more » « less