More Like this

1. PolyFrame: a retargetable query-based approach to scaling dataframes

   https://doi.org/10.14778/3476249.3476281  

   Sinthong, Phanwadee; Carey, Michael J. (July 2021, Proceedings of the VLDB Endowment)

   null (Ed.)

   In the last few years, the field of data science has been growing rapidly as various businesses have adopted statistical and machine learning techniques to empower their decision-making and applications. Scaling data analyses to large volumes of data requires the utilization of distributed frameworks. This can lead to serious technical challenges for data analysts and reduce their productivity. AFrame, a data analytics library, is implemented as a layer on top of Apache AsterixDB, addressing these issues by providing the data scientists' familiar interface, Pandas Dataframe, and transparently scaling out the evaluation of analytical operations through a Big Data management system. While AFrame is able to leverage data management facilities (e.g., indexes and query optimization) and allows users to interact with a large volume of data, the initial version only generated SQL++ queries and only operated against AsterixDB. In this work, we describe a new design that retargets AFrame's incremental query formation to other query-based database systems, making it more flexible for deployment against other data management systems with composable query languages. 

   more » « less
2. Energon: A Data Acquisition System for Portable Building Analytics

   https://doi.org/10.1145/3447555.3464850  

   He, Fang; Deng, Yang; Xu, Yanhui; Xu, Cheng; Hong, Dezhi; Wang, Dan (June 2021, e-Energy)

   Emerging building analytics rely on data-driven machine learning algorithms. However, writing these analytics is still challenging— developers need to know not only what data are required by the analytics but also how to reach the data in each individual building, despite the existing solutions to standardizing data and resource management in buildings. To bridge the gap between analytics development and the specific details of reaching actual data in each building, we present Energon, an open-source system that enables portable building analytics. The core of Energon is a new data organization for building as well as tools that can effectively manage building data and support building analytics development. More specifically, we propose a new "logic partition" of data resources in buildings, and this abstraction universally applies to all buildings. We develop a declarative query language accordingly to f ind data resources in this new logic view with high-level queries, thus substantially reducing development efforts. We also develop a query engine with automatic data extraction by traversing building ontology that widely exists in buildings. In this way, Energon enables mapping of analytics requirements to building resources in a building-agnostic manner. Using four types of real-world building analytics, we demonstrate the use of Energon and its effectiveness in reducing development efforts. 

   more » « less
3. DB Spinner: making a case for iterative processing in databases

   Sofoklis Floratos, Ahmed Ghazal (April 2021, 2021 37th IEEE International Conference on Data Engineering)

   null (Ed.)

   Relational database management systems (RDBMS) have limited iterative processing support. Recursive queries were added to ANSI SQL, however, their semantics do not allow aggregation functions, which disqualifies their use for several applications, such as PageRank and shortest path computations. Recently, another SQL extension, iterative Common Table Expressions (CTEs), is proposed to enable users to perform general iterative computations on RDBMSs.In this work 1 , we demonstrate how iterative CTEs can be efficiently incorporated into a production RDBMS without major intrusion to the system. We have prototyped our approach on Futurewei's MPPDB, a shared nothing relational parallel database engine. The implementation is based on a functional rewrite that translates iterative CTEs to other existing SQL operators. Thus, query plans of iterative CTEs can be optimized and executed by the engine with minimal modification to the code base. We have also applied several optimizations specifically for iterative CTEs to i) minimize data movement, ii) reuse results that remain constant and iii) push down predicates to avoid unnecessary data processing. We verified our implementation through extensive experimental evaluation using real world datasets and queries. The results show the feasibility of the rewrite approach and the effectiveness of the optimizations, which improve performance by an order of magnitude in some cases. 

   more » « less
4. BEAD: Batched Evaluation of Iterative Graph Queries with Evolving Analytics Demands

   https://doi.org/10.1109/BigData50022.2020.9378211  

   Mazloumi, Abbas; Xu, Chengshuo; Zhao, Zhijia; Gupta, Rajiv (December 2020, IEEE International Conference on Big Data (Big Data))

   null (Ed.)

   Simultaneous evaluating a batch of iterative graph queries on a distributed system enables amortization of high communication and computation costs across multiple queries. As demonstrated by our prior work on MultiLyra [BigData'19], batched graph query processing can deliver significant speedups and scale up to batch sizes of hundreds of queries.In this paper, we greatly expand the applicable scenarios for batching by developing BEAD, a system that supports Batching in the presence of Evolving Analytics Demands. First, BEAD allows the graph data set to evolve (grow) over time, more vertices (e.g., users) and edges (e.g., interactions) are added. In addition, as the graph data set evolves, BEAD also allows the user to add more queries of interests to the query batch to accommodate new user demands. The key to the superior efficiency offered by BEAD lies in a series of incremental evaluation techniques that leverage the results of prior request to "fast-foward" the evaluation of the current request.We performed experiments comparing batching in BEAD with batching in MultiLyra for multiple input graphs and algorithms. Experiments demonstrate that BEAD's batched evaluation of 256 queries, following graph changes that add up to 100K edges to a billion edge Twitter graph and also query changes of up to 32 new queries, outperforms MultiLyra's batched evaluation by factors of up to 26.16 × and 5.66 × respectively. 

   more » « less
5. To Ship or Not to (Function) Ship

   Liu, Feilong; Kamat, Niranjan; Blanas, Spyros; Nandi, Arnab (September 2018, 2018 IEEE High Performance Extreme Computing Conference (HPEC '18))

   Sampling is often used to reduce query latency for interactive big data analytics. The established parallel data processing paradigm relies on function shipping, where a coordinator dispatches queries to worker nodes and then collects the results. The commoditization of high-performance networking makes data shipping possible, where the coordinator directly reads data in the workers’ memory using RDMA while workers process other queries. In this work, we explore when to use function shipping or data shipping for interactive query processing with sampling. Whether function shipping or data shipping should be preferred depends on the amount of data transferred, the current CPU utilization and the sampling method. The results show that data shipping is up to 6.5× faster when performing clustered sampling with heavily-utilized workers. 

   more » « less