Search for: All records

The rapid advancement of large language models (LLMs) has caused their parameter sizes to grow beyond the memory capacity of a single GPU. Although distributed inference across multiple GPUs is a solution in enterprise settings, it remains inaccessible for most non-commercial users. Thus, there is a growing demand to run LLMs on a single GPU when the model does not fit entirely in GPU memory. A common approach is to offload parts of the model from the GPU to the CPU during inference. However, repeatedly transfer- ring parameters between these devices incurs significant overhead. To address this challenge, we propose a new buffer management policy, LIRS-M, which maximizes buffer hits and minimizes data transfer. Experimental results show that our approach achieves a 2.0× speedup compared to SoTA offloading techniques while delivering robust buffer-hit performance. 

Query optimization is critical in relational database management systems (DBMSs) for ensuring efficient query processing. The query optimizer relies on precise selectivity and cost estimates to generate optimal query plans for execution. However, this static query optimization approach falls short for DBMSs handling machine learning (ML) queries. ML-centric DBMSs face distinct challenges in query optimization. First, performance bottlenecks shift to user-defined functions (UDFs), often encapsulating deep learning models, making it difficult to estimate UDF statistics without profiling the query. Second, optimal query plans for ML queries are data-dependent, requiring dynamic plan adjustments during execution. To address these challenges, we introduce Aero, an ML-centric DBMS that utilizes adaptive query processing (AQP) for efficiently processing ML queries. Aero optimizes the evaluation of UDF-based query predicates by dynamically adjusting predicate evaluation order and enhancing UDF execution scalability. By integrating AQP, Aero continuously monitors UDF statistics, routes data to predicates in an optimal order, and dynamically allocates resources for evaluating predicates. Aero achieves up to 6.4x speedup compared to a state-of-the-art ML-centric DBMS across four diverse use cases, with no impact on accuracy.