Search for: All records

Query optimization is a key component in database management systems (DBMS) and distributed data processing platforms. Re- cent research in the database community incorporated techniques from artificial intelligence to enhance query optimization. Various learning models have been extended and applied to the query optimization tasks, including query execution plan, query rewriting, and cost estimation. The tasks involved in query optimization differ based on the type of data being processed, such as relational data or spatial geometries. This tutorial reviews recent learning-based approaches for spatial query optimization tasks. We go over methods designed specifically for spatial data, as well as solutions proposed for high-dimensional data. Additionally, we present learning-based spatial indexing and spatial partitioning methods, which are also vital components in spatial data processing. We also identify several open research problems in these fields. 

Progressive query processing enables data scientists to efficiently analyze and explore large datasets. Data scientists can start further analyses earlier if the progressive result can represent the complete results well. Most progressive processing frameworks carefully control which parts of the input to process in order to improve the quality of progressive results. The input control strategies work well when the data are processed uniformly. However, the progressive results will be biased towards the join keys if the processed data are not uniform. A recently proposed input&output framework named QPJ corrects the bias by temporarily hiding some results. The framework dynamically estimates the distribution of the complete result and outputs progressive results with a similar distribution to the estimated complete result. This demo presents QPJVis, which is a progressive query processing system designed to inherently process the progressive queries using the QPJ frame- work. Additionally, we also implement an input control framework, Prism, in QPJVis so that users can compare the difference between the input&output framework and a purely input framework. 

Quantum sensors based on solid-state defects, in particular nitrogen-vacancy (NV) centers in diamond, enable precise measurement of magnetic fields, temperature, rotation, and electric fields. Cavity quantum electrodynamic (cQED) readout, in which an NV ensemble is hybridized with a microwave mode, can overcome limitations in optical spin detection and has resulted in leading magnetic sensitivities at the pT-level. This approach, however, remains far from the intrinsic spin-projection noise limit due to thermal Johnson-Nyquist noise and spin saturation effects. Here we tackle these challenges by combining recently demonstrated spin refrigeration techniques with comprehensive nonlinear modeling of the cQED sensor operation. We demonstrate that the optically-polarized NV ensemble simultaneously provides magnetic sensitivity and acts as a heat sink for the deleterious thermal microwave noise background, even when actively probed by a microwave field. Optimizing the NV-cQED system, we demonstrate a broadband sensitivity of 576 ± 6 fT/$$\sqrt{{{{\rm{Hz}}}}}$$$\sqrt{\mathrm{Hz}}$around 15 kHz in ambient conditions. We then discuss the implications of this approach for the design of future magnetometers, including near-projection-limited devices approaching 3 fT/$$\sqrt{{{{\rm{Hz}}}}}$$$\sqrt{\mathrm{Hz}}$sensitivity enabled by spin refrigeration.

 

Progressive visual analytics enable data scientists to efficiently explore large datasets and examine progressive results with low latency. Most progressive visualization frameworks use a progressive query processing module that controls the quality of the results and then feeds these results into a visualization module. The goal is to avoid poor-quality progressive results which could mislead data scientists. This method misses some optimization opportunities as it improves the quality of the intermediate result while ignoring how this result affects the final visualization. This work presents a work-in-progress quality-aware progressive visualization input control component, named QPV. The key idea of the proposed framework is to integrate the visualization module into the progressive query results so that the quality control takes into account the final visualization. With limited computational resources, QPV solves an optimization problem to allocate resources and alleviate the misleading effects in the progressive plots. 

Real-time, all-electronic control of non-Newtonian fluid flow through a microscale channel is crucial for various applications in manufacturing and healthcare. However, existing methods lack the sensitivity required for accurate measurement and the real-time responsiveness necessary for effective adjustment. Here, we demonstrate an all-electronic system that enables closed-loop, real-time, high-sensitivity control of various waveforms of non-Newtonian fluid flow (0.76 μl min−1) through a micro-sized outlet. Our approach combines a contactless, cuff-like flow sensor with a neural-network control program. This system offers a simple, miniaturized, versatile, yet high-performance solution for non-Newtonian fluid flow control, easily integrated into existing setups.

 

We address the challenge of estimating regression coefficients and selecting relevant predictors in the context of mixed linear regression in high dimensions, where the number of predictors greatly exceeds the sample size. Recent advancements in this field have centered on incorporating sparsity-inducing penalties into the expectation-maximization (EM) algorithm, which seeks to maximize the conditional likelihood of the response given the predictors. However, existing procedures often treat predictors as fixed or overlook their inherent variability. In this paper, we leverage the independence between the predictor and the latent indicator variable of mixtures to facilitate efficient computation and also achieve synergistic variable selection across all mixture components. We establish the non-asymptotic convergence rate of the proposed fast group-penalized EM estimator to the true regression parameters. The effectiveness of our method is demonstrated through extensive simulations and an application to the Cancer Cell Line Encyclopedia dataset for the prediction of anticancer drug sensitivity.

 

Progressive query processing enables data scientists to efficiently analyze and explore large datasets. Data scientists can start further analyses earlier if the progressive result can represent the complete results well. Most progressive processing frameworks carefully control which parts of the input to process in order to improve the quality of progressive results. The input control strategies work well when the data are processed uniformly. However, the progressive results will be biased towards the join keys if the processed data are not uniform. A recently proposed input&output framework named QPJ corrects the bias by temporarily hiding some results. The framework dynamically estimates the distribution of the complete result and outputs progressive results with a similar distribution to the estimated complete result. This demo presents QPJVis, which is a progressive query processing system designed to inherently process the progressive queries using the QPJ framework. Additionally, we also implement an input control framework, Prism, in QPJVis so that users can compare the difference between the input&output framework and a purely input framework.

 

Query optimization is a key component in database management systems (DBMS) and distributed data processing platforms. Recent research in the database community incorporated techniques from artificial intelligence to enhance query optimization. Various learning models have been extended and applied to the query optimization tasks, including query execution plan, query rewriting, and cost estimation. The tasks involved in query optimization differ based on the type of data being processed, such as relational data or spatial geometries. This tutorial reviews recent learning-based approaches for spatial query optimization tasks. We go over methods designed specifically for spatial data, as well as solutions proposed for high-dimensional data. Additionally, we present learning-based spatial indexing and spatial partitioning methods, which are also vital components in spatial data processing. We also identify several open research problems in these fields.