More Like this

1. Dual Accuracy-Quality-Driven Neural Network for Prediction Interval Generation

   https://doi.org/10.1109/TNNLS.2023.3339470  

   Morales, Giorgio; Sheppard, John W. (January 2023, IEEE Transactions on Neural Networks and Learning Systems)

   Accurate uncertainty quantification is necessary to enhance the reliability of deep learning (DL) models in realworld applications. In the case of regression tasks, prediction intervals (PIs) should be provided along with the deterministic predictions of DL models. Such PIs are useful or “high-quality (HQ)” as long as they are sufficiently narrow and capture most of the probability density. In this article, we present a method to learn PIs for regression-based neural networks (NNs) automatically in addition to the conventional target predictions. In particular, we train two companion NNs: one that uses one output, the target estimate, and another that uses two outputs, the upper and lower bounds of the corresponding PI. Our main contribution is the design of a novel loss function for the PI-generation network that takes into account the output of the target-estimation network and has two optimization objectives: minimizing the mean PI width and ensuring the PI integrity using constraints that maximize the PI probability coverage implicitly. Furthermore, we introduce a self-adaptive coefficient that balances both objectives within the loss function, which alleviates the task of fine-tuning. Experiments using a synthetic dataset, eight benchmark datasets, and a real-world crop yield prediction dataset showed that our method was able to maintain a nominal probability coverage and produce significantly narrower PIs without detriment to its target estimation accuracy when compared to those PIs generated by three state-of-the-art neuralnetwork-based methods. In other words, our method was shown to produce higher quality PIs. 

   more » « less
2. WebGazer: Scalable Webcam Eye Tracking Using User Interactions

   Papoutsaki, Alexandra; Sangkloy, Patsorn; Laskey, James; Daskalova, Nediyana; Huang, Jeff; Hays, James (January 2016, Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence - IJCAI 2016)

   We introduce WebGazer, an online eye tracker that uses common webcams already present in laptops and mobile devices to infer the eye-gaze locations of web visitors on a page in real time. The eye tracking model self-calibrates by watching web visitors interact with the web page and trains a mapping between features of the eye and positions on the screen. This approach aims to provide a natural experience to everyday users that is not restricted to laboratories and highly controlled user studies. WebGazer has two key components: a pupil detector that can be combined with any eye detection library, and a gaze estimator using regression analysis informed by user interactions. We perform a large remote online study and a small in-person study to evaluate WebGazer. The findings show that WebGazer can learn from user interactions and that its accuracy is sufficient for approximating the user's gaze. As part of this paper, we release the first eye tracking library that can be easily integrated in any website for real-time gaze interactions, usability studies, or web research. 

   more » « less
3. Simple and Effective Approaches for Uncertainty Prediction in Facial Action Unit Intensity Regression

   https://doi.org/10.1109/FG47880.2020.00045  

   Wörtwein, Torsten; Morency, Louis-Philippe (January 2020, Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition)

   Knowing how much to trust a prediction is important for many critical applications. We describe two simple approaches to estimate uncertainty in regression prediction tasks and compare their performance and complexity against popular approaches. We operationalize uncertainty in regression as the absolute error between a model's prediction and the ground truth. Our two proposed approaches use a secondary model to predict the uncertainty of a primary predictive model. Our first approach leverages the assumption that similar observations are likely to have similar uncertainty and predicts uncertainty with a non-parametric method. Our second approach trains a secondary model to directly predict the uncertainty of the primary predictive model. Both approaches outperform other established uncertainty estimation approaches on the MNIST, DISFA, and BP4D+ datasets. Furthermore, we observe that approaches that directly predict the uncertainty generally perform better than approaches that indirectly estimate uncertainty. 

   more » « less
4. Leveraging synthetic aperture radar (SAR) with the National Water Model (NWM) to improve above-normal flow prediction in ungauged basins

   https://doi.org/10.1088/1748-9326/ad8808  

   Fang, Shiqi; Johnson, J Michael; Sankarasubramanian, A (October 2024, Environmental Research Letters)

   Abstract Effective flood prediction supports developing proactive risk management strategies, but its application in ungauged basins faces tremendous challenges due to limited/no streamflow record. This study investigates the potential for integrating streamflow derived from synthetic aperture radar (SAR) data and U.S. National Water Model (NWM) reanalysis estimates to develop improved predictions of above-normal flow (ANF) over the coterminous US. Leveraging the SAR data from the Global Flood Detection System to estimate the antecedent conditions using principal component regression, we apply the spatial-temporal hierarchical model (STHM) using NWM outputs for improving ANF prediction. Our evaluation shows promising results with the integrated model, STHM-SAR, significantly improving NWE, especially in 60% of the sites in the coastal region. Spatial and temporal validations underscore the model’s robustness, with SAR data contributing to explained variance by 24% on average. This approach not only improves NWM prediction, but also uniquely combines existing remote sensing data with national-scale predictions, showcasing its potential to improve hydrological modeling, particularly in regions with limited stream gauges. 

   more » « less
5. Transfer Learning with Uncertainty Quantification: Random Effect Calibration of Source to Target (RECaST)

   Hickey, Jimmy; Williams, Jonathan P; Hector, Emily C (November 2024, Journal of machine learning research)

   Transfer learning uses a data model, trained to make predictions or inferences on data from one population, to make reliable predictions or inferences on data from another population. Most existing transfer learning approaches are based on fine-tuning pre-trained neural network models, and fail to provide crucial uncertainty quantification. We develop a statistical framework for model predictions based on transfer learning, called RECaST. The primary mechanism is a Cauchy random effect that recalibrates a source model to a target population; we mathematically and empirically demonstrate the validity of our RECaST approach for transfer learning between linear models, in the sense that prediction sets will achieve their nominal stated coverage, and we numerically illustrate the method's robustness to asymptotic approximations for nonlinear models. Whereas many existing techniques are built on particular source models, RECaST is agnostic to the choice of source model, and does not require access to source data. For example, our RECaST transfer learning approach can be applied to a continuous or discrete data model with linear or logistic regression, deep neural network architectures, etc. Furthermore, RECaST provides uncertainty quantification for predictions, which is mostly absent in the literature. We examine our method's performance in a simulation study and in an application to real hospital data. 

   more » « less