An official website of the United States government
The remarkable success of the Transformer model in Natural Language Processing (NLP) is increasingly capturing the attention of vision researchers in contemporary times. The Vision Transformer (ViT) model effectively models long-range dependencies while utilizing a self-attention mechanism by converting image information into meaningful representations. Moreover, the parallelism property of ViT ensures better scalability and model generalization compared to Recurrent Neural Networks (RNN). However, developing robust ViT models for high-risk vision applications, such as self-driving cars, is critical. Deterministic ViT models are susceptible to noise and adversarial attacks and incapable of yielding a level of confidence in output predictions. Quantifying the confidence (or uncertainty) level in the decision is highly important in such real-world applications. In this work, we introduce a probabilistic framework for ViT to quantify the level of uncertainty in the model's decision. We approximate the posterior distribution of network parameters using variational inference. While progressing through non-linear layers, the first-order Taylor approximation was deployed. The developed framework propagates the mean and covariance of the posterior distribution through layers of the probabilistic ViT model and quantifies uncertainty at the output predictions. Quantifying uncertainty aids in providing warning signals to real-world applications in case of noisy situations. Experimental results from extensive simulation conducted on numerous benchmark datasets (e.g., MNIST and Fashion-MNIST) for image classification tasks exhibit 1) higher accuracy of proposed probabilistic ViT under noise or adversarial attacks compared to the deterministic ViT. 2) Self-evaluation through uncertainty becomes notably pronounced as noise levels escalate. Simulations were conducted at the Texas Advanced Computing Center (TACC) on the Lonestar6 supercomputer node. With the help of this vital resource, we completed all the experiments within a reasonable period.
more »
« less
Probabilistic Neural Network to Quantify Uncertainty of Wind Power Estimation
Each year a growing number of wind farms are being added to power grids to generate sustainable energy. The power curve of a wind turbine, which exhibits the relationship between generated power and wind speed, plays a major role in assessing the performance of a wind farm. Neural networks have been used for power curve estimation. However, they do not produce a confidence measure for their output, unless computationally prohibitive Bayesian methods are used. In this paper, a probabilistic neural network with Monte Carlo dropout is considered to quantify the model or epistemic uncertainty of the power curve estimation. This approach offers a minimal increase in computational complexity and thus evaluation time. Furthermore, by adding a probabilistic loss function, the noise or aleatoric uncertainty in the data is estimated. The developed network captures both model and noise uncertainty which are found to be useful tools in assessing performance. Also, the developed network is compared with the existing ones across a public domain dataset showing superior performance in terms of prediction accuracy. The results obtained indicate that the developed network provides the quantification of uncertainty while maintaining accurate power estimation.
more »
« less
- Award ID(s):
- 1839733
- PAR ID:
- 10394613
- Date Published:
- Journal Name:
- 2022 IEEE 15th Dallas Circuit And System Conference (DCAS)
- Page Range / eLocation ID:
- 1 to 7
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The increasing uncertainties caused by the high-penetration of stochastic renewable generation resources and flexible loads pose challenges to the power system voltage stability. To address this issue, this paper proposes a probabilistic transferable deep kernel emulator (DKE) to extract the hidden relationship between uncertain sources, i.e., wind generations and loads, and load margin for probabilistic load margin assessment (PLMA). This emulator extends the Gaussian process kernel to the deep neural network (DNN) structure and thus gains the advantages of DNN in dealing with high-dimension uncertain inputs and the uncertainty quantification capability of the Gaussian process. A transfer learning framework is also developed to reduce the invariant representation space distance between the old topology and new one. It allows the DKE to be quickly fine tuned with only a few samples under the new topology. Numerical results carried out on the modified IEEE 39-bus and 118-bus power systems demonstrate the strong robustness of the proposed transferable DKE to uncertain wind and load power as well as topology changes while maintaining high accuracy.more » « less
-
Barambones, Oscar (Ed.)Accurate quantification of uncertainty in solar photovoltaic (PV) generation forecasts is imperative for the efficient and reliable operation of the power grid. In this paper, a data-driven non-parametric probabilistic method based on the Naïve Bayes (NB) classification algorithm and Dempster–Shafer theory (DST) of evidence is proposed for day-ahead probabilistic PV power forecasting. This NB-DST method extends traditional deterministic solar PV forecasting methods by quantifying the uncertainty of their forecasts by estimating the cumulative distribution functions (CDFs) of their forecast errors and forecast variables. The statistical performance of this method is compared with the analog ensemble method and the persistence ensemble method under three different weather conditions using real-world data. The study results reveal that the proposed NB-DST method coupled with an artificial neural network model outperforms the other methods in that its estimated CDFs have lower spread, higher reliability, and sharper probabilistic forecasts with better accuracy.more » « less
-
With increasing needs of fast and reliable commu- nication between devices, wireless communication techniques are rapidly evolving to meet such needs. Multiple input and output (MIMO) systems are one of the key techniques that utilize multiple antennas for high-throughput and reliable communication. However, increasing the number of antennas in communication also adds to the complexity of channel esti- mation, which is essential to accurately decode the transmitted data. Therefore, development of accurate and efficient channel estimation methods is necessary. We report the performance of machine learning-based channel estimation approaches to enhance channel estimation performance in high-noise envi- ronments. More specifically, bit error rate (BER) performance of 2 × 2 and 4 × 4 MIMO communication systems with space- time block coding model (STBC) and two neural network-based channel estimation algorithms is analyzed. Most significantly, the results demonstrate that a generalized regression neural network (GRNN) model matches BER results of a known-channel communication for 4 × 4 MIMO with 8-bit pilots, when trained in a specific signal to noise ratio (SNR) regime. Moreover, up to 9dB improvement in signal-to-noise ratio (SNR) for a target BER is observed, compared to least square (LS) channel estimation, especially when the model is trained in the low SNR regime. A deep artificial neural network (Deep ANN) model shows worse BER performance compared to LS in all tested environments. These preliminary results present an opportunity for achieving better performance in channel estimation through GRNN and highlight further research topics for deployment in the wild.more » « less
-
Gaussian Mixture Models (GMM) are an effective representation of resource uncertainty in power systems planning, as they can be tractably incorporated within stochastic optimization models. However, the skewness, multimodality, and bounded physical support of long-term wind power forecasts can entail requiring a large number of mixture components to achieve a good fit, leading to complex optimization problems. We propose a probabilistic model for wind generation uncertainty to address this challenge, termed Discrete-Gaussian Mixture Model (DGMM), that combines continuous Gaussian components with discrete masses. The model generalizes classical GMMs that have been widely used to estimate wind power outputs. We employ a modified Expectation-Maximization algorithm (called FixedEM) to estimate the parameters of the DGMM. We provide empirical results on the ACTIVSg2000 synthetic wind generation dataset, where we demonstrate that the fitted DGMM is capable of capturing the high frequencies of time windows when wind generating units are either producing at maximum capacity or not producing any power at all. Furthermore, we find that the Bayesian Information Criterion of the DGMM is significantly lower compared to that of existing GMMs using the same number of Gaussian components. This improvement is particularly advantageous when the allowed number of Gaussian components is limited, facilitating the efficient solution to optimization problems for long-term planning.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/DCAS53974.2022.9845651
