An official website of the United States government
Abstract Timely and accurate prediction of solar flares is a crucial task due to the danger they pose to human life and infrastructure beyond Earth’s atmosphere. Although various machine learning algorithms have been employed to improve solar flare prediction, there has been limited focus on improving performance using outlier detection. In this study, we propose the use of a tree-based outlier detection algorithm, Isolation Forest (iForest), to identify multivariate time-series instances within the flare-forecasting benchmark data set, Space Weather Analytics for Solar Flares (SWAN-SF). By removing anomalous samples from the nonflaring class (N-class) data, we observe a significant improvement in both the true skill score and the updated Heidke skill score in two separate experiments. We focus on analyzing outliers detected by iForest at a 2.4% contamination rate, considered the most effective overall. Our analysis reveals a co-occurrence between the outliers we discovered and strong flares. Additionally, we investigated the similarity between the outliers and the strong-flare data and quantified it using Kullback–Leibler divergence. This analysis demonstrates a higher similarity between our outliers and strong-flare data when compared to the similarity between the outliers and the rest of the N-class data, supporting our rationale for using outlier detection to enhance SWAN-SF data for flare prediction. Furthermore, we explore a novel approach by treating our outliers as if they belong to flaring-class data in the training phase of our machine learning, resulting in further enhancements to our models’ performance.
more »
« less
How to Train Neural Networks for Flare Removal
When a camera is pointed at a strong light source, the resulting photograph may contain lens flare artifacts. Flares appear in a wide variety of patterns (halos, streaks, color bleeding, haze, etc.) and this diversity in appearance makes flare removal challenging. Existing analytical solutions make strong assumptions about the artifact’s geometry or brightness, and therefore only work well on a small subset of flares. Machine learning techniques have shown success in removing other types of artifacts, like reflections, but have not been widely applied to flare removal due to the lack of training data. To solve this problem, we explicitly model the optical causes of flare either empirically or using wave optics, and generate semi-synthetic pairs of flare-corrupted and clean images. This enables us to train neural networks to remove lens flare for the first time. Experiments show our data synthesis approach is critical for accurate flare removal, and that models trained with our technique generalize well to real lens flares across different scenes, lighting conditions, and cameras.
more »
« less
- Award ID(s):
- 1652633
- PAR ID:
- 10319131
- Date Published:
- Journal Name:
- IEEE/CVF International Conference on Computer Vision
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Supervised Machine Learning (ML) models for solar flare prediction rely on accurate labels for a given input data set, commonly obtained from the GOES/XRS X-ray flare catalog. With increasing interest in utilizing ultraviolet (UV) and extreme ultraviolet (EUV) image data as input to these models, we seek to understand if flaring activity can be defined and quantified using EUV data alone. This would allow us to move away from the GOES single pixel measurement definition of flares and use the same data we use for flare prediction for label creation. In this work, we present a Solar Dynamics Observatory (SDO) Atmospheric Imaging Assembly (AIA)-based flare catalog covering flare of GOES X-ray magnitudes C, M and X from 2010 to 2017. We use active region (AR) cutouts of full disk AIA images to match the corresponding SDO/Helioseismic and Magnetic Imager (HMI) SHARPS (Space weather HMI Active Region Patches) that have been extensively used in ML flare prediction studies, thus allowing for labeling of AR number as well as flare magnitude and timing. Flare start, peak, and end times are defined using a peak-finding algorithm on AIA time series data obtained by summing the intensity across the AIA cutouts. An extremely randomized trees (ERT) regression model is used to map SDO/AIA flare magnitudes to GOES X-ray magnitude, achieving a low-variance regression. We find an accurate overlap on 85% of M/X flares between our resulting AIA catalog and the GOES flare catalog. However, we also discover a number of large flares unrecorded or mislabeled in the GOES catalog.more » « less
-
Helioseismic response to solar flares ("sunquakes") occurs due to localized force or/and momentum impacts observed during the flare impulsive phase in the lower atmosphere. Such impacts may be caused by precipitation of high-energy particles, downward shocks, or magnetic Lorentz force. Understanding the mechanism of sunquakes is a key problem of the flare energy release and transport. Our statistical analysis of M-X class flares observed by the Solar Dynamics Observatory during Solar Cycle 24 has shown that contrary to expectations, many relatively weak M-class flares produced strong sunquakes, while for some powerful X-class flares, helioseismic waves were not observed or were weak. The analysis also revealed that there were active regions characterized by the most efficient generation of sunquakes during the solar cycle. We found that the sunquake power correlates with maximal values of the X-ray flux derivative better than with the X-ray class. The sunquake data challenge the current theories of solar flares.more » « less
-
Rutkowski, L.; Scherer, R.; Korytkowski, M.; Pedrycz W.; Tadeusiewicz R.; Zurada J. (Ed.)Solar flares not only pose risks to outer space technologies and astronauts’ well being, but also cause disruptions on earth to our high-tech, interconnected infrastructure our lives highly depend on. While a number of machine-learning methods have been proposed to improve flare prediction, none of them, to the best of our knowledge, have investigated the impact of outliers on the reliability and robustness of those models’ performance. In this study, we investigate the impact of outliers in a multivariate time series benchmark dataset, namely SWAN-SF, on flare prediction models, and test our hypothesis. That is, there exist outliers in SWAN-SF, removal of which enhances the performance of the prediction models on unseen datasets. We employ Isolation Forest to detect the outliers among the weaker flare instances. Several experiments are carried out using a large range of contamination rates which determine the percentage of present outliers. We assess the quality of each dataset in terms of its actual contamination using TimeSeriesSVC. In our best findings, we achieve a 279% increase in True Skill Statistic and 68% increase in Heidke Skill Score. The results show that overall a significant improvement can be achieved for flare prediction if outliers are detected and removed properly.more » « less
-
ABSTRACT Using blazar light curves from the optical All-Sky Automated Survey for Supernovae (ASAS-SN) and the γ-ray Fermi-LAT telescope, we performed the most extensive statistical correlation study between both bands, using a sample of 1180 blazars. This is almost an order of magnitude larger than other recent studies. Blazars represent more than 98 per cent of the AGNs detected by Fermi-LAT and are the brightest γ-ray sources in the extragalactic sky. They are essential for studying the physical properties of astrophysical jets from central black holes. However, their γ-ray flare mechanism is not fully understood. Multiwavelength correlations help constrain the dominant mechanisms of blazar variability. We search for temporal relationships between optical and γ-ray bands. Using a Bayesian Block Decomposition, we detect 1414 optical and 510 γ-ray flares, we find a strong correlation between both bands. Among all the flares, we find 321 correlated flares from 133 blazars, and derive an average rest-frame time delay of only 1.1$$_{-8.5}^{+7.1}$$ d, with no difference between the flat-spectrum radio quasars, BL Lacertae-like objects or low, intermediate, and high-synchrotron peaked blazar classes. Our time-delay limit rules out the hadronic proton-synchrotron model as the driver for non-orphan flares and suggests a leptonic single-zone model. Limiting our search to well-defined light curves and removing 976 potential but unclear ‘orphan’ flares, we find 191 (13 per cent) and 115 (22 per cent) clear ‘orphan’ optical and γ-ray flares. The presence of ‘orphan’ flares in both bands challenges the standard one-zone blazar flare leptonic model and suggests multizone synchrotron sites or a hadronic model for some blazars.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICCV48922.2021.00224
