skip to main content

Explore Research Products in the NSF-PAR

Title: How to Train Neural Networks for Flare Removal
When a camera is pointed at a strong light source, the resulting photograph may contain lens flare artifacts. Flares appear in a wide variety of patterns (halos, streaks, color bleeding, haze, etc.) and this diversity in appearance makes flare removal challenging. Existing analytical solutions make strong assumptions about the artifact’s geometry or brightness, and therefore only work well on a small subset of flares. Machine learning techniques have shown success in removing other types of artifacts, like reflections, but have not been widely applied to flare removal due to the lack of training data. To solve this problem, we explicitly model the optical causes of flare either empirically or using wave optics, and generate semi-synthetic pairs of flare-corrupted and clean images. This enables us to train neural networks to remove lens flare for the first time. Experiments show our data synthesis approach is critical for accurate flare removal, and that models trained with our technique generalize well to real lens flares across different scenes, lighting conditions, and cameras.  more » « less
Award ID(s):
1652633
NSF-PAR ID:
10319131
Author(s) / Creator(s):
; ; ; ; ; ;
Date Published:
Journal Name:
IEEE/CVF International Conference on Computer Vision
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ( , Lecture notes in computer science)
    Rutkowski, L. ; Scherer, R. ; Korytkowski, M. ; Pedrycz W. ; Tadeusiewicz R. ; Zurada J. (Ed.)
    Solar flares not only pose risks to outer space technologies and astronauts’ well being, but also cause disruptions on earth to our high-tech, interconnected infrastructure our lives highly depend on. While a number of machine-learning methods have been proposed to improve flare prediction, none of them, to the best of our knowledge, have investigated the impact of outliers on the reliability and robustness of those models’ performance. In this study, we investigate the impact of outliers in a multivariate time series benchmark dataset, namely SWAN-SF, on flare prediction models, and test our hypothesis. That is, there exist outliers in SWAN-SF, removal of which enhances the performance of the prediction models on unseen datasets. We employ Isolation Forest to detect the outliers among the weaker flare instances. Several experiments are carried out using a large range of contamination rates which determine the percentage of present outliers. We assess the quality of each dataset in terms of its actual contamination using TimeSeriesSVC. In our best findings, we achieve a 279% increase in True Skill Statistic and 68% increase in Heidke Skill Score. The results show that overall a significant improvement can be achieved for flare prediction if outliers are detected and removed properly. 
    more » « less
  2. ; ; ; ( , Frontiers in Astronomy and Space Sciences)

    Supervised Machine Learning (ML) models for solar flare prediction rely on accurate labels for a given input data set, commonly obtained from the GOES/XRS X-ray flare catalog. With increasing interest in utilizing ultraviolet (UV) and extreme ultraviolet (EUV) image data as input to these models, we seek to understand if flaring activity can be defined and quantified using EUV data alone. This would allow us to move away from the GOES single pixel measurement definition of flares and use the same data we use for flare prediction for label creation. In this work, we present a Solar Dynamics Observatory (SDO) Atmospheric Imaging Assembly (AIA)-based flare catalog covering flare of GOES X-ray magnitudes C, M and X from 2010 to 2017. We use active region (AR) cutouts of full disk AIA images to match the corresponding SDO/Helioseismic and Magnetic Imager (HMI) SHARPS (Space weather HMI Active Region Patches) that have been extensively used in ML flare prediction studies, thus allowing for labeling of AR number as well as flare magnitude and timing. Flare start, peak, and end times are defined using a peak-finding algorithm on AIA time series data obtained by summing the intensity across the AIA cutouts. An extremely randomized trees (ERT) regression model is used to map SDO/AIA flare magnitudes to GOES X-ray magnitude, achieving a low-variance regression. We find an accurate overlap on 85% of M/X flares between our resulting AIA catalog and the GOES flare catalog. However, we also discover a number of large flares unrecorded or mislabeled in the GOES catalog.

     
    more » « less
  3. ; ; ; ; ; ; ; ; ; ( , Monthly Notices of the Royal Astronomical Society)
    ABSTRACT Using blazar light curves from the optical All-Sky Automated Survey for Supernovae (ASAS-SN) and the γ-ray Fermi-LAT telescope, we performed the most extensive statistical correlation study between both bands, using a sample of 1180 blazars. This is almost an order of magnitude larger than other recent studies. Blazars represent more than 98 per cent of the AGNs detected by Fermi-LAT and are the brightest γ-ray sources in the extragalactic sky. They are essential for studying the physical properties of astrophysical jets from central black holes. However, their γ-ray flare mechanism is not fully understood. Multiwavelength correlations help constrain the dominant mechanisms of blazar variability. We search for temporal relationships between optical and γ-ray bands. Using a Bayesian Block Decomposition, we detect 1414 optical and 510 γ-ray flares, we find a strong correlation between both bands. Among all the flares, we find 321 correlated flares from 133 blazars, and derive an average rest-frame time delay of only 1.1$_{-8.5}^{+7.1}$ d, with no difference between the flat-spectrum radio quasars, BL Lacertae-like objects or low, intermediate, and high-synchrotron peaked blazar classes. Our time-delay limit rules out the hadronic proton-synchrotron model as the driver for non-orphan flares and suggests a leptonic single-zone model. Limiting our search to well-defined light curves and removing 976 potential but unclear ‘orphan’ flares, we find 191 (13 per cent) and 115 (22 per cent) clear ‘orphan’ optical and γ-ray flares. The presence of ‘orphan’ flares in both bands challenges the standard one-zone blazar flare leptonic model and suggests multizone synchrotron sites or a hadronic model for some blazars. 
    more » « less
  4. ( , EGU General Assembly 2021)
    Helioseismic response to solar flares ("sunquakes") occurs due to localized force or/and momentum impacts observed during the flare impulsive phase in the lower atmosphere. Such impacts may be caused by precipitation of high-energy particles, downward shocks, or magnetic Lorentz force. Understanding the mechanism of sunquakes is a key problem of the flare energy release and transport. Our statistical analysis of M-X class flares observed by the Solar Dynamics Observatory during Solar Cycle 24 has shown that contrary to expectations, many relatively weak M-class flares produced strong sunquakes, while for some powerful X-class flares, helioseismic waves were not observed or were weak. The analysis also revealed that there were active regions characterized by the most efficient generation of sunquakes during the solar cycle. We found that the sunquake power correlates with maximal values of the X-ray flux derivative better than with the X-ray class. The sunquake data challenge the current theories of solar flares. 
    more » « less
  5. ; ; ; ; ; ( , The Astrophysical Journal Supplement Series)
    Abstract We present a case study of solar flare forecasting by means of metadata feature time series, by treating it as a prominent class-imbalance and temporally coherent problem. Taking full advantage of pre-flare time series in solar active regions is made possible via the Space Weather Analytics for Solar Flares (SWAN-SF) benchmark data set, a partitioned collection of multivariate time series of active region properties comprising 4075 regions and spanning over 9 yr of the Solar Dynamics Observatory period of operations. We showcase the general concept of temporal coherence triggered by the demand of continuity in time series forecasting and show that lack of proper understanding of this effect may spuriously enhance models’ performance. We further address another well-known challenge in rare-event prediction, namely, the class-imbalance issue. The SWAN-SF is an appropriate data set for this, with a 60:1 imbalance ratio for GOES M- and X-class flares and an 800:1 imbalance ratio for X-class flares against flare-quiet instances. We revisit the main remedies for these challenges and present several experiments to illustrate the exact impact that each of these remedies may have on performance. Moreover, we acknowledge that some basic data manipulation tasks such as data normalization and cross validation may also impact the performance; we discuss these problems as well. In this framework we also review the primary advantages and disadvantages of using true skill statistic and Heidke skill score, two widely used performance verification metrics for the flare-forecasting task. In conclusion, we show and advocate for the benefits of time series versus point-in-time forecasting, provided that the above challenges are measurably and quantitatively addressed. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email