More Like this

1. Comparing feature sets and machine-learning models for prediction of solar flares: Topology, physics, and model complexity

   V. Deshmukh; S. Baskar; T. E. Berger; E. Bradley; J. D. Meiss (April 2023, Astronomy and astrophysics)

   Context. Machine-learning methods for predicting solar flares typically employ physics-based features that have been carefully cho- sen by experts in order to capture the salient features of the photospheric magnetic fields of the Sun. Aims. Though the sophistication and complexity of these models have grown over time, there has been little evolution in the choice of feature sets, or any systematic study of whether the additional model complexity leads to higher predictive skill. Methods. This study compares the relative prediction performance of four different machine-learning based flare prediction models with increasing degrees of complexity. It evaluates three different feature sets as input to each model: a “traditional” physics-based feature set, a novel “shape-based” feature set derived from topological data analysis (TDA) of the solar magnetic field, and a com- bination of these two sets. A systematic hyperparameter tuning framework is employed in order to assure fair comparisons of the models across different feature sets. Finally, principal component analysis is used to study the effects of dimensionality reduction on these feature sets. Results. It is shown that simpler models with fewer free parameters perform better than the more complicated models on the canonical 24-h flare forecasting problem. In other words, more complex machine-learning architectures do not necessarily guarantee better prediction performance. In addition, it is found that shape-based feature sets contain just as much useful information as physics-based feature sets for the purpose of flare prediction, and that the dimension of these feature sets – particularly the shape-based one – can be greatly reduced without impacting predictive accuracy. 

   more » « less
2. Learning multi-scale local conditional probability models of images

   Kadkhodaie, Zahra; Guth, Florentin; Mallat, Stéphane; Simoncelli, Eero P (March 2023, ICLR 2023)

   Deep neural networks can learn powerful prior probability models for images, as evidenced by the high-quality generations obtained with recent score-based diffusion methods. But the means by which these networks capture complex global statistical structure, apparently without suffering from the curse of dimensionality, remain a mystery. To study this, we incorporate diffusion methods into a multi-scale decomposition, reducing dimensionality by assuming a stationary local Markov model for wavelet coefficients conditioned on coarser-scale coefficients. We instantiate this model using convolutional neural networks (CNNs) with local receptive fields, which enforce both the stationarity and Markov properties. Global structures are captured using a CNN with receptive fields covering the entire (but small) low-pass image. We test this model on a dataset of face images, which are highly non-stationary and contain large-scale geometric structures. Remarkably, denoising, super-resolution, and image synthesis results all demonstrate that these structures can be captured with significantly smaller conditioning neighborhoods than required by a Markov model implemented in the pixel domain. Our results show that score estimation for large complex images can be reduced to low-dimensional Markov conditional models across scales, alleviating the curse of dimensionality. 

   more » « less
3. Geometrically Motivated Reparameterization for Identifiability Analysis in Power Systems Models

   https://doi.org/10.1109/NAPS.2018.8600617  

   Transtrum, Mark K.; Francis, Benjamin L.; Youn, Clifford C.; Saric, Andrija T.; Stankovic, Aleksander M. (September 2018, 2018 North American Power Symposium (NAPS))

   This paper describes a geometric approach to parameter identifiability analysis in models of power systems dynamics. When a model of a power system is to be compared with measurements taken at discrete times, it can be interpreted as a mapping from parameter space into a data or prediction space. Generically, model mappings can be interpreted as manifolds with dimensionality equal to the number of structurally identifiable parameters. Empirically it is observed that model mappings often correspond to bounded manifolds. We propose a new definition of practical identifiability based the topological definition of a manifold with boundary. In many ways, our proposed definition extends the properties of structural identifiability. We construct numerical approximations to geodesics on the model manifold and use the results, combined with insights derived from the mathematical form of the equations, to identify combinations of practically identifiable and unidentifiable parameters. We give several examples of application to dynamic power systems models. 

   more » « less
4. FibeRed: Fiberwise Dimensionality Reduction of Topologically Complex Data with Vector Bundles

   https://doi.org/10.4230/LIPIcs.SoCG.2023.56  

   Scoccola, Luis; Perea, Jose A. (January 2023, Leibniz international proceedings in informatics)

   Chambers, Erin W.; Gudmundsson, Joachim (Ed.)

   Datasets with non-trivial large scale topology can be hard to embed in low-dimensional Euclidean space with existing dimensionality reduction algorithms. We propose to model topologically complex datasets using vector bundles, in such a way that the base space accounts for the large scale topology, while the fibers account for the local geometry. This allows one to reduce the dimensionality of the fibers, while preserving the large scale topology. We formalize this point of view and, as an application, we describe a dimensionality reduction algorithm based on topological inference for vector bundles. The algorithm takes as input a dataset together with an initial representation in Euclidean space, assumed to recover part of its large scale topology, and outputs a new representation that integrates local representations obtained through local linear dimensionality reduction. We demonstrate this algorithm on examples coming from dynamical systems and chemistry. In these examples, our algorithm is able to learn topologically faithful embeddings of the data in lower target dimension than various well known metric-based dimensionality reduction algorithms. 

   more » « less
5. Handling Missing Data with Graph Representation Learning

   You, Jiaxuan; Ma, Xiaobai; Ding, Daisy Yi; Kochenderfer, Mykel; Leskovec, Jure. (January 2020, Neural Information Processing Systems (NeurIPS)

   Machine learning with missing data has been approached in two different ways, including feature imputation where missing feature values are estimated based on observed values and label prediction where downstream labels are learned directly from incomplete data. However, existing imputation models tend to have strong prior assumptions and cannot learn from downstream tasks, while models targeting label prediction often involve heuristics and can encounter scalability issues. Here we propose GRAPE, a graph-based framework for feature imputation as well as label prediction. GRAPE tackles the missing data problem using a graph representation, where the observations and features are viewed as two types of nodes in a bipartite graph, and the observed feature values as edges. Under the GRAPE framework, the feature imputation is formulated as an edge-level prediction task and the label prediction as a node-level prediction task. These tasks are then solved with Graph Neural Networks. Experimental results on nine benchmark datasets show that GRAPE yields 20% lower mean absolute error for imputation tasks and 10% lower for label prediction tasks, compared with existing state-of-the-art methods. 

   more » « less