More Like this

1. Directed Chain Generative Adversarial Networks

   Ming Min, Ruimeng Hu ( January 2023 , Proceedings of the 40th International Conference on Machine Learning)

   Real-world data can be multimodal distributed, e.g., data describing the opinion divergence in a community, the interspike interval distribution of neurons, and the oscillators’ natural frequen- cies. Generating multimodal distributed real- world data has become a challenge to existing generative adversarial networks (GANs). For ex- ample, it is often observed that Neural SDEs have only demonstrated successful performance mainly in generating unimodal time series datasets. In this paper, we propose a novel time series gen- erator, named directed chain GANs (DC-GANs), which inserts a time series dataset (called a neigh- borhood process of the directed chain or input) into the drift and diffusion coefficients of the di- rected chain SDEs with distributional constraints. DC-GANs can generate new time series of the same distribution as the neighborhood process, and the neighborhood process will provide the key step in learning and generating multimodal dis- tributed time series. The proposed DC-GANs are examined on four datasets, including two stochas- tic models from social sciences and computa- tional neuroscience, and two real-world datasets on stock prices and energy consumption. To our best knowledge, DC-GANs are the first work that can generate multimodal time series data and con- sistently outperforms state-of-the-art benchmarks with respect to measures of distribution, data sim- ilarity, and predictive ability. 

   more » « less
2. Second‐order preserving point process permutations

   https://doi.org/10.1002/sta4.558  

   Mohler, George ; Mateu, Jorge ( March 2023 , Stat)

   While random permutations of point processes are useful for generating counterfactuals in bivariate interaction tests, such permutations require that the underlying intensity be separable. In many real‐world datasets where clustering or inhibition is present, such an assumption does not hold. Here, we introduce a simple combinatorial optimization algorithm that generates second‐order preserving (SOP) point process permutations, for example, permutations of the times of events such that the function of the permuted process matches the function of the data. We apply the algorithm to synthetic data generated by a self‐exciting Hawkes process and a self‐avoiding point process, along with data from Los Angeles on earthquakes and arsons and data from Indianapolis on law enforcement drug seizures and overdoses. In all cases, we are able to generate a diverse sample of permuted point processes where the distribution of the functions closely matches that of the data. We then show how SOP point process permutations can be used in two applications: (1) bivariate Knox tests and (2) data augmentation to improve deep learning‐based space‐time forecasts.

    

   more » « less
3. AE-OT-GAN: Training GANs from data specific latent distribution

   Dongsheng An, Yang Guo ( January 2020 , European Conference on Computer Vision (ECCV2020))

   Though generative adversarial networks (GANs) are prominent models to generate realistic and crisp images, they are unstable to train and suffer from the mode collapse problem. The problems of GANs come from approximating the intrinsic discontinuous distribution transform map with continuous DNNs. The recently proposed AE-OT model addresses the discontinuity problem by explicitly computing the discontinuous optimal transform map in the latent space of the autoencoder. Though have no mode collapse, the generated images by AE-OT are blurry. In this paper, we propose the AE-OT-GAN model to utilize the advantages of the both models: generate high quality images and at the same time overcome the mode collapse problems. Specifically, we firstly embed the low dimensional image manifold into the latent space by autoencoder (AE). Then the extended semi-discrete optimal transport (SDOT) map is used to generate new latent codes. Finally, our GAN model is trained to generate high quality images from the latent distribution induced by the extended SDOT map. The distribution transform map from this dataset related latent distribution to the data distribution will be continuous, and thus can be well approximated by the continuous DNNs. Additionally, the paired data between the latent codes and the real images gives us further restriction about the generator and stabilizes the training process. Experiments on simple MNIST dataset and complex datasets like CIFAR10 and CelebA show the advantages of the proposed method. 

   more » « less
4. Learning stochastic closures using ensemble Kalman inversion

   https://doi.org/10.1093/imatrm/tnab003  

   Schneider, Tapio ; Stuart, Andrew M ; Wu, Jin-Long ( January 2021 , Transactions of Mathematics and Its Applications)

   Abstract Although the governing equations of many systems, when derived from first principles, may be viewed as known, it is often too expensive to numerically simulate all the interactions they describe. Therefore, researchers often seek simpler descriptions that describe complex phenomena without numerically resolving all the interacting components. Stochastic differential equations (SDEs) arise naturally as models in this context. The growth in data acquisition, both through experiment and through simulations, provides an opportunity for the systematic derivation of SDE models in many disciplines. However, inconsistencies between SDEs and real data at short time scales often cause problems, when standard statistical methodology is applied to parameter estimation. The incompatibility between SDEs and real data can be addressed by deriving sufficient statistics from the time-series data and learning parameters of SDEs based on these. Here, we study sufficient statistics computed from time averages, an approach that we demonstrate to lead to sufficient statistics on a variety of problems and that has the secondary benefit of obviating the need to match trajectories. Following this approach, we formulate the fitting of SDEs to sufficient statistics from real data as an inverse problem and demonstrate that this inverse problem can be solved by using ensemble Kalman inversion. Furthermore, we create a framework for non-parametric learning of drift and diffusion terms by introducing hierarchical, refinable parameterizations of unknown functions, using Gaussian process regression. We demonstrate the proposed methodology for the fitting of SDE models, first in a simulation study with a noisy Lorenz ’63 model, and then in other applications, including dimension reduction in deterministic chaotic systems arising in the atmospheric sciences, large-scale pattern modeling in climate dynamics and simplified models for key observables arising in molecular dynamics. The results confirm that the proposed methodology provides a robust and systematic approach to fitting SDE models to real data. 

   more » « less
5. PMGAN: Paralleled Mix-Generator Generative Adversarial Networks with Balance Control

   Xiao, Xia ; Rajasekaran, Sanguthevar ( January 2018 , The 27th International Conference on Artificial Neural Networks)

   A Generative Adversarial Network (GAN) is an unsupervised generative framework to generate a sample distribution that is identical to the data distribution. Recently, mix strategy multi-generator/discriminator GANs have been shown to outperform single pair GANs. However, the mixed model suffers from the problem of linearly growing training time. Also, imbalanced training among generators makes it difficult to parallelize. In this paper, we propose a balanced mix-generator GAN that works in parallel by mixing multiple disjoint generators to approximate the real distribution. The weights of the discriminator and the classifier are controlled by a balance strategy. We also present an efficient loss function, to force each generator to embrace few modes with a high probability. Our model is naturally adaptive to large parallel computation frameworks. Each generator can be trained on multiple GPUs asynchronously. We have performed extensive experiments on synthetic datasets, MNIST1000, CIFAR-10, and ImageNet. The results establish that our model can achieve the state-of-the-art performance (in terms of the modes coverage and the inception score), with significantly reduced training time. We also show that the missing mode problem can be relieved with a growing number of generators. 

   more » « less