An official website of the United States government
Knowledge discovery and information extraction of large and complex datasets has attracted great attention in wide-ranging areas from statistics and biology to medicine. Tools from machine learning, data mining, and neurocomputing have been extensively explored and utilized to accomplish such compelling data analytics tasks. However, for time-series data presenting active dynamic characteristics, many of the state-of-the-art techniques may not perform well in capturing the inherited temporal structures in these data. In this paper, integrating the Koopman operator and linear dynamical systems theory with support vector machines (SVMs), we develop a novel dynamic data mining framework to construct low-dimensional linear models that approximate the nonlinear flow of high-dimensional time-series data generated by unknown nonlinear dynamical systems. This framework then immediately enables pattern recognition, e.g., classification, of complex time-series data to distinguish their dynamic behaviors by using the trajectories generated by the reduced linear systems. Moreover, we demonstrate the applicability and efficiency of this framework through the problems of time-series classification in bioinformatics and healthcare, including cognitive classification and seizure detection with fMRI and EEG data, respectively. The developed Koopman dynamic learning framework then lays a solid foundation for effective dynamic data mining and promises a mathematically justified method for extracting the dynamics and significant temporal structures of nonlinear dynamical systems.
more »
« less
Nonlinear time series classification using bispectrum‐based deep convolutional neural networks
Abstract Time series classification using novel techniques has experienced a recent resurgence and growing interest from statisticians, subject‐domain scientists, and decision makers in business and industry. This is primarily due to the ever increasing amount of big and complex data produced as a result of technological advances. A motivating example is that of Google trends data, which exhibit highly nonlinear behavior. Although a rich literature exists for addressing this problem, existing approaches mostly rely on first‐ and second‐order properties of the time series, since they typically assume linearity of the underlying process. Often, these are inadequate for effective classification of nonlinear time series data such as Google Trends data. Given these methodological deficiencies and the abundance of nonlinear time series that persist among real‐world phenomena, we introduce an approach that merges higher order spectral analysis with deep convolutional neural networks for classifying time series. The effectiveness of our approach is illustrated using simulated data and two motivating industry examples that involve Google trends data and electronic device energy consumption data.
more »
« less
- Award ID(s):
- 1853096
- PAR ID:
- 10456242
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Applied Stochastic Models in Business and Industry
- Volume:
- 36
- Issue:
- 5
- ISSN:
- 1524-1904
- Format(s):
- Medium: X Size: p. 877-890
- Size(s):
- p. 877-890
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Landsat 5 has produced imagery for decades that can now be viewed and manipulated in Google Earth Engine, but a general, automated way of producing a coherent time series from these images—particularly over cloudy areas in the distant past—is elusive. Here, we create a land use and land cover (LULC) time series for part of tropical Mato Grosso, Brazil, using the Bayesian Updating of Land Cover: Unsupervised (BULC-U) technique. The algorithm built backward in time from the GlobCover 2009 data set, a multi-category global LULC data set at 300 m resolution for the year 2009, combining it with Landsat time series imagery to create a land cover time series for the period 1986–2000. Despite the substantial LULC differences between the 1990s and 2009 in this area, much of the landscape remained the same: we asked whether we could harness those similarities and differences to recreate an accurate version of the earlier LULC. The GlobCover basis and the Landsat-5 images shared neither a common spatial resolution nor time frame, But BULC-U successfully combined the labels from the coarser classification with the spatial detail of Landsat. The result was an accurate fine-scale time series that quantified the expansion of deforestation in the study area, which more than doubled in size during this time. Earth Engine directly enabled the fusion of these different data sets held in its catalog: its flexible treatment of spatial resolution, rapid prototyping, and overall processing speed permitted the development and testing of this study. Many would-be users of remote sensing data are currently limited by the need to have highly specialized knowledge to create classifications of older data. The approach shown here presents fewer obstacles to participation and allows a wide audience to create their own time series of past decades. By leveraging both the varied data catalog and the processing speed of Earth Engine, this research can contribute to the rapid advances underway in multi-temporal image classification techniques. Given Earth Engine’s power and deep catalog, this research further opens up remote sensing to a rapidly growing community of researchers and managers who need to understand the long-term dynamics of terrestrial systems.more » « less
-
Abstract Most current algorithms for multivariate time series classification tend to overlook the correlations between time series of different variables. In this research, we propose a framework that leverages Eigen-entropy along with a cumulative moving window to derive time series signatures to support the classification task. These signatures are enumerations of correlations among different time series considering the temporal nature of the dataset. To manage dataset’s dynamic nature, we employ preprocessing with dense multi scale entropy. Consequently, the proposed framework, Eigen-entropy-based Time Series Signatures, captures correlations among multivariate time series without losing its temporal and dynamic aspects. The efficacy of our algorithm is assessed using six binary datasets sourced from the University of East Anglia, in addition to a publicly available gait dataset and an institutional sepsis dataset from the Mayo Clinic. We use recall as the evaluation metric to compare our approach against baseline algorithms, including dependent dynamic time warping with 1 nearest neighbor and multivariate multi-scale permutation entropy. Our method demonstrates superior performance in terms of recall for seven out of the eight datasets.more » « less
-
null (Ed.)This paper presents a generalizable methodology for data-driven identification of nonlinear dynamics that bounds the model error in terms of the prediction horizon and the magnitude of the derivatives of the system states. Using higher order derivatives of general nonlinear dynamics that need not be known, we construct a Koopman operator-based linear representation and utilize Taylor series accuracy analysis to derive an error bound. The resulting error formula is used to choose the order of derivatives in the basis functions and obtain a data-driven Koopman model using a closed-form expression that can be computed in real time. Using the inverted pendulum system, we illustrate the robustness of the error bounds given noisy measurements of unknown dynamics, where the derivatives are estimated numerically. When combined with control, the Koopman representation of the nonlinear system has marginally better performance than competing nonlinear modeling methods, such as SINDy and NARX. In addition, as a linear model, the Koopman approach lends itself readily to efficient control design tools, such as LQR, whereas the other modeling approaches require nonlinear control methods. The efficacy of the approach is further demonstrated with simulation and experimental results on the control of a tail-actuated robotic fish. Experimental results show that the proposed data-driven control approach outperforms a tuned PID (Proportional Integral Derivative) controller and that updating the data-driven model online significantly improves performance in the presence of unmodeled fluid disturbance. This paper is complemented with a video: https://youtu.be/9 wx0tdDta0.more » « less
-
Single-molecule and related experiments yield time series of an observable as it fluctuates due to thermal motion. In such data, it can be difficult to distinguish fluctuating signal from fluctuating noise. We present a method of separating signal from noise using nonlinear-correlation functions. The method is fully nonparametric: No a priori model for the system is required, no knowledge of whether the system is continuous or discrete is needed, the number of states is not fixed, and the system can be Markovian or not. The noise-corrected, nonlinear-correlation functions can be converted to the system’s Green’s function; the noise-corrected moments yield the system’s equilibrium-probability distribution. As a demonstration, we analyze synthetic data from a three-state system. The correlation method is compared to another fully nonparametric approach—time binning to remove noise, and histogramming to obtain the distribution. The correlation method has substantially better resolution in time and in state space. We develop formulas for the limits on data quality needed for signal recovery from time series and test them on datasets of varying size and signal-to-noise ratio. The formulas show that the signal-to-noise ratio needs to be on the order of or greater than one-half before convergence scales at a practical rate. With experimental benchmark data, the positions and populations of the states and their exchange rates are recovered with an accuracy similar to parametric methods. The methods demonstrated here are essential components in building a complete analysis of time series using only high-order correlation functions.more » « less
