An official website of the United States government
For stationary time series with regularly varying marginal distributions, an important problem is to estimate the associated tail index which characterizes the power‐law behavior of the tail distribution. For this, various results have been developed for independent data and certain types of dependent data. In this article, we consider the problem of tail index estimation under a recently proposed notion of serial tail dependence called the tail adversarial stability. Using the technique of adversarial innovation coupling and a martingale approximation scheme, we establish the consistency and central limit theorem of the tail index estimator for a general class of tail dependent time series. Based on the asymptotic normal distribution from the obtained central limit theorem, we further consider an application to cluster a large number of regularly varying time series based on their tail indices by using a robust mixture algorithm. The results are illustrated using numerical examples including Monte Carlo simulations and a real data analysis.
more »
« less
Minimum variance unbiased estimation in the presence of an adversary
Consider a setup in which a central estimator seeks to estimate an unknown deterministic parameter using measurements from multiple sensors. Some of the sensors may be adversarial in that their utility increases with the Euclidean distance between the estimate of the central estimator and their own local estimate. These sensors may misreport their measurements to the central estimator at a falsification cost. We formulate a Stackelberg game in which the central estimator acts as the leader and the adversarial sensors act as the follower. We present the optimal linear fusion scheme for the estimator and the optimal attack pattern for the adversarial sensors in the Nash equilibrium sense. Interestingly, the estimate at the central estimator may be better than if the measurements from the adversarial sensors were altogether ignored.
more »
« less
- Award ID(s):
- 1739295
- PAR ID:
- 10076436
- Date Published:
- Journal Name:
- 2017 IEEE 56th Annual Conference on Decision and Control (CDC)
- Page Range / eLocation ID:
- 151 to 156
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In this paper, we study the impact of the presence of byzantine sensors on the reduced-rank linear least squares (LS) estimator. A sensor network with N sensors makes observations of the physical phenomenon and transmits them to a fusion center which computes the LS estimate of the parameter of interest. It is well-known that rank reduction exploits the bias-variance trade-off in the full-rank estimator by putting higher priority on highly informative content of the data. The low-rank LS estimator is constructed using this highly informative content, while the remaining data can be discarded without affecting the overall performance of the estimator. We consider the scenario where a fraction of the N sensors are subject to data falsification attack from byzantine sensors, wherein an intruder injects a higher noise power (compared to the unattacked sensors) to the measurements of the attacked sensors. Our main contribution is an analytical characterization of the impact of data falsification attack of the above type on the performance of reduced-rank LS estimator. In particular, we show how optimally prioritizing the highly informative content of the data gets affected in the presence of attacks. A surprising result is that, under sensor attacks, when the elements of the data matrix are all positive the error performance of the low rank estimator experiences a phenomenon wherein the estimate of the mean-squared error comprises negative components. A complex nonlinear programming-based recipe is known to exist that resolves this undesirable effect; however, the phenomenon is oftentimes considered very objectionable in the statistical literature. On the other hand, to our advantage this effect can serve to detect cyber attacks on sensor systems. Numerical results are presented to complement the theoretical findings of the paper.more » « less
-
Experimental design is a classical area in statistics and has also found new applications in machine learning. In the combinatorial experimental design problem, the aim is to estimate an unknown m-dimensional vector x from linear measurements where a Gaussian noise is introduced in each measurement. The goal is to pick k out of the given n experiments so as to make the most accurate estimate of the unknown parameter x. Given a set S of chosen experiments, the most likelihood estimate x0 can be obtained by a least squares computation. One of the robust measures of error estimation is the D-optimality criterion which aims to minimize the generalized variance of the estimator. This corresponds to minimizing the volume of the standard confidence ellipsoid for the estimation error x − x0. The problem gives rise to two natural variants depending on whether repetitions of experiments is allowed or not. The latter variant, while being more general, has also found applications in geographical location of sensors. We show a close connection between approximation algorithms for the D-optimal design problem and constructions of approximately m-wise positively correlated distributions. This connection allows us to obtain first approximation algorithms for the D-optimal design problem with and without repetitions. We then consider the case when the number of experiments chosen is much larger than the dimension m and show one can obtain asymptotically optimal algorithms in this case.more » « less
-
We study the instantaneous inference of an unbounded planar flow from sparse noisy pressure measurements. The true flow field comprises one or more regularized point vortices of various strength and size. We interpret the true flow’s measurements with a vortex estimator, also consisting of regularized vortices, and attempt to infer the positions and strengths of this estimator assuming little prior knowledge. The problem often has several possible solutions, many due to a variety of symmetries. To deal with this ill-posedness and to quantify the uncertainty, we develop the vortex estimator in a Bayesian setting. We use Markov-chain Monte Carlo and a Gaussian mixture model to sample and categorize the probable vortex states in the posterior distribution, tailoring the prior to avoid spurious solutions. Through experiments with one or more true vortices, we reveal many aspects of the vortex inference problem. With fewer sensors than states, the estimator infers a manifold of equally-possible states. Using one more sensor than states ensures that no cases of rank deficiency arise. Uncertainty grows rapidly with distance when a vortex lies outside of the vicinity of the sensors. Vortex size cannot be reliably inferred, but the position and strength of a larger vortex can be estimated with a much smaller one. In estimates of multiple vortices their individual signs are discernible because of the non-linear coupling in the pressure. When the true vortex state is inferred from an estimator of fewer vortices, the estimate approximately aggregates the true vortices where possible.more » « less
-
Location estimation is one of the most basic questions in parametric statistics. Suppose we have a known distribution density f , and we get n i.i.d. samples from f (x − μ) for some unknown shift μ. The task is to estimate μ to high accuracy with high probability. The maximum likelihood estimator (MLE) is known to be asymptotically optimal as n → ∞, but what is possible for finite n? In this paper, we give two location estimators that are optimal under different criteria: 1) an estimator that has minimax-optimal estimation error subject to succeeding with probability 1 − ¶ and 2) a confidence interval estimator which, subject to its output interval containing μ with probability at least 1 − ¶, has the minimum expected squared interval width among all shift-invariant estimators. The latter construction can be generalized to minimizing the expectation of any loss function on the interval width.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/CDC.2017.8263658
