skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Phase transitions of spectral initialization for high-dimensional non-convex estimation
Abstract We study a spectral initialization method that serves a key role in recent work on estimating signals in non-convex settings. Previous analysis of this method focuses on the phase retrieval problem and provides only performance bounds. In this paper, we consider arbitrary generalized linear sensing models and present a precise asymptotic characterization of the performance of the method in the high-dimensional limit. Our analysis also reveals a phase transition phenomenon that depends on the ratio between the number of samples and the signal dimension. When the ratio is below a minimum threshold, the estimates given by the spectral method are no better than random guesses drawn from a uniform distribution on the hypersphere, thus carrying no information; above a maximum threshold, the estimates become increasingly aligned with the target signal. The computational complexity of the method, as measured by the spectral gap, is also markedly different in the two phases. Worked examples and numerical results are provided to illustrate and verify the analytical predictions. In particular, simulations show that our asymptotic formulas provide accurate predictions for the actual performance of the spectral method even at moderate signal dimensions.  more » « less
Award ID(s):
1718698
PAR ID:
10125407
Author(s) / Creator(s):
 ;  
Publisher / Repository:
Oxford University Press
Date Published:
Journal Name:
Information and Inference: A Journal of the IMA
ISSN:
2049-8772
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract On-chip spectrometers have the potential to offer dramatic size, weight, and power advantages over conventional benchtop instruments for many applications such as spectroscopic sensing, optical network performance monitoring, hyperspectral imaging, and radio-frequency spectrum analysis. Existing on-chip spectrometer designs, however, are limited in spectral channel count and signal-to-noise ratio. Here we demonstrate a transformative on-chip digital Fourier transform spectrometer that acquires high-resolution spectra via time-domain modulation of a reconfigurable Mach-Zehnder interferometer. The device, fabricated and packaged using industry-standard silicon photonics technology, claims the multiplex advantage to dramatically boost the signal-to-noise ratio and unprecedented scalability capable of addressing exponentially increasing numbers of spectral channels. We further explore and implement machine learning regularization techniques to spectrum reconstruction. Using an ‘elastic-D1’ regularized regression method that we develop, we achieved significant noise suppression for both broad (>600 GHz) and narrow (<25 GHz) spectral features, as well as spectral resolution enhancement beyond the classical Rayleigh criterion. 
    more » « less
  2. ABSTRACT Low-mass X-ray binaries have long been theorized as potential sources of continuous gravitational-wave radiation, yet there is no observational evidence from recent LIGO/Virgo observing runs. Even for the theoretically ‘loudest’ source, Sco X-1, the upper limit on gravitational-wave strain has been pushed ever lower. Such searches require precise measurements of the source properties for sufficient sensitivity and computational feasibility. Collating over 20 yr of high-quality spectroscopic observations of the system, we present a precise and comprehensive ephemeris for Sco X-1 through radial velocity measurements, performing a full homogeneous re-analysis of all relevant data sets and correcting previous analyses. Our Bayesian approach accounts for observational systematics and maximizes not only precision, but also the fidelity of uncertainty estimates – crucial for informing principled continuous-wave searches. Our extensive data set and analysis also enables us to construct the highest signal-to-noise ratio, highest resolution phase-averaged spectrum of a low-mass X-ray binary to date. Doppler tomography reveals intriguing transient structures present in the accretion disc and flow driven by modulation of the accretion rate, necessitating further characterization of the system at high temporal and spectral resolution. Our ephemeris corrects and supersedes previous ephemerides, and provides a factor three reduction in the number of templates in the search space, facilitating precision searches for continuous gravitational-wave emission from Sco X-1 throughout the upcoming LIGO/Virgo/KAGRA O4 observing run and beyond. 
    more » « less
  3. Stress drop, a crucial source parameter in earthquake studies, significantly influences ground motion prediction and seismic hazard assessment. Despite several existing methods to estimate stress drops, the resulting stress drop estimates often exhibit a wide variation of up to 3-4 orders of magnitude. In this study, we address the robustness of stress drop estimation by introducing a point-wise spectral ratio stacking approach based on empirical Green’s functions (eGfs). Conventional trace-wise stacking can lead to data exclusion due to high signal-to-noise ratio requirements across a wide range of frequency. By adopting point-wise stacking, we maximize the utilization of useful recording information, leading to more accurate stress drop estimates. We applied the point-wise spectral ratio stacking method to a comprehensive dataset comprising global earthquakes from 1990 to 2020 with magnitude larger than Mw5.5 and depth shallower than 50 km. We first verified the moment magnitudes of earthquakes estimated from the resulting seismic moment ratios. We found that the moment magnitude of master events best consistent with catalog magnitudes when the magnitude difference between master and their eGfs differs by about 0.5. Our analysis indicates that stress drop of shallow earthquakes exhibits no depth dependence, while showing a slight increase with magnitude. The results obtained through our optimized stacking process shed new light on stress drop estimate of shallow earthquakes and have the potential to enhance the understanding of earthquake mechanics. 
    more » « less
  4. This paper leverages machine-learned predictions to design competitive algorithms for online conversion problems with the goal of improving the competitive ratio when predictions are accurate (i.e., consistency), while also guaranteeing a worst-case competitive ratio regardless of the prediction quality (i.e., robustness). We unify the algorithmic design of both integral and fractional conversion problems, which are also known as the 1-max-search and one-way trading problems, into a class of online threshold-based algorithms (OTA). By incorporating predictions into design of OTA, we achieve the Pareto-optimal trade-off of consistency and robustness, i.e., no online algorithm can achieve a better consistency guarantee given for a robustness guarantee. We demonstrate the performance of OTA using numerical experiments on Bitcoin conversion. 
    more » « less
  5. We consider the problem of comparing two complex multivariate random signal realizations of unequal lengths, to ascertain whether they have identical power spectral densities. A binary hypothesis testing approach is formulated and a generalized likelihood ratio test (GLRT) is derived. An asymptotic analytical solution for calculating the test threshold is provided. The results are illustrated via computer simulations. Past work on this problem is limited to either complex or real signals of equal lengths, or to real-valued scalar signals of unequal lengths. The proposed test has applications in diverse areas including user authentication in wireless networks with multiantenna receivers. 
    more » « less