The cumulative empirical spectral measure (CESM) $\Phi[\mathbf{A}] : \mathbb{R} \to [0,1]$ of a $n\times n$ symmetric matrix $\mathbf{A}$ is defined as the fraction of eigenvalues of $\mathbf{A}$ less than a given threshold, i.e., $\Phi[\mathbf{A}](x) := \sum_{i=1}^{n} \frac{1}{n} {\large\unicode{x1D7D9}}[ \lambda_i[\mathbf{A}]\leq x]$. Spectral sums $\operatorname{tr}(f[\mathbf{A}])$ can be computed as the Riemann–Stieltjes integral of $f$ against $\Phi[\mathbf{A}]$, so the task of estimating CESM arises frequently in a number of applications, including machine learning. We present an error analysis for stochastic Lanczos quadrature (SLQ). We show that SLQ obtains an approximation to the CESM within a Wasserstein distance of $t \: | \lambda_{\text{max}}[\mathbf{A}] - \lambda_{\text{min}}[\mathbf{A}] |$ with probability at least $1-\eta$, by applying the Lanczos algorithm for $\lceil 12 t^{-1} + \frac{1}{2} \rceil$ iterations to $\lceil 4 ( n+2 )^{-1}t^{-2} \ln(2n\eta^{-1}) \rceil$ vectors sampled independently and uniformly from the unit sphere. We additionally provide (matrix-dependent) a posteriori error bounds for the Wasserstein and Kolmogorov–Smirnov distances between the output of this algorithm and the true CESM. The quality of our bounds is demonstrated using numerical experiments.
more »
« less
Edge Expansion and Spectral Gap of Nonnegative Matrices
The classic graphical Cheeger inequalities state that if $M$ is an
$n\times n$ \emph{symmetric} doubly stochastic matrix, then
\[
\frac{1-\lambda_{2}(M)}{2}\leq\phi(M)\leq\sqrt{2\cdot(1-\lambda_{2}(M))}
\]
where $\phi(M)=\min_{S\subseteq[n],|S|\leq n/2}\left(\frac{1}{|S|}\sum_{i\in S,j\not\in S}M_{i,j}\right)$
is the edge expansion of $M$, and $\lambda_{2}(M)$ is the second
largest eigenvalue of $M$. We study the relationship between $\phi(A)$
and the spectral gap $1-\re\lambda_{2}(A)$ for \emph{any} doubly
stochastic matrix $A$ (not necessarily symmetric), where $\lambda_{2}(A)$
is a nontrivial eigenvalue of $A$ with maximum real part. Fiedler
showed that the upper bound on $\phi(A)$ is unaffected, i.e., $\phi(A)\leq\sqrt{2\cdot(1-\re\lambda_{2}(A))}$.
With regards to the lower bound on $\phi(A)$, there are known constructions
with
\[
\phi(A)\in\Theta\left(\frac{1-\re\lambda_{2}(A)}{\log n}\right),
\]
indicating that at least a mild dependence on $n$ is necessary to
lower bound $\phi(A)$.
In our first result, we provide an \emph{exponentially} better construction
of $n\times n$ doubly stochastic matrices $A_{n}$, for which
\[
\phi(A_{n})\leq\frac{1-\re\lambda_{2}(A_{n})}{\sqrt{n}}.
\]
In fact, \emph{all} nontrivial eigenvalues of our matrices are $0$,
even though the matrices are highly \emph{nonexpanding}. We further
show that this bound is in the correct range (up to the exponent of
$n$), by showing that for any doubly stochastic matrix $A$,
\[
\phi(A)\geq\frac{1-\re\lambda_{2}(A)}{35\cdot n}.
\]
As a consequence, unlike the symmetric case, there is a (necessary)
loss of a factor of $n^{\alpha}$ for $\frac{1}{2}\leq\alpha\leq1$
in lower bounding $\phi$ by the spectral gap in the nonsymmetric
setting.
Our second result extends these bounds to general matrices $R$ with
nonnegative entries, to obtain a two-sided \emph{gapped} refinement
of the Perron-Frobenius theorem. Recall from the Perron-Frobenius
theorem that for such $R$, there is a nonnegative eigenvalue $r$
such that all eigenvalues of $R$ lie within the closed disk of radius
$r$ about $0$. Further, if $R$ is irreducible, which means $\phi(R)>0$
(for suitably defined $\phi$), then $r$ is positive and all other
eigenvalues lie within the \textit{open} disk, so (with eigenvalues
sorted by real part), $\re\lambda_{2}(R)
more »
« less
- NSF-PAR ID:
- 10190072
- Date Published:
- Journal Name:
- Proceedings of the 2020 ACM-SIAM Symposium on Discrete Algorithms
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
An \ell _p oblivious subspace embedding is a distribution over r \times n matrices \Pi such that for any fixed n \times d matrix A , \[ \Pr _{\Pi }[\textrm {for all }x, \ \Vert Ax\Vert _p \le \Vert \Pi Ax\Vert _p \le \kappa \Vert Ax\Vert _p] \ge 9/10,\] where r is the dimension of the embedding, \kappa is the distortion of the embedding, and for an n -dimensional vector y , \Vert y\Vert _p = (\sum _{i=1}^n |y_i|^p)^{1/p} is the \ell _p -norm. Another important property is the sparsity of \Pi , that is, the maximum number of non-zero entries per column, as this determines the running time of computing \Pi A . While for p = 2 there are nearly optimal tradeoffs in terms of the dimension, distortion, and sparsity, for the important case of 1 \le p \lt 2 , much less was known. In this article, we obtain nearly optimal tradeoffs for \ell _1 oblivious subspace embeddings, as well as new tradeoffs for 1 \lt p \lt 2 . Our main results are as follows: (1) We show for every 1 \le p \lt 2 , any oblivious subspace embedding with dimension r has distortion \[ \kappa = \Omega \left(\frac{1}{\left(\frac{1}{d}\right)^{1 / p} \log ^{2 / p}r + \left(\frac{r}{n}\right)^{1 / p - 1 / 2}}\right).\] When r = {\operatorname{poly}}(d) \ll n in applications, this gives a \kappa = \Omega (d^{1/p}\log ^{-2/p} d) lower bound, and shows the oblivious subspace embedding of Sohler and Woodruff (STOC, 2011) for p = 1 is optimal up to {\operatorname{poly}}(\log (d)) factors. (2) We give sparse oblivious subspace embeddings for every 1 \le p \lt 2 . Importantly, for p = 1 , we achieve r = O(d \log d) , \kappa = O(d \log d) and s = O(\log d) non-zero entries per column. The best previous construction with s \le {\operatorname{poly}}(\log d) is due to Woodruff and Zhang (COLT, 2013), giving \kappa = \Omega (d^2 {\operatorname{poly}}(\log d)) or \kappa = \Omega (d^{3/2} \sqrt {\log n} \cdot {\operatorname{poly}}(\log d)) and r \ge d \cdot {\operatorname{poly}}(\log d) ; in contrast our r = O(d \log d) and \kappa = O(d \log d) are optimal up to {\operatorname{poly}}(\log (d)) factors even for dense matrices. We also give (1) \ell _p oblivious subspace embeddings with an expected 1+\varepsilon number of non-zero entries per column for arbitrarily small \varepsilon \gt 0 , and (2) the first oblivious subspace embeddings for 1 \le p \lt 2 with O(1) -distortion and dimension independent of n . Oblivious subspace embeddings are crucial for distributed and streaming environments, as well as entrywise \ell _p low-rank approximation. Our results give improved algorithms for these applications.more » « less
-
more » « less
Abstract Given a sequence $\{Z_d\}_{d\in \mathbb{N}}$ of smooth and compact hypersurfaces in ${\mathbb{R}}^{n-1}$, we prove that (up to extracting subsequences) there exists a regular definable hypersurface $\Gamma \subset {\mathbb{R}}\textrm{P}^n$ such that each manifold $Z_d$ is diffeomorphic to a component of the zero set on $\Gamma$ of some polynomial of degree $d$. (This is in sharp contrast with the case when $\Gamma$ is semialgebraic, where for example the homological complexity of the zero set of a polynomial $p$ on $\Gamma$ is bounded by a polynomial in $\deg (p)$.) More precisely, given the above sequence of hypersurfaces, we construct a regular, compact, semianalytic hypersurface $\Gamma \subset {\mathbb{R}}\textrm{P}^{n}$ containing a subset $D$ homeomorphic to a disk, and a family of polynomials $\{p_m\}_{m\in \mathbb{N}}$ of degree $\deg (p_m)=d_m$ such that $(D, Z(p_m)\cap D)\sim ({\mathbb{R}}^{n-1}, Z_{d_m}),$ i.e. the zero set of $p_m$ in $D$ is isotopic to $Z_{d_m}$ in ${\mathbb{R}}^{n-1}$. This says that, up to extracting subsequences, the intersection of $\Gamma$ with a hypersurface of degree $d$ can be as complicated as we want. We call these ‘pathological examples’. In particular, we show that for every $0 \leq k \leq n-2$ and every sequence of natural numbers $a=\{a_d\}_{d\in \mathbb{N}}$ there is a regular, compact semianalytic hypersurface $\Gamma \subset {\mathbb{R}}\textrm{P}^n$, a subsequence $\{a_{d_m}\}_{m\in \mathbb{N}}$ and homogeneous polynomials $\{p_{m}\}_{m\in \mathbb{N}}$ of degree $\deg (p_m)=d_m$ such that (0.1)$$\begin{equation}b_k(\Gamma\cap Z(p_m))\geq a_{d_m}.\end{equation}$$ (Here $b_k$ denotes the $k$th Betti number.) This generalizes a result of Gwoździewicz et al. [13]. On the other hand, for a given definable $\Gamma$ we show that the Fubini–Study measure, in the Gaussian probability space of polynomials of degree $d$, of the set $\Sigma _{d_m,a, \Gamma }$ of polynomials verifying (0.1) is positive, but there exists a constant $c_\Gamma$ such that $$\begin{equation*}0<{\mathbb{P}}(\Sigma_{d_m, a, \Gamma})\leq \frac{c_{\Gamma} d_m^{\frac{n-1}{2}}}{a_{d_m}}.\end{equation*}$$ This shows that the set of ‘pathological examples’ has ‘small’ measure (the faster $a$ grows, the smaller the measure and pathologies are therefore rare). In fact we show that given $\Gamma$, for most polynomials a Bézout-type bound holds for the intersection $\Gamma \cap Z(p)$: for every $0\leq k\leq n-2$ and $t>0$: $$\begin{equation*}{\mathbb{P}}\left(\{b_k(\Gamma\cap Z(p))\geq t d^{n-1} \}\right)\leq \frac{c_\Gamma}{td^{\frac{n-1}{2}}}.\end{equation*}$$
-
The densest subgraph problem in a graph (\dsg), in the simplest form, is the following. Given an undirected graph $G=(V,E)$ find a subset $S \subseteq V$ of vertices that maximizes the ratio $|E(S)|/|S|$ where $E(S)$ is the set of edges with both endpoints in $S$. \dsg and several of its variants are well-studied in theory and practice and have many applications in data mining and network analysis. In this paper we study fast algorithms and structural aspects of \dsg via the lens of \emph{supermodularity}. For this we consider the densest supermodular subset problem (\dssp): given a non-negative supermodular function $f: 2^V \rightarrow \mathbb{R}_+$, maximize $f(S)/|S|$. For \dsg we describe a simple flow-based algorithm that outputs a $(1-\eps)$-approximation in deterministic $\tilde{O}(m/\eps)$ time where $m$ is the number of edges. Our algorithm is the first to have a near-linear dependence on $m$ and $1/\eps$ and improves previous methods based on an LP relaxation. It generalizes to hypergraphs, and also yields a faster algorithm for directed \dsg. Greedy peeling algorithms have been very popular for \dsg and several variants due to their efficiency, empirical performance, and worst-case approximation guarantees. We describe a simple peeling algorithm for \dssp and analyze its approximation guarantee in a fashion that unifies several existing results. Boob et al.\ \cite{bgpstww-20} developed an \emph{iterative} peeling algorithm for \dsg which appears to work very well in practice, and made a conjecture about its convergence to optimality. We affirmatively answer their conjecture, and in fact prove that a natural generalization of their algorithm converges to a $(1-\eps)$-approximation for \emph{any} supermodular function $f$; the key to our proof is to consider an LP formulation that is derived via the \Lovasz extension of a supermodular function. For \dsg the bound on the number of iterations we prove is $O(\frac{\Delta \ln |V|}{\lambda^*}\cdot \frac{1}{\eps^2})$ where $\Delta$ is the maximum degree and $\lambda^*$ is the optimum value. Our work suggests that iterative peeling can be an effective heuristic for several objectives considered in the literature. Finally, we show that the $2$-approximation for densest-at-least-$k$ subgraph \cite{ks-09} extends to the supermodular setting. We also give a unified analysis of the peeling algorithm for this problem, and via this analysis derive an approximation guarantee for a generalization of \dssp to maximize $f(S)/g(|S|)$ for a concave function $g$.more » « less
-
more » « less
Abstract In this paper, we consider discrete Schrödinger operators of the form, $$\begin{equation*} (Hu)(n) = u({n+1})+u({n-1})+V(n)u(n). \end{equation*}$$We view $H$ as a perturbation of the free operator $H_0$, where $(H_0u)(n)= u({n+1})+u({n-1})$. For $H_0$ (no perturbation), $\sigma _{\textrm{ess}}(H_0)=\sigma _{\textrm{ac}}(H)=[-2,2]$ and $H_0$ does not have eigenvalues embedded into $(-2,2)$. It is an interesting and important problem to identify the perturbation such that the operator $H_0+V$ has one eigenvalue (finitely many eigenvalues or countable eigenvalues) embedded into $(-2,2)$. We introduce the almost sign type potentials and develop the Prüfer transformation to address this problem, which leads to the following five results. 1: We obtain the sharp spectral transition for the existence of irrational type eigenvalues or rational type eigenvalues with even denominators.2: Suppose $\limsup _{n\to \infty } n|V(n)|=a<\infty .$ We obtain a lower/upper bound of $a$ such that $H_0+V$ has one rational type eigenvalue with odd denominator.3: We obtain the asymptotical behavior of embedded eigenvalues around the boundaries of $(-2,2)$.4: Given any finite set of points $\{ E_j\}_{j=1}^N$ in $(-2,2)$ with $0\notin \{ E_j\}_{j=1}^N+\{ E_j\}_{j=1}^N$, we construct the explicit potential $V(n)=\frac{O(1)}{1+|n|}$ such that $H=H_0+V$ has eigenvalues $\{ E_j\}_{j=1}^N$.5: Given any countable set of points $\{ E_j\}$ in $(-2,2)$ with $0\notin \{ E_j\}+\{ E_j\}$, and any function $h(n)>0$ going to infinity arbitrarily slowly, we construct the explicit potential $|V(n)|\leq \frac{h(n)}{1+|n|}$ such that $H=H_0+V$ has eigenvalues $\{ E_j\}$.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1137/1.9781611975994.73
