Search for: All records

Creators/Authors contains: "Ma, Cong"

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. Free, publicly-accessible full text available July 30, 2027
  2. Abstract Statistical learning under distribution shift is challenging when neither prior knowledge nor data from the target distribution is available. Distributionally robust learning (DRL) aims to control the worst-case statistical performance within a set of candidate distributions, but how to properly specify the set remains challenging. To enable distributional robustness without being overly conservative, in this paper we propose a shape-constrained approach to DRL, which incorporates prior information about the way in which the unknown target distribution differs from its estimate—specifically, we assume the unknown density ratio between the target distribution and its estimate is isotonic with respect to some partial order. At the population level, we provide a solution to the shape-constrained optimization problem that can be solved without the challenge of an explicit isotonic constraint. At the sample level, we provide consistency results for an empirical estimator of the target in a range of different settings. Empirical studies on both synthetic and real data demonstrate the improved efficiency of the proposed shape-constrained approach. 
    more » « less
    Free, publicly-accessible full text available July 6, 2027
  3. Free, publicly-accessible full text available May 20, 2027
  4. Free, publicly-accessible full text available April 3, 2027
  5. Free, publicly-accessible full text available April 10, 2027
  6. Abstract We propose $$\mathsf{ScaledGD(\lambda )}$$, a preconditioned gradient descent ($$\mathsf{GD}$$) method to tackle the low-rank matrix sensing problem when the true rank is unknown, and when the matrix is possibly ill-conditioned. Using overparameterized factor representations, $$\mathsf{ScaledGD(\lambda )}$$ starts from a small random initialization, and proceeds by GD with a specific form of damped preconditioning to combat bad curvatures induced by overparameterization and ill-conditioning. $$\mathsf{ScaledGD(\lambda )}$$ is remarkably robust to ill-conditioning compared to vanilla $$\mathsf{GD}$$ even with overparameterization. Specifically, we show that, under the restricted isometry property of the sensing operator, $$\mathsf{ScaledGD(\lambda )}$$ converges to the true low-rank matrix at a constant linear rate after a small number of iterations that scales only logarithmically with respect to the condition number and the problem dimension. This significantly improves over the convergence rate of vanilla $$\mathsf{GD}$$ which suffers from a polynomial dependency on the condition number. Furthermore, we show that in the presence of measurement noise, $$\mathsf{ScaledGD(\lambda )}$$ converges to the minimax optimal error up to a multiplicative factor of the condition number at the same rate as in the noiseless setting, which is the first nearly minimax-optimal overparameterized gradient method for low-rank matrix sensing scaling with the true rank rather than the (possibly much larger) overparameterized rank. Our results also extend to the setting when the matrix is only approximately low-rank under the Gaussian design. Our work provides evidence on the power of preconditioning in accelerating the convergence without hurting generalization in overparameterized learning. 
    more » « less
    Free, publicly-accessible full text available February 19, 2027
  7. Free, publicly-accessible full text available December 10, 2026