The weighted nearest neighbors (WNN) estimator has been popularly used as a flexible and easy-to-implement nonparametric tool for mean regression estimation. The bagging technique is an elegant way to form WNN estimators with weights automatically generated to the nearest neighbors (Steele, 2009; Biau et al., 2010); we name the resulting estimator as the distributional nearest neighbors (DNN) for easy reference. Yet, there is a lack of distributional results for such estimator, limiting its application to statistical inference. Moreover, when the mean regression function has higher-order smoothness, DNN does not achieve the optimal nonparametric convergence rate, mainly because of the bias issue. In this work, we provide an in-depth technical analysis of the DNN, based on which we suggest a bias reduction approach for the DNN estimator by linearly combining two DNN estimators with different subsampling scales, resulting in the novel two-scale DNN (TDNN) estimator. The two-scale DNN estimator has an equivalent representation of WNN with weights admitting explicit forms and some being negative. We prove that, thanks to the use of negative weights, the two-scale DNN estimator enjoys the optimal nonparametric rate of convergence in estimating the regression function under the fourth order smoothness condition. We further go beyond estimation and establish that the DNN and two-scale DNN are both asymptotically normal as the subsampling scales and sample size diverge to infinity. For the practical implementation, we also provide variance estimators and a distribution estimator using the jackknife and bootstrap techniques for the two-scale DNN. These estimators can be exploited for constructing valid confidence intervals for nonparametric inference of the regression function. The theoretical results and appealing nite-sample performance of the suggested two-scale DNN method are illustrated with several simulation examples and a real data application.
more »
« less
The bias of isotonic regression
We study the bias of the isotonic regression estimator. While there is extensive work characterizing the mean squared error of the iso- tonic regression estimator, relatively little is known about the bias. In this paper, we provide a sharp characterization, proving that the bias scales as O(n−β/3) up to log factors, where 1 ≤ β ≤ 2 is the exponent correspond- ing to H ̈older smoothness of the underlying mean. Importantly, this result only requires a strictly monotone mean and that the noise distribution has subexponential tails, without relying on symmetric noise or other restrictive assumptions.
more »
« less
- Award ID(s):
- 1811767
- NSF-PAR ID:
- 10164435
- Date Published:
- Journal Name:
- Electronic journal of statistics
- Volume:
- 14
- ISSN:
- 1935-7524
- Page Range / eLocation ID:
- 801-834
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
One of the top priorities in observational astronomy is the direct imaging and characterization of extrasolar planets (exoplanets) and planetary systems. Direct images of rocky exoplanets are of particular interest in the search for life beyond the Earth, but they tend to be rather challenging targets since they are orders-of-magnitude dimmer than their host stars and are separated by small angular distances that are comparable to the classical
diffraction limit, even for the coming generation of 30 m class telescopes. Current and planned efforts for ground-based direct imaging of exoplanets combine high-order adaptive optics (AO) with a stellar coronagraph observing at wavelengths ranging from the visible to the mid-IR. The primary barrier to achieving high contrast with current direct imaging methods is quasi-static speckles, caused largely by non-common path aberrations (NCPAs) in the coronagraph optical train. Recent work has demonstrated that millisecond imaging, which effectively “freezes” the atmosphere’s turbulent phase screens, should allow the wavefront sensor (WFS) telemetry to be used as a probe of the optical system to measure NCPAs. Starting with a realistic model of a telescope with an AO system and a stellar coronagraph, this paper provides simulations of several closely related regression models that take advantage of millisecond telemetry from the WFS and coronagraph’s science camera. The simplest regression model, called the naïve estimator, does not treat the noise and other sources of information loss in the WFS. Despite its flaws, in one of the simulations presented herein, the naïve estimator provides a useful estimate of an NCPA of radian RMS ( ), with an accuracy of radian RMS in 1 min of simulated sky time on a magnitude 8 star. The bias-corrected estimator generalizes the regression model to account for the noise and information loss in the WFS. A simulation of the bias-corrected estimator with 4 min of sky time included an NCPA ofradian RMS ( ) and an extended exoplanet scene. The joint regression of the bias-corrected estimator simultaneously achieved an NCPA estimate with an accuracy of radian RMS and an estimate of the exoplanet scene that was free of the self-subtraction artifacts typically associated with differential imaging. The contrast achieved by imaging of the exoplanet scene was at a distance of from the star and at . These contrast values are comparable to the very best on-sky results obtained from multi-wavelength observations that employ both angular differential imaging (ADI) and spectral differential imaging (SDI). This comparable performance is despite the fact that our simulations are quasi-monochromatic, which makes SDI impossible, nor do they have diurnal field rotation, which makes ADI impossible. The error covariance matrix of the joint regression shows substantial correlations in the exoplanet and NCPA estimation errors, indicating that exoplanet intensity and NCPA need to be estimated self-consistently to achieve high contrast. -
more » « less
The leading difficulty in achieving the contrast necessary to directly image exoplanets and associated structures (e.g., protoplanetary disks) at wavelengths ranging from the visible to the infrared is quasi-static speckles (QSSs). QSSs are hard to distinguish from planets at the necessary level of precision to achieve high contrast. QSSs are the result of hardware aberrations that are not compensated for by the adaptive optics (AO) system; these aberrations are called non-common path aberrations (NCPAs). In 2013, Frazin showed how simultaneous millisecond telemetry from the wavefront sensor (WFS) and a science camera behind a stellar coronagraph can be used as input into a regression scheme that simultaneously and self-consistently estimates NCPAs and the sought-after image of the planetary system (
exoplanet image). When run in a closed-loop configuration, the WFS measures the corrected wavefront, called theAO residual (AOR)wavefront . The physical principle underlying the regression method is rather simple: when an image is formed at the science camera, the AOR modules both the speckles arising from NCPAs as well as the planetary image. Therefore, the AOR can be used as a probe to estimate NCPA and the exoplanet image via regression techniques. The regression approach is made more difficult by the fact that the AOR is not exactly known since it can be estimated only from the WFS telemetry. The simulations in the Part I paper provide results on the joint regression on NCPAs and the exoplanet image from three different methods, calledideal ,naïve , andbias-corrected estimators. The ideal estimator is not physically realizable (it is useful as a benchmark for simulation studies), but the other two are. The ideal estimator uses true AOR values (available in simulation studies), but it treats the noise in focal plane images via standard linearized regression. Naïve regression uses the same regression equations as the ideal estimator, except that it substitutes the estimated values of the AOR for true AOR values in the regression formulas, which can result in problematic biases (however, Part I provides an example in which the naïve estimate makes a useful estimate of NCPAs). The bias-corrected estimator treats the errors in AOR estimates, but it requires the probability distribution that governs the errors in AOR estimates. This paper provides the regression equations for ideal, naïve, and bias-corrected estimators, as well as a supporting technical discussion. -
null (Ed.)Abstract Estimating the mean of a probability distribution using i.i.d. samples is a classical problem in statistics, wherein finite-sample optimal estimators are sought under various distributional assumptions. In this paper, we consider the problem of mean estimation when independent samples are drawn from $d$-dimensional non-identical distributions possessing a common mean. When the distributions are radially symmetric and unimodal, we propose a novel estimator, which is a hybrid of the modal interval, shorth and median estimators and whose performance adapts to the level of heterogeneity in the data. We show that our estimator is near optimal when data are i.i.d. and when the fraction of ‘low-noise’ distributions is as small as $\varOmega \left (\frac{d \log n}{n}\right )$, where $n$ is the number of samples. We also derive minimax lower bounds on the expected error of any estimator that is agnostic to the scales of individual data points. Finally, we extend our theory to linear regression. In both the mean estimation and regression settings, we present computationally feasible versions of our estimators that run in time polynomial in the number of data points.more » « less
-
more » « less
Summary Variance estimation is a fundamental problem in statistical modelling. In ultrahigh dimensional linear regression where the dimensionality is much larger than the sample size, traditional variance estimation techniques are not applicable. Recent advances in variable selection in ultrahigh dimensional linear regression make this problem accessible. One of the major problems in ultrahigh dimensional regression is the high spurious correlation between the unobserved realized noise and some of the predictors. As a result, the realized noises are actually predicted when extra irrelevant variables are selected, leading to a serious underestimate of the level of noise. We propose a two-stage refitted procedure via a data splitting technique, called refitted cross-validation, to attenuate the influence of irrelevant variables with high spurious correlations. Our asymptotic results show that the resulting procedure performs as well as the oracle estimator, which knows in advance the mean regression function. The simulation studies lend further support to our theoretical claims. The naive two-stage estimator and the plug-in one-stage estimators using the lasso and smoothly clipped absolute deviation are also studied and compared. Their performances can be improved by the refitted cross-validation method proposed.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1214/20-EJS1677
