skip to main content


Title: How to Obtain the Redshift Distribution from Probabilistic Redshift Estimates
Abstract

A reliable estimate of the redshift distributionn(z) is crucial for using weak gravitational lensing and large-scale structures of galaxy catalogs to study cosmology. Spectroscopic redshifts for the dim and numerous galaxies of next-generation weak-lensing surveys are expected to be unavailable, making photometric redshift (photo-z) probability density functions (PDFs) the next best alternative for comprehensively encapsulating the nontrivial systematics affecting photo-zpoint estimation. The established stacked estimator ofn(z) avoids reducing photo-zPDFs to point estimates but yields a systematically biased estimate ofn(z) that worsens with a decreasing signal-to-noise ratio, the very regime where photo-zPDFs are most necessary. We introduce Cosmological Hierarchical Inference with Probabilistic Photometric Redshifts (CHIPPR), a statistically rigorous probabilistic graphical model of redshift-dependent photometry that correctly propagates the redshift uncertainty information beyond the best-fit estimator ofn(z) produced by traditional procedures and is provably the only self-consistent way to recovern(z) from photo-zPDFs. We present thechipprprototype code, noting that the mathematically justifiable approach incurs computational cost. TheCHIPPRapproach is applicable to any one-point statistic of any random variable, provided the prior probability density used to produce the posteriors is explicitly known; if the prior is implicit, as may be the case for popular photo-ztechniques, then the resulting posterior PDFs cannot be used for scientific inference. We therefore recommend that the photo-zcommunity focus on developing methodologies that enable the recovery of photo-zlikelihoods with support over all redshifts, either directly or via a known prior probability density.

 
more » « less
NSF-PAR ID:
10364681
Author(s) / Creator(s):
;
Publisher / Repository:
DOI PREFIX: 10.3847
Date Published:
Journal Name:
The Astrophysical Journal
Volume:
928
Issue:
2
ISSN:
0004-637X
Format(s):
Medium: X Size: Article No. 127
Size(s):
["Article No. 127"]
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Upcoming photometric surveys will discover tens of thousands of Type Ia supernovae (SNe Ia), vastly outpacing the capacity of our spectroscopic resources. In order to maximize the scientific return of these observations in the absence of spectroscopic information, we must accurately extract key parameters, such as SN redshifts, with photometric information alone. We present Photo-zSNthesis, a convolutional neural network-based method for predicting full redshift probability distributions from multi-band supernova lightcurves, tested on both simulated Sloan Digital Sky Survey (SDSS) and Vera C. Rubin Legacy Survey of Space and Time data as well as observed SDSS SNe. We show major improvements over predictions from existing methods on both simulations and real observations as well as minimal redshift-dependent bias, which is a challenge due to selection effects, e.g., Malmquist bias. Specifically, we show a 61× improvement in prediction bias 〈Δz〉 on PLAsTiCC simulations and 5× improvement on real SDSS data compared to results from a widely used photometric redshift estimator, LCFIT+Z. The PDFs produced by this method are well constrained and will maximize the cosmological constraining power of photometric SNe Ia samples.

     
    more » « less
  2. null (Ed.)
    ABSTRACT Photometric galaxy surveys constitute a powerful cosmological probe but rely on the accurate characterization of their redshift distributions using only broad-band imaging, and can be very sensitive to incomplete or biased priors used for redshift calibration. A hierarchical Bayesian model has recently been developed to estimate those from the robust combination of prior information, photometry of single galaxies, and the information contained in the galaxy clustering against a well-characterized tracer population. In this work, we extend the method so that it can be applied to real data, developing some necessary new extensions to it, especially in the treatment of galaxy clustering information, and we test it on realistic simulations. After marginalizing over the mapping between the clustering estimator and the actual density distribution of the sample galaxies, and using prior information from a small patch of the survey, we find the incorporation of clustering information with photo-z’s tightens the redshift posteriors and overcomes biases in the prior that mimic those happening in spectroscopic samples. The method presented here uses all the information at hand to reduce prior biases and incompleteness. Even in cases where we artificially bias the spectroscopic sample to induce a shift in mean redshift of $\Delta \bar{z} \approx 0.05,$ the final biases in the posterior are $\Delta \bar{z} \lesssim 0.003.$ This robustness to flaws in the redshift prior or training samples would constitute a milestone for the control of redshift systematic uncertainties in future weak lensing analyses. 
    more » « less
  3. null (Ed.)
    ABSTRACT Determining the distribution of redshifts of galaxies observed by wide-field photometric experiments like the Dark Energy Survey (DES) is an essential component to mapping the matter density field with gravitational lensing. In this work we describe the methods used to assign individual weak lensing source galaxies from the DES Year 3 Weak Lensing Source Catalogue to four tomographic bins and to estimate the redshift distributions in these bins. As the first application of these methods to data, we validate that the assumptions made apply to the DES Y3 weak lensing source galaxies and develop a full treatment of systematic uncertainties. Our method consists of combining information from three independent likelihood functions: self-organizing map p(z) (sompz), a method for constraining redshifts from galaxy photometry; clustering redshifts (WZ), constraints on redshifts from cross-correlations of galaxy density functions; and shear ratios (SRs), which provide constraints on redshifts from the ratios of the galaxy-shear correlation functions at small scales. Finally, we describe how these independent probes are combined to yield an ensemble of redshift distributions encapsulating our full uncertainty. We calibrate redshifts with combined effective uncertainties of σ〈z〉 ∼ 0.01 on the mean redshift in each tomographic bin. 
    more » « less
  4. ABSTRACT We present the calibration of the Dark Energy Survey Year 3 (DES Y3) weak lensing (WL) source galaxy redshift distributions n(z) from clustering measurements. In particular, we cross-correlate the WL source galaxies sample with redMaGiC galaxies (luminous red galaxies with secure photometric redshifts) and a spectroscopic sample from BOSS/eBOSS to estimate the redshift distribution of the DES sources sample. Two distinct methods for using the clustering statistics are described. The first uses the clustering information independently to estimate the mean redshift of the source galaxies within a redshift window, as done in the DES Y1 analysis. The second method establishes a likelihood of the clustering data as a function of n(z), which can be incorporated into schemes for generating samples of n(z) subject to combined clustering and photometric constraints. Both methods incorporate marginalization over various astrophysical systematics, including magnification and redshift-dependent galaxy-matter bias. We characterize the uncertainties of the methods in simulations; the first method recovers the mean z of tomographic bins to RMS (precision) of ∼0.014. Use of the second method is shown to vastly improve the accuracy of the shape of n(z) derived from photometric data. The two methods are then applied to the DES Y3 data. 
    more » « less
  5. Abstract The accurate estimation of photometric redshifts is crucial to many upcoming galaxy surveys, for example, the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). Almost all Rubin extragalactic and cosmological science requires accurate and precise calculation of photometric redshifts; many diverse approaches to this problem are currently in the process of being developed, validated, and tested. In this work, we use the photometric redshift code GPz to examine two realistically complex training set imperfections scenarios for machine learning based photometric redshift calculation: (i) where the spectroscopic training set has a very different distribution in color–magnitude space to the test set, and (ii) where the effect of emission line confusion causes a fraction of the training spectroscopic sample to not have the true redshift. By evaluating the sensitivity of GPz to a range of increasingly severe imperfections, with a range of metrics (both of photo- z point estimates as well as posterior probability distribution functions, PDFs), we quantify the degree to which predictions get worse with higher degrees of degradation. In particular, we find that there is a substantial drop-off in photo- z quality when line-confusion goes above ∼1%, and sample incompleteness below a redshift of 1.5, for an experimental setup using data from the Buzzard Flock synthetic sky catalogs. 
    more » « less