More Like this

1. Six Maxims of Statistical Acumen for Astronomical Data Analysis

   https://doi.org/10.3847/1538-4365/ad8440  

   Tak, Hyungsuk; Chen, Yang; Kashyap, Vinay_L; Mandel, Kaisey_S; Meng, Xiao-Li; Siemiginowska, Aneta; van_Dyk, David_A (November 2024, The Astrophysical Journal Supplement Series)

   Abstract The acquisition of complex astronomical data is accelerating, especially with newer telescopes producing ever more large-scale surveys. The increased quantity, complexity, and variety of astronomical data demand a parallel increase in skill and sophistication in developing, deciding, and deploying statistical methods. Understanding limitations and appreciating nuances in statistical and machine learning methods and the reasoning behind them is essential for improving data-analytic proficiency and acumen. Aiming to facilitate such improvement in astronomy, we delineate cautionary tales in statistics via six maxims, with examples drawn from the astronomical literature. Inspired by the significant quality improvement in business and manufacturing processes by the routine adoption of Six Sigma, we hope the routine reflection on these six maxims will improve the quality of both data analysis and scientific findings in astronomy. 

   more » « less
2. Maximum-likelihood Regression with Systematic Errors for Astronomy and the Physical Sciences. I. Methodology and Goodness-of-fit Statistic of Poisson Data

   https://doi.org/10.3847/1538-4357/ad9b1e  

   Bonamente, Massimiliano; Chen, Yang; Zimmerman, Dale (February 2025, The Astrophysical Journal)

   Abstract This paper presents a new statistical method that enables the use of systematic errors in the maximum-likelihood regression of integer-count Poisson data to a parametric model. The method is primarily aimed at the characterization of the goodness-of-fit statistic in the presence of the over-dispersion that is induced by sources of systematic error, and is based on a quasi-maximum-likelihood method that retains the Poisson distribution of the data. We show that the Poisson deviance, which is the usual goodness-of-fit statistic and that is commonly referred to in astronomy as the Cash statistics, can be easily generalized in the presence of systematic errors, under rather general conditions. The method and the associated statistics are first developed theoretically, and then they are tested with the aid of numerical simulations and further illustrated with real-life data from astronomical observations. The statistical methods presented in this paper are intended as a simple general-purpose framework to include additional sources of uncertainty for the analysis of integer-count data in a variety of practical data analysis situations. 

   more » « less
3. Recent technical and scientific highlights from the CHARA Array

   https://doi.org/10.1117/12.2629048  

   Gies, Douglas R.; Anderson, Matthew D.; Anugu, Narsireddy; ten Brummelaar, Theo A.; Castillo, Victor; Farrington, Christopher D.; Golden, Steven; Jones, Jeremy W.; Klement, Robert; Koehler, Rainer; et al (August 2022, Optical and Infrared Interferometry and Imaging VIII)

   Mérand, Antoine; Sallum, Stephanie; Sanchez-Bermudez, Joel (Ed.)

   The Center for High Angular Resolution Astronomy (CHARA) Array is a six-element interferometer with baselines ranging from 34 to 331 m. Three new beam combiners are entering operation: MYSTIC is a 6-telescope combiner for K-band; SPICA is a 6-telescope combiner for the visible R-band; and SILMARIL is a 3-telescope combiner for high sensitivity in H and K-bands. A seventh, portable telescope will use fiber optics for beam transport and will increase the baselines to 1 km. Observing time is available through a program funded by NSF. The programs are solicited and peer-reviewed by NSF’s National Optical Infrared Astronomy Research Laboratory. The open community access has significantly expanded the range of astronomical investigations of stars and their environments. Here we summarize the scientific work and the on-going technical advances of the CHARA Array. 

   more » « less
4. A Scalable Cloud-Based Analysis Platform for Survey Astronomy

   Stetzler, S; Slater, C; Zecevic, P; Juric, M. (January 2020, Gateways 2020)

   null (Ed.)

   Research in astronomy is undergoing a major paradigm shift, transformed by the advent of large, automated, sky-surveys into a data-rich field where multi-TB to PB-sized spatio-temporal data sets are commonplace. For example the Legacy Survey of Space and Time; LSST) is about to begin delivering observations of >10^10 objects, including a database with >4 x 10^13 rows of time series data. This volume presents a challenge: how should a domain-scientist with little experience in data management or distributed computing access data and perform analyses at PB-scale? We present a possible solution to this problem built on (adapted) industry standard tools and made accessible through web gateways. We have i) developed Astronomy eXtensions for Spark, AXS, a series of astronomy-specific modifications to Apache Spark allowing astronomers to tap into its computational scalability ii) deployed datasets in AXS-queriable format in Amazon S3, leveraging its I/O scalability, iii) developed a deployment of Spark on Kubernetes with auto-scaling configurations requiring no end-user interaction, and iv) provided a Jupyter notebook, web-accessible, front-end via JupyterHub including a rich library of pre-installed common astronomical software (accessible at http://hub.dirac.institute). We use this system to enable the analysis of data from the Zwicky Transient Facility, presently the closest precursor survey to the LSST, and discuss initial results. To our knowledge, this is a first application of cloud-based scalable analytics to astronomical datasets approaching LSST-scale. The code is available at https://github.com/astronomy-commons. 

   more » « less
5. Globally Optimal and Scalable N-way Matching of Astronomy Catalogs

   https://doi.org/10.3847/1538-3881/ac6bf6  

   Nguyen, Tu; Basu, Amitabh; Budavári, Tamás (May 2022, The Astronomical Journal)

   Abstract Building on previous Bayesian approaches, we introduce a novel formulation of probabilistic cross-identification, where detections are directly associated to (hypothesized) astronomical objects in a globally optimal way. We show that this new method scales better for processing multiple catalogs than enumerating all possible candidates, especially in the limit of crowded fields, which is the most challenging observational regime for new-generation astronomy experiments such as the Rubin Observatory Legacy Survey of Space and Time. Here we study simulated catalogs where the ground truth is known and report on the statistical and computational performance of the method. The paper is accompanied by a public software tool to perform globally optimal catalog matching based on directional data. 

   more » « less