An official website of the United States government
The sequential multiple assignment randomized trial (SMART) is the gold standard trial design to generate data for the evaluation of multistage treatment regimes. As with conventional (single‐stage) randomized clinical trials, interim monitoring allows early stopping; however, there are few methods for principled interim analysis in SMARTs. Because SMARTs involve multiple stages of treatment, a key challenge is that not all enrolled participants will have progressed through all treatment stages at the time of an interim analysis. Wu et al. (2021) propose basing interim analyses on an estimator for the mean outcome under a given regime that uses data only from participants who have completed all treatment stages. We propose an estimator for the mean outcome under a given regime that gains efficiency by using partial information from enrolled participants regardless of their progression through treatment stages. Using the asymptotic distribution of this estimator, we derive associated Pocock and O'Brien‐Fleming testing procedures for early stopping. In simulation experiments, the estimator controls type I error and achieves nominal power while reducing expected sample size relative to the method of Wu et al. (2021). We present an illustrative application of the proposed estimator based on a recent SMART evaluating behavioral pain interventions for breast cancer patients.
more »
« less
Technical Comment on “Policy impacts of statistical uncertainty and privacy”
Steed et al . ( 1 ) illustrates the crucial impact that the quality of official statistical data products may exert on the accuracy, stability, and equity of policy decisions on which they are based. The authors remind us that data, however responsibly curated, can be fallible. With this comment, we underscore the importance of conducting principled quality assessment of official statistical data products. We observe that the quality assessment procedure employed by Steed et al . needs improvement, due to (i) the inadmissibility of the estimator used, and (ii) the inconsistent probability model it induces on the joint space of the estimator and the observed data. We discuss the design of alternative statistical methods to conduct principled quality assessments for official statistical data products, showcasing two simulation-based methods for admissible minimax shrinkage estimation via multilevel empirical Bayesian modeling. For policymakers and stakeholders to accurately gauge the context-specific usability of data, the assessment should take into account both uncertainty sources inherent to the data and the downstream use cases, such as policy decisions based on those data products.
more »
« less
- PAR ID:
- 10438215
- Date Published:
- Journal Name:
- Science
- Volume:
- 380
- Issue:
- 6648
- ISSN:
- 0036-8075
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We consider the problem of discrete-time signal denoising, focusing on a specific family of non-linear convolution-type estimators. Each such estimator is associated with a time-invariant filter which is obtained adaptively, by solving a certain convex optimization problem. Adaptive convolution-type estimators were demonstrated to have favorable statistical properties, see (Juditsky & Nemirovski, 2009; 2010; Harchaoui et al., 2015b; Ostrovsky et al., 2016). Our first contribution is an efficient implementation of these estimators via the known first-order proximal algorithms. Our second contribution is a computational complexity analysis of the proposed procedures, which takes into account their statistical nature and the related notion of statistical accuracy. The proposed procedures and their analysis are illustrated on a simulated data benchmark.more » « less
-
Domenech, Josep; Vicente, María Rosalía (Ed.)Citizenry decision-making relies on data for informed actions, and official statistics provide many of the relevant data needed for these decisions. However, the wide, distributed, and diverse datasets available from official statistics remain hard to access, scrutinise and manipulate, especially for non-experts. As a result, the complexities involved in official statistical databases create barriers to broader access to these data, often rendering the data non-actionable or irrelevant for the speed at which decisions are made in social and public life. To address this problem, this paper proposes an approach to automatically generating basic, factual questions from an existing dataset of official statistics. The question generating process, now specifically instantiated for geospatial data, starts from a raw dataset and gradually builds toward formulating and presenting users with examples of questions that the dataset can answer, and for which geographic units. This approach exemplifies a novel paradigm of question-first data rendering, where questions, rather than data tables, are used as a human-centred and relevant access points to explore, manipulate, navigate and cross-link data to support decision making. This approach can automate time-consuming aspects of data transformation and facilitate broader access to data.more » « less
-
Abstract. The International Monitoring System (IMS) was established in the late 1990sfor verification of the Comprehensive Nuclear-Test-Ban Treaty (CTBT). Uponcompletion, 60 infrasound stations distributed over the globe will monitorthe Earth's atmosphere for low-frequency pressure waves. In this study, wepresent advanced infrasound data products of the 53 currently certified IMSinfrasound stations for atmospheric studies and civilian applications. Forthis purpose, 18 years of raw IMS infrasound waveform data (2003–2020) were reprocessed using the Progressive Multi-Channel Correlation (PMCC) method. A one-third octave frequency band configuration between 0.01 and 4 Hz was chosen to run this array-processing algorithm which detects coherent infrasound waves within the background noise. From the comprehensive detection lists, four products were derived for each of the certified 53 IMS infrasound stations. The four products cover different frequency ranges and are provided at the following different temporal resolutions: a very low-frequency set (0.02–0.07 Hz, 30 min; https://doi.org/10.25928/bgrseis_bblf-ifsd, Hupe et al., 2021a), two so-called microbarom frequency sets – covering both the lower (0.15–0.35 Hz, 15 min; https://doi.org/10.25928/bgrseis_mblf-ifsd, Hupe et al., 2021b) and a higher (0.45–0.65 Hz, 15 min; https://doi.org/10.25928/bgrseis_mbhf-ifsd, Hupe et al., 2021c) part – named after the dominant ambient noise of interacting ocean waves that are quasi-continuously detected at IMS stations, and observations with center frequencies of 1 to 3 Hz (5 min), called the high-frequency product (https://doi.org/10.25928/bgrseis_bbhf-ifsd, Hupe et al., 2021d). Within these frequency ranges and time windows, the dominant repetitive signal directions are summarized. Along with several detection parameters, calculated quantities for assessing the relative quality of the products are provided. The validity of the data products is demonstrated through example case studies of recent events that produced infrasound detected at IMS infrasound stations and through a global assessment and summary of the products. The four infrasound data products cover globally repeating infrasound sources such as ocean ambient noise or persistently active volcanoes, which have previously been suggested as sources for probing the winds in the middle atmosphere. Therefore, our infrasound data products open up the IMS observations also to user groups who do not have unconstrained access to IMS data or who are unfamiliar with infrasound data processing using the PMCC method. These types of data products could potentially serve as a basis for volcanic eruption early warning systems in the future.more » « less
-
Abstract In response to Li, Reigh, He, and Miller's commentary,Can we and should we use artificial intelligence for formative assessment in science, we argue that artificial intelligence (AI) is already being widely employed in formative assessment across various educational contexts. While agreeing with Li et al.'s call for further studies on equity issues related to AI, we emphasize the need for science educators to adapt to the AI revolution that has outpaced the research community. We challenge the somewhat restrictive view of formative assessment presented by Li et al., highlighting the significant contributions of AI in providing formative feedback to students, assisting teachers in assessment practices, and aiding in instructional decisions. We contend that AI‐generated scores should not be equated with the entirety of formative assessment practice; no single assessment tool can capture all aspects of student thinking and backgrounds. We address concerns raised by Li et al. regarding AI bias and emphasize the importance of empirical testing and evidence‐based arguments in referring to bias. We assert that AI‐based formative assessment does not necessarily lead to inequity and can, in fact, contribute to more equitable educational experiences. Furthermore, we discuss how AI can facilitate the diversification of representational modalities in assessment practices and highlight the potential benefits of AI in saving teachers’ time and providing them with valuable assessment information. We call for a shift in perspective, from viewing AI as a problem to be solved to recognizing its potential as a collaborative tool in education. We emphasize the need for future research to focus on the effective integration of AI in classrooms, teacher education, and the development of AI systems that can adapt to diverse teaching and learning contexts. We conclude by underlining the importance of addressing AI bias, understanding its implications, and developing guidelines for best practices in AI‐based formative assessment.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1126/science.adf9724
