skip to main content

Attention:

The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, October 10 until 2:00 AM ET on Friday, October 11 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the NSF-PAR

Title: Statistical Significance: Reliability of P-Values Compared to Other Statistical Summaries
Statistical inference has strongly relied on the use of p-values to draw conclusions. For over a decade this reliance on the p-value has been questioned by researches and academics. The question of whether p-values are truly the best standard, and what other possible statistics could replace p-values l has been discussed deeply. We set out to understand the amount of variation within p-values, and to find if they really are as reliable as the frequency of their use would suggest. To answer this question, we studied a set of clinical trials over the past two years. We also aim to describe the variety of information included in drag labels, and determine whether this information conforms to FDA guidelines. We found a large variation in the presentation of clinical trial data, much of which was not in line with the guidelines of the FDA. Our findings also show that among the clinical trials we studied there is more variation among the p-values than among the estimates. From this, we can conclude that the estimates from clinical trials should hold a heavy weight in the decision of whether or not to approve the drug. This finding suggests that there is validity to the skepticism of the reliance on p-values, and that further studies need to be done to find a new, more reliable, standard in statistical inference.  more » « less
Award ID(s):
1712839
NSF-PAR ID:
10170222
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Current trends on biostatistics biometrics
Volume:
2
Issue:
1
ISSN:
2644-1381
Page Range / eLocation ID:
171-175
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ( , Journal of The Royal Society Interface)

    A quantitative analysis of human gait patterns in space–time provides an opportunity to observe variability within and across individuals of varying motor capabilities. Impaired gait significantly affects independence and quality of life, and thus a large part of clinical research is dedicated to improving gait through rehabilitative therapies. Evaluation of these paradigms relies on understanding the characteristic differences in the kinematics and underlying biomechanics of impaired and unimpaired locomotion, which has motivated quantitative measurement and analysis of the gait cycle. Previous analysis has largely been limited to a statistical comparison of manually selected pointwise metrics identified through expert knowledge. Here, we use a recent statistical-geometric framework, elastic functional data analysis (FDA), to decompose kinematic data into continuous ‘amplitude’ (spatial) and ‘phase’ (temporal) components, which can then be integrated with established dimensionality reduction techniques. We demonstrate the utility of elastic FDA through two unsupervised applications to post-stroke gait datasets. First, we distinguish between unimpaired, paretic and non-paretic gait presentations. Then, we use FDA to reveal robust, interpretable groups of differential response to exosuit assistance. The proposed methods aim to benefit clinical practice for post-stroke gait rehabilitation, and more broadly, to automate the quantitative analysis of motion.

     
    more » « less
  2. ; ; ; ( , Biometrics)
    Summary

    Cluster analysis has proved to be an invaluable tool for the exploratory and unsupervised analysis of high-dimensional datasets. Among methods for clustering, hierarchical approaches have enjoyed substantial popularity in genomics and other fields for their ability to simultaneously uncover multiple layers of clustering structure. A critical and challenging question in cluster analysis is whether the identified clusters represent important underlying structure or are artifacts of natural sampling variation. Few approaches have been proposed for addressing this problem in the context of hierarchical clustering, for which the problem is further complicated by the natural tree structure of the partition, and the multiplicity of tests required to parse the layers of nested clusters. In this article, we propose a Monte Carlo based approach for testing statistical significance in hierarchical clustering which addresses these issues. The approach is implemented as a sequential testing procedure guaranteeing control of the family-wise error rate. Theoretical justification is provided for our approach, and its power to detect true clustering structure is illustrated through several simulation studies and applications to two cancer gene expression datasets.

     
    more » « less
  3. ; ( , Scientific Reports)
    null (Ed.)
    Abstract In this study, we propose to use machine learning to understand terminated clinical trials. Our goal is to answer two fundamental questions: (1) what are common factors/markers associated to terminated clinical trials? and (2) how to accurately predict whether a clinical trial may be terminated or not? The answer to the first question provides effective ways to understand characteristics of terminated trials for stakeholders to better plan their trials; and the answer to the second question can direct estimate the chance of success of a clinical trial in order to minimize costs. By using 311,260 trials to build a testbed with 68,999 samples, we use feature engineering to create 640 features, reflecting clinical trial administration, eligibility, study information, criteria etc. Using feature ranking, a handful of features, such as trial eligibility, trial inclusion/exclusion criteria, sponsor types etc. , are found to be related to the clinical trial termination. By using sampling and ensemble learning, we achieve over 67% Balanced Accuracy and over 0.73 AUC (Area Under the Curve) scores to correctly predict clinical trial termination, indicating that machine learning can help achieve satisfactory prediction results for clinical trial study. 
    more » « less
  4. ; ; ( , Statistics in Medicine)

    Covariate‐adaptive randomization (CAR) procedures have been developed in clinical trials to mitigate the imbalance of treatments among covariates. In recent years, an increasing number of trials have started to use CAR for the advantages in statistical efficiency and enhancing credibility. At the same time, sample size re‐estimation (SSR) has become a common technique in industry to reduce time and cost while maintaining a good probability of success. Despite the widespread popularity of combining CAR designs with SSR, few researchers have investigated this combination theoretically. More importantly, the existing statistical inference must be adjusted to protect the desired type I error rate when a model that omits some covariates is used. In this article, we give a framework for the application of SSR in CAR trials and study the underlying theoretical properties. We give the adjusted test statistic and derive the sample size calculation formula under the CAR setting. We can tackle the difficulties caused by the adaptive features in CAR and prove the asymptotic independence between stages. Numerical studies are conducted under multiple parameter settings and scenarios that are commonly encountered in practice. The results show that all advantages of CAR and SSR can be preserved and further improved in terms of power and sample size.

     
    more » « less
  5. ; ; ; ; ; ( , Malaria Journal)
    Abstract Background

    Plasmodium vivax blood-stage relapses originating from re-activating hypnozoites are a major barrier for control and elimination of this disease. Radical cure is a form of therapy capable of addressing this problem. Recent clinical trials of radical cure have yielded efficacy estimates ranging from 65 to 94%, with substantial variation across trial sites.

     Methods

    An analysis of simulated trial data using a transmission model was performed to demonstrate that variation in efficacy estimates across trial sites can arise from differences in the conditions under which trials are conducted.

     Results

    The analysis revealed that differences in transmission intensity, heterogeneous exposure and relapse rate can yield efficacy estimates ranging as widely as 12–78%, despite simulating trial data under the uniform assumption that treatment had a 75% chance of clearing hypnozoites. A longer duration of prophylaxis leads to a greater measured efficacy, particularly at higher transmission intensities, making the comparison between the protection of different radical cure treatment regimens against relapse more challenging. Simulations show that vector control and parasite genotyping offer two potential means to yield more standardized efficacy estimates that better reflect prevention of relapse.

     Conclusions

    Site-specific biases are likely to contribute to variation in efficacy estimates both within and across clinical trials. Future clinical trials can reduce site-specific biases by conducting trials in low-transmission settings where re-infections from mosquito bite are less common, by preventing re-infections using vector control measures, or by identifying and excluding likely re-infections that occur during follow-up, by using parasite genotyping methods.

     
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email