Search for: test

Fentanyl test strips (FTS) are a commonly deployed tool in drug checking, used to test for the presence of fentanyl in street drug samples prior to consumption. Previous reports indicate that in addition to fentanyl, FTS can also detect fentanyl analogs like acetyl fentanyl and butyryl fentanyl, with conflicting reports on their ability to detect fentanyl analogs like Carfentanil and furanyl fentanyl. Yet with hundreds of known fentanyl analogs, there has been no large-scale study rationalizing FTS reactivity to different fentanyl analogs.

In this study, 251 synthetic opioids—including 214 fentanyl analogs—were screened on two brands of fentanyl test strips to (1) assess the differences in the ability of two brands of fentanyl test strips to detect fentanyl-related compounds and (2) determine which moieties in fentanyl analog chemical structures are most crucial for FTS detection. Two FTS brands were assessed in this study: BTNX Rapid Response and WHPM DanceSafe.

Of 251 screened compounds assessed, 121 compounds were detectable at or below 20,000 ng/mL by both BTNX and DanceSafe FTS, 50 were not detectable by either brand, and 80 were detectable by one brand but not the other (n = 52 BTNX,n = 28 DanceSafe). A structural analysis of fentanyl analogs screened revealed that in general, bulky modifications to the phenethyl moiety inhibit detection by BTNX FTS while bulky modifications to the carbonyl moiety inhibit detection by DanceSafe FTS.

The different “blind spots” are caused by different haptens used to elicit the antibodies for these different strips. By utilizing both brands of FTS in routine drug checking, users could increase the chances of detecting fentanyl analogs in the “blind spot” of one brand.

 

In metagenomics, the study of environmentally associated microbial communities from their sampled DNA, one of the most fundamental computational tasks is that of determining which genomes from a reference database are present or absent in a given sample metagenome. Existing tools generally return point estimates, with no associated confidence or uncertainty associated with it. This has led to practitioners experiencing difficulty when interpreting the results from these tools, particularly for low-abundance organisms as these often reside in the “noisy tail” of incorrect predictions. Furthermore, few tools account for the fact that reference databases are often incomplete and rarely, if ever, contain exact replicas of genomes present in an environmentally derived metagenome.

We present solutions for these issues by introducing the algorithm YACHT: Yes/No Answers to Community membership via Hypothesis Testing. This approach introduces a statistical framework that accounts for sequence divergence between the reference and sample genomes, in terms of ANI, as well as incomplete sequencing depth, thus providing a hypothesis test for determining the presence or absence of a reference genome in a sample. After introducing our approach, we quantify its statistical power and how this changes with varying parameters. Subsequently, we perform extensive experiments using both simulated and real data to confirm the accuracy and scalability of this approach.

The source code implementing this approach is available via Conda and at https://github.com/KoslickiLab/YACHT. We also provide the code for reproducing experiments at https://github.com/KoslickiLab/YACHT-reproducibles.

 

An important problem in evolutionary genomics is to investigate whether a certain trait measured on each sample is associated with the sample phylogenetic tree. The phylogenetic tree represents the shared evolutionary history of the samples and it is usually estimated from molecular sequence data at a locus or from other type of genetic data. We propose a model for trait evolution inspired by the Chinese Restaurant Process that includes a parameter that controls the degree of preferential attachment, that is, the tendency of nodes in the tree to subtend from nodes of the same type. This model with no preferential attachment is equivalent to a structured coalescent model with simultaneous migration and coalescence events and serves as a null model. We derive a test for phylogenetic binary trait association with linear computational complexity and empirically demonstrate that it is more powerful than some other methods. We apply our test to study the phylogenetic association of some traits in swordtail fish, breast cancer, yellow fever virus, and influenza A H1N1 virus. R-package implementation of our methods is available at https://github.com/jyzhang27/CRPTree.

 

Performance of classifiers is often measured in terms of average accuracy on test data. Despite being a standard measure, average accuracy fails in characterising the fit of the model to the underlying conditional law of labels given the features vector (Y∣X), e.g. due to model misspecification, over fitting, and high-dimensionality. In this paper, we consider the fundamental problem of assessing the goodness-of-fit for a general binary classifier. Our framework does not make any parametric assumption on the conditional law Y∣X and treats that as a black-box oracle model which can be accessed only through queries. We formulate the goodness-of-fit assessment problem as a tolerance hypothesis testing of the form H0:E[Df(Bern(η(X))‖Bern(η^(X)))]≤τ where Df represents an f-divergence function, and η(x), η^(x), respectively, denote the true and an estimate likelihood for a feature vector x admitting a positive label. We propose a novel test, called Goodness-of-fit with Randomisation and Scoring Procedure (GRASP) for testing H0, which works in finite sample settings, no matter the features (distribution-free). We also propose model-X GRASP designed for model-X settings where the joint distribution of the features vector is known. Model-X GRASP uses this distributional information to achieve better power. We evaluate the performance of our tests through extensive numerical experiments.

 

Genome-wide Association Studies (GWAS) methods have identified individual single-nucleotide polymorphisms (SNPs) significantly associated with specific phenotypes. Nonetheless, many complex diseases are polygenic and are controlled by multiple genetic variants that are usually non-linearly dependent. These genetic variants are marginally less effective and remain undetected in GWAS analysis. Kernel-based tests (KBT), which evaluate the joint effect of a group of genetic variants, are therefore critical for complex disease analysis. However, choosing different kernel functions in KBT can significantly influence the type I error control and power, and selecting the optimal kernel remains a statistically challenging task. A few existing methods suffer from inflated type 1 errors, limited scalability, inferior power or issues of ambiguous conclusions. Here, we present a new Bayesian framework, BayesKAT (https://github.com/wangjr03/BayesKAT), which overcomes these kernel specification issues by selecting the optimal composite kernel adaptively from the data while testing genetic associations simultaneously. Furthermore, BayesKAT implements a scalable computational strategy to boost its applicability, especially for high-dimensional cases where other methods become less effective. Based on a series of performance comparisons using both simulated and real large-scale genetics data, BayesKAT outperforms the available methods in detecting complex group-level associations and controlling type I errors simultaneously. Applied on a variety of groups of functionally related genetic variants based on biological pathways, co-expression gene modules and protein complexes, BayesKAT deciphers the complex genetic basis and provides mechanistic insights into human diseases.

 

Validity is a fundamental consideration of test development and test evaluation. The purpose of this study is to define and reify three key aspects of validity and validation, namely test‐score interpretation, test‐score use, and the claims supporting interpretation and use. This study employed a Delphi methodology to explore how experts in validity and validation conceptualize test‐score interpretation, use, and claims. Definitions were developed through multiple iterations of data collection and analysis. By clarifying the language used when conducting validation, validation may be more accessible to a broader audience, including but not limited to test developers, test users, and test consumers.

 

We propose an accurate and efficient machine learning approach for monitoring particle detectors in real-time. The goal is to assess the compatibility of incoming experimental data with a reference dataset, characterising the data behaviour under normal circumstances, via a likelihood-ratio hypothesis test. The model is based on a modern implementation of kernel methods, nonparametric algorithms that can learn any continuous function given enough data. The resulting approach is efficient and agnostic to the type of anomaly that may be present in the data. Our study demonstrates the effectiveness of this strategy on multivariate data from drift tube chamber muon detectors.

 

Two-sample tests are important areas aiming to determine whether two collections of observations follow the same distribution or not. We propose two-sample tests based on integral probability metric (IPM) for high-dimensional samples supported on a low-dimensional manifold. We characterize the properties of proposed tests with respect to the number of samples $n$ and the structure of the manifold with intrinsic dimension $d$. When an atlas is given, we propose a two-step test to identify the difference between general distributions, which achieves the type-II risk in the order of $n^{-1/\max \{d,2\}}$. When an atlas is not given, we propose Hölder IPM test that applies for data distributions with $(s,\beta )$-Hölder densities, which achieves the type-II risk in the order of $n^{-(s+\beta )/d}$. To mitigate the heavy computation burden of evaluating the Hölder IPM, we approximate the Hölder function class using neural networks. Based on the approximation theory of neural networks, we show that the neural network IPM test has the type-II risk in the order of $n^{-(s+\beta )/d}$, which is in the same order of the type-II risk as the Hölder IPM test. Our proposed tests are adaptive to low-dimensional geometric structure because their performance crucially depends on the intrinsic dimension instead of the data dimension.

 

Undergraduates (n = 132) learned about the human respiratory system and then taught what they learned by explaining aloud on video. Following a 2 × 2 design, students either generated their own words or visuals on paper while explaining aloud, or they viewed instructor‐provided words or visuals while explaining aloud. One week after teaching, students completed explanation, drawing, and transfer tests. Teaching with provided or generated visualizations resulted in significantly higher transfer test performance than teaching with provided or generated words. Furthermore, teaching with provided visuals led to significantly higher drawing test performance than teaching with generated visuals. Finally, the number of elaborations in students' explanations during teaching did not significantly differ across groups but was significantly associated with subsequent explanation and transfer test performance. Overall, the findings partially support the hypothesis that visuals facilitate learning by explaining, yet the benefits appeared stronger for instructor‐provided visuals than learner‐generated drawings.

 

We present a generic framework for creating differentially private versions of any hypothesis test in a black-box way. We analyze the resulting tests analytically and experimentally. Most crucially, we show good practical performance for small data sets, showing that at ϵ = 1 we only need 5-6 times as much data as in the fully public setting. We compare our work to the one existing framework of this type, as well as to several individually-designed private hypothesis tests. Our framework is higher power than other generic solutions and at least competitive with (and often better than) individually-designed tests.