More Like this

1. Counterexamples to the Low-Degree Conjecture

   Holmgren, Justin; Wein, Alexander S (April 2021, 12th Innovations in Theoretical Computer Science Conference (ITCS 2021))

   Lee, James R (Ed.)

   A conjecture of Hopkins (2018) posits that for certain high-dimensional hypothesis testing problems, no polynomial-time algorithm can outperform so-called "simple statistics", which are low-degree polynomials in the data. This conjecture formalizes the beliefs surrounding a line of recent work that seeks to understand statistical-versus-computational tradeoffs via the low-degree likelihood ratio. In this work, we refute the conjecture of Hopkins. However, our counterexample crucially exploits the specifics of the noise operator used in the conjecture, and we point out a simple way to modify the conjecture to rule out our counterexample. We also give an example illustrating that (even after the above modification), the symmetry assumption in the conjecture is necessary. These results do not undermine the low-degree framework for computational lower bounds, but rather aim to better understand what class of problems it is applicable to. 

   more » « less
2. Significance Tests of Feature Relevance for a Black-Box Learner

   https://doi.org/10.1109/TNNLS.2022.3185742  

   Dai, Ben; Shen, Xiaotong; Pan, Wei (July 2023, IEEE Transactions on Neural Networks and Learning Systems)

   An exciting recent development is the uptake of deep neural networks in many scientific fields, where the main objective is outcome prediction with a black-box nature. Significance testing is promising to address the black-box issue and explore novel scientific insights and interpretations of the decision-making process based on a deep learning model. However, testing for a neural network poses a challenge because of its black-box nature and unknown limiting distributions of parameter estimates while existing methods require strong assumptions or excessive computation. In this article, we derive one-split and two-split tests relaxing the assumptions and computational complexity of existing black-box tests and extending to examine the significance of a collection of features of interest in a dataset of possibly a complex type, such as an image. The one-split test estimates and evaluates a black-box model based on estimation and inference subsets through sample splitting and data perturbation. The two-split test further splits the inference subset into two but requires no perturbation. Also, we develop their combined versions by aggregating the p-values based on repeated sample splitting. By deflating the bias-sd-ratio, we establish asymptotic null distributions of the test statistics and the consistency in terms of Type II error. Numerically, we demonstrate the utility of the proposed tests on seven simulated examples and six real datasets. Accompanying this article is our python library dnn-inference (https://dnninference.readthedocs.io/en/latest/) that implements the proposed tests. 

   more » « less
3. Over-the-Air Collaborative Learning in Joint Decision Making

   https://doi.org/10.1109/GLOBECOM48099.2022.10001286  

   Feres, Carlos; Levy, Bernard C.; Ding, Zhi (December 2022, 2022 IEEE Global Telecommunications Conference)

   We propose an over-the-air learning framework for collaborative decision making in wireless sensor networks. The low complexity framework leverages low-latency sensor transmission for a decision server to coordinate measurement sensors for hypothesis testing through over-the-air aggregation of sensor data over a multiple-access channel. We formulate several collaborative over-the-air hypothesis testing problems under different practical protocols for collaborative learning and decision making. We develop hypothesis tests for these network protocols and deployment scenarios including channel fading. We provide performance benchmark for both basic likelihood ratio test and generalized likelihood ratio test under different deployment conditions. Our results clearly demonstrate gain provided by increasing number of collaborative sensors. 

   more » « less
4. Over-the-Air Collaborative Learning in Joint Decision Making

   Feres, Carlos; Levy, Bernard C.; Ding, Zhi (January 2022, IEEE Global Communications Conference)

   We propose an over-the-air learning framework for collaborative decision making in wireless sensor networks. The low complexity framework leverages low-latency sensor transmission for a decision server to coordinate measurement sensors for hypothesis testing through over-the-air aggregation of sensor data over a multiple-access channel. We formulate several collaborative over-the-air hypothesis testing problems under different practical protocols for collaborative learning and decision making. We develop hypothesis tests for these network protocols and deployment scenarios including channel fading. We provide performance benchmark for both basic likelihood ratio test and generalized likelihood ratio test under different deployment conditions. Our results clearly demonstrate gain provided by increasing number of collaborative sensors. 

   more » « less
5. Quality Assurance Criteria for Cyclic Direct Simple Shear Tests for Evaluating Liquefaction Triggering Characteristics of Cohesionless Soils

   https://doi.org/10.32075/17ECSMGE-2019-0470  

   Ulmer, K.J. (September 2019, Proceedings of the XVII ECSMGE-2019)

   Undrained or constant volume direct simple shear (CDSS) tests are commonly used to evaluate the liquefaction triggering characteristics of cohesionless soils. However, while the American Society for Testing of Materials (ASTM) has developed standards for monotonic direct simple shear testing, they have not developed a standard for CDSS. As a result, herein the authors review their test database and assign “grades” A-D to different aspects of the tests, e.g.: accumulated shear strain and imposed shear stress on the specimen during the consoli-dation phase, and maximum axial strain that occurs during the cyclic phase of constant volume CDSS testing. Additional grades are also assigned to the tests based on unusual behaviors in the stress paths. Acceptance criteria based on the cumulative test scores are then proposed for “high” quality tests. The slope of the relationship between cyclic stress ratio (CSR) and number of cycles to liquefaction (NL) is influenced by the exclusion of tests using the acceptance criteria, even though the excluded tests were of sufficient quality to have been included in most published studies. 

   more » « less