More Like this

1. A Manifold View of Adversarial Risk

   Zhang, Wenjia; Zhang, Yikai; Hu, Xiaoling; Goswami, Mayank; Chen, Chao; Metaxas, Dimitris N. (April 2022, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics)

   The adversarial risk of a machine learning model has been widely studied. Most previous works assume that the data lies in the whole ambient space. We propose to take a new angle and take the manifold assumption into consideration. Assuming data lies in a manifold, we investigate two new types of adversarial risk, the normal adversarial risk due to perturbation along normal direction, and the in-manifold adversarial risk due to perturbation within the manifold. We prove that the classic adversarial risk can be bounded from both sides using the normal and in-manifold adversarial risks. We also show with a surprisingly pessimistic case that the standard adversarial risk can be nonzero even when both normal and in-manifold risks are zero. We finalize the paper with empirical studies supporting our theoretical results. Our results suggest the possibility of improving the robustness of a classifier by only focusing on the normal adversarial risk. 

   more » « less
2. Manifold-driven decomposition for adversarial robustness

   https://doi.org/10.3389/fcomp.2023.1274695  

   Zhang, Wenjia; Zhang, Yikai; Hu, Xiaoling; Yao, Yi; Goswami, Mayank; Chen, Chao; Metaxas, Dimitris (January 2024, Frontiers in Computer Science)

   The adversarial risk of a machine learning model has been widely studied. Most previous studies assume that the data lie in the whole ambient space. We propose to take a new angle and take the manifold assumption into consideration. Assuming data lie in a manifold, we investigate two new types of adversarial risk, the normal adversarial risk due to perturbation along normal direction and the in-manifold adversarial risk due to perturbation within the manifold. We prove that the classic adversarial risk can be bounded from both sides using the normal and in-manifold adversarial risks. We also show a surprisingly pessimistic case that the standard adversarial risk can be non-zero even when both normal and in-manifold adversarial risks are zero. We finalize the study with empirical studies supporting our theoretical results. Our results suggest the possibility of improving the robustness of a classifier without sacrificing model accuracy, by only focusing on the normal adversarial risk. 

   more » « less
3. Dual Manifold Adversarial Robustness: Defense against Lp and non-Lp Adversarial Attacks

   Lin, Wei-An; Pong Lau, Chun; Levine, Alexander; Chellappa, Rama; Feizi, Soheil (January 2020, Advances in Neural Information Processing Systems Foundation (NeurIPS))

   null (Ed.)

   Adversarial training is a popular defense strategy against attack threat models with bounded Lp norms. However, it often degrades the model performance on normal images and the defense does not generalize well to novel attacks. Given the success of deep generative models such as GANs and VAEs in characterizing the underlying manifold of images, we investigate whether or not the aforementioned problems can be remedied by exploiting the underlying manifold information. To this end, we construct an "On-Manifold ImageNet" (OM-ImageNet) dataset by projecting the ImageNet samples onto the manifold learned by StyleGSN. For this dataset, the underlying manifold information is exact. Using OM-ImageNet, we first show that adversarial training in the latent space of images improves both standard accuracy and robustness to on-manifold attacks. However, since no out-of-manifold perturbations are realized, the defense can be broken by Lp adversarial attacks. We further propose Dual Manifold Adversarial Training (DMAT) where adversarial perturbations in both latent and image spaces are used in robustifying the model. Our DMAT improves performance on normal images, and achieves comparable robustness to the standard adversarial training against Lp attacks. In addition, we observe that models defended by DMAT achieve improved robustness against novel attacks which manipulate images by global color shifts or various types of image filtering. Interestingly, similar improvements are also achieved when the defended models are tested on out-of-manifold natural images. These results demonstrate the potential benefits of using manifold information in enhancing robustness of deep learning models against various types of novel adversarial attacks. 

   more » « less
4. Algorithms for Contractibility of Compressed Curves on 3-Manifold Boundaries

   https://doi.org/10.1007/s00454-022-00411-x  

   Chambers, Erin Wolf; Lazarus, Francis; de Mesmay, Arnaud; Parsa, Salman (September 2023, Discrete & Computational Geometry)

   In this paper we prove that the problem of deciding contractibility of an arbitrary closed curve on the boundary of a 3-manifold is in NP. We emphasize that the manifold and the curve are both inputs to the problem. Moreover, our algorithm also works if the curve is given as a compressed word. Previously, such an algorithm was known for simple (non-compressed) curves, and, in very limited cases, for curves with self-intersections. Furthermore, our algorithm is fixed-parameter tractable in the size of the input 3-manifold. As part of our proof, we obtain new polynomial-time algorithms for compressed curves on surfaces, which we believe are of independent interest. We provide a polynomial-time algorithm which, given an orientable surface and a compressed loop on the surface, computes a canonical form for the loop as a compressed word. In particular, contractibility of compressed curves on surfaces can be decided in polynomial time; prior published work considered only constant genus surfaces. More generally, we solve the following normal subgroup membership problem in polynomial time: given an arbitrary orientable surface, a compressed closed curve, and a collection of disjoint normal curves, there is a polynomial-time algorithm to decide if the curve lies in the normal subgroup generated by components of the normal curves in the fundamental group of the surface after attaching the curves to a basepoint. 

   more » « less
5. Tail associations in ecological variables and their impact on extinction risk

   https://doi.org/10.1002/ecs2.3132  

   Ghosh, Shyamolina; Sheppard, Lawrence W.; Reuman, Daniel C. (May 2020, Ecosphere)

   Abstract Extreme climatic events (ECEs) are becoming more frequent and more intense due to climate change. Furthermore, there is reason to believe ECEs may modify tail associations between distinct population vital rates, or between values of an environmental variable measured in different locations. Tail associations between two variables are associations that occur between values in the left or right tails of the distributions of the variables. Two positively associated variables can be principally left‐tail associated (i.e., more correlated when they take low values than when they take high values) or right‐tail associated (more correlated when they take high than low values), even with the same overall correlation coefficient in both cases. We tested, in the context of non‐spatial stage‐structured matrix models, whether tail associations between stage‐specific vital rates may influence extinction risk. We also tested whether the nature of spatial tail associations of environmental variables can influence metapopulation extinction risk. For instance, if low values of an environmental variable reduce the growth rates of local populations, one may expect that left‐tail associations increase metapopulation extinction risks because then environmental catastrophes are spatially synchronized, presumably reducing the potential for rescue effects. For the non‐spatial, stage‐structured models we considered, left‐tail associations between vital rates did accentuate extinction risk compared to right‐tail associations, but the effect was small. In contrast, we showed that density dependence interacts with tail associations to influence metapopulation extinction risk substantially: For population models showing undercompensatory density dependence, left‐tail associations in environmental variables often strongly accentuated and right‐tail associations mitigated extinction risk, whereas the reverse was usually true for models showing overcompensatory density dependence. Tail associations and their asymmetries are taken into account in assessing risks in finance and other fields, but to our knowledge, our study is one of the first to consider how tail associations influence population extinction risk. Our modeling results provide an initial demonstration of a new mechanism influencing extinction risks and, in our view, should help motivate more comprehensive study of the mechanism and its importance for real populations in future work. 

   more » « less