skip to main content

Search for: All records

Creators/Authors contains: "Deng, Z"

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. Singh, A (Ed.)
    Robust optimization has been widely used in nowadays data science, especially in adversarial training. However, little research has been done to quantify how robust optimization changes the optimizers and the prediction losses comparing to standard training. In this paper, inspired by the influence function in robust statistics, we introduce the Adversarial Influence Function (AIF) as a tool to investigate the solution produced by robust optimization. The proposed AIF enjoys a closed-form and can be calculated efficiently. To illustrate the usage of AIF, we apply it to study model sensitivity — a quantity defined to capture the change of prediction losses on the natural data after implementing robust optimization. We use AIF to analyze how model complexity and randomized smoothing affect the model sensitivity with respect to specific models. We further derive AIF for kernel regressions, with a particular application to neural tangent kernels, and experimentally demonstrate the effectiveness of the proposed AIF. Lastly, the theories of AIF will be extended to distributional robust optimization.
  2. Daumé, H ; Singh, A (Ed.)
    An acknowledged weakness of neural networks is their vulnerability to adversarial perturbations to the inputs. To improve the robustness of these models, one of the most popular defense mechanisms is to alternatively maximize the loss over the constrained perturbations (or called adversaries) on the inputs using projected gradient ascent and minimize over weights. In this paper, we analyze the dynamics of the maximization step towards understanding the experimentally observed effectiveness of this defense mechanism. Specifically, we investigate the non-concave landscape of the adversaries for a two-layer neural network with a quadratic loss. Our main result proves that projected gradient ascent finds a local maximum of this non-concave problem in a polynomial number of iterations with high probability. To our knowledge, this is the first work that provides a convergence analysis of the first-order adversaries. Moreover, our analysis demonstrates that, in the initial phase of adversarial training, the scale of the inputs matters in the sense that a smaller input scale leads to faster convergence of adversarial training and a “more regular” landscape. Finally, we show that these theoretical findings are in excellent agreement with a series of experiments.
  3. Four inter-related measures of phase are described to study the phase synchronization of cellular oscillators, and computation of these measures is described and illustrated on single cell fluorescence data from the model filamentous fungus, Neurospora crassa. One of these four measures is the phase shift ϕ in a sinusoid of the form x(t) = A(cos(ωt + ϕ), where t is time. The other measures arise by creating a replica of the periodic process x(t) called the Hilbert transform x̃(t), which is 90 degrees out of phase with the original process x(t). The second phase measure is the phase angle FH(t) between the replica x̃(t) and X(t), taking values between -π and π. At extreme values the Hilbert Phase is discontinuous, and a continuous form FC(t) of the Hilbert Phase is used, measuring time on the nonnegative real axis (t). The continuous Hilbert Phase FC(t) is used to define the phase MC(t1,t0) for an experiment beginning at time t0 and ending at time t1. In that phase differences at time t0 are often of ancillary interest, the Hilbert Phase FC(t0) is subtracted from FC(t1). This difference is divided by 2π to obtain the phase MC(t1,t0) in cycles. Both the Hilbert Phasemore »FC(t) and the phase MC(t1,t0) are functions of time and useful in studying when oscillators phase-synchronize in time in signal processing and circadian rhythms in particular. The phase of cellular clocks is fundamentally different from circadian clocks at the macroscopic scale because there is an hourly cycle superimposed on the circadian cycle.« less
  4. This paper reports the measurement on light entrainment of single cell circadian oscillator of a model fungal system, Neurospora crassa (N. crassa), through a high-throughput microfluidic droplet platform [1]. The results demonstrated for the first time that single cell circadian oscillators could be entrained by light.