Search for: All records

Abstract It remains an interesting and challenging problem to synthesize a vivid and realistic singing face driven by music. In this paper, we present a method for this task with natural motions for the lips, facial expression, head pose, and eyes. Due to the coupling of mixed information for the human voice and backing music in common music audio signals, we design a decouple-and-fuse strategy to tackle the challenge. We first decompose the input music audio into a human voice stream and a backing music stream. Due to the implicit and complicated correlation between the two-stream input signals and the dynamics of the facial expressions, head motions, and eye states, we model their relationship with an attention scheme, where the effects of the two streams are fused seamlessly. Furthermore, to improve the expressivenes of the generated results, we decompose head movement generation in terms of speed and direction, and decompose eye state generation into short-term blinking and long-term eye closing, modeling them separately. We have also built a novel dataset, SingingFace, to support training and evaluation of models for this task, including future work on this topic. Extensive experiments and a user study show that our proposed method is capable of synthesizing vivid singing faces, qualitatively and quantitatively better than the prior state-of-the-art. 

Robust explanations of machine learning models are critical to establish human trust in the models. Due to limited cognition capability, most humans can only interpret the top few salient features. It is critical to make top salient features robust to adversarial attacks, especially those against the more vulnerable gradient-based explanations. Existing defense measures robustness using lp norms, which have weaker protection power. We define explanation thickness for measuring salient features ranking stability, and derive tractable surrogate bounds of the thickness to design the R2ET algorithm to efficiently maximize the thickness and anchor top salient features. Theoretically, we prove a connection between R2ET and adversarial training. Experiments with a wide spectrum of network architectures and data modalities, including brain networks, demonstrate that R2ET attains higher explanation robustness under stealthy attacks while retaining accuracy. 

Abstract Due to the highly nonlinear nature of the beam-loading, it is currently not possible to analytically determine the beam parameters needed in a two-bunch plasma wakefield accelerator for maintaining a low energy spread. Therefore in this paper, by using the Broyden–Fletcher–Goldfarb–Shanno algorithm for the parameter scanning with the code QuickPIC and the polynomial regression together with k -fold cross-validation method, we obtain two fitting formulas for calculating the parameters of tri-Gaussian electron beams when minimizing the energy spread based on the beam-loading effect in a nonlinear plasma wakefield accelerator. One formula allows the optimization of the normalized charge per unit length of a trailing beam to achieve the minimal energy spread, i.e. the optimal beam-loading. The other one directly gives the transformer ratio when the trailing beam achieves the optimal beam-loading. A simple scaling law for charges of drive beams and trailing beams is obtained from the fitting formula, which indicates that the optimal beam-loading is always achieved for a given charge ratio of the two beams when the length and separation of two beams and the plasma density are fixed. The formulas can also help obtain the optimal plasma densities for the maximum accelerated charge and the maximum acceleration efficiency under the optimal beam-loading respectively. These two fitting formulas will significantly enhance the efficiency for designing and optimizing a two-bunch plasma wakefield acceleration stage.