skip to main content

Search for: All records

Creators/Authors contains: "Deng, Y."

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. In this paper we prove that Local (S)GD (or FedAvg) can optimize deep neural networks with Rectified Linear Unit (ReLU) activation function in polynomial time. Despite the established convergence theory of Local SGD on optimizing general smooth functions in communication-efficient distributed optimization, its convergence on non-smooth ReLU networks still eludes full theoretical understanding. The key property used in many Local SGD analysis on smooth function is gradient Lipschitzness, so that the gradient on local models will not drift far away from that on averaged model. However, this decent property does not hold in networks with non-smooth ReLU activation function. We show that, even though ReLU network does not admit gradient Lipschitzness property, the difference between gradients on local models and average model will not change too much, under the dynamics of Local SGD. We validate our theoretical results via extensive experiments. This work is the first to show the convergence of Local SGD on non-smooth functions, and will shed lights on the optimization theory of federated training of deep neural networks.
    Free, publicly-accessible full text available October 28, 2023
  2. Despite the established convergence theory of Optimistic Gradient Descent Ascent (OGDA) and Extragradient (EG) methods for the convex-concave minimax problems, little is known about the theoretical guarantees of these methods in nonconvex settings. To bridge this gap, for the first time, this paper establishes the convergence of OGDA and EG methods under the nonconvex-strongly-concave (NC-SC) and nonconvex-concave (NC-C) settings by providing a unified analysis through the lens of single-call extra-gradient methods. We further establish lower bounds on the convergence of GDA/OGDA/EG, shedding light on the tightness of our analysis. We also conduct experiments supporting our theoretical results. We believe our results will advance the theoretical understanding of OGDA and EG methods for solving complicated nonconvex minimax real-world problems, e.g., Generative Adversarial Networks (GANs) or robust neural networks training.
    Free, publicly-accessible full text available October 15, 2023
  3. Gaussian process (GP) is a popular machine learning technique that is widely used in many application domains, especially in robotics. However, GP is very computation intensive and time consuming during the inference phase, thereby bringing severe challenges for its large-scale deployment in real-time applications. In this paper, we propose two efficient hardware architecture for GP accelerator. One architecture targets for general GP inference, and the other architecture is specifically optimized for the scenario when the data point is gradually observed. Evaluation results show that the proposed hardware accelerator provides significant hardware performance improvement than the general-purpose computing platform.
  4. Binary granular soil mixtures, as common heterogeneous soils, are ubiquitous in nature and man-made deposits. Fines content and particle size ratio are two important gradation parameters for a binary mixture, which have potential influences on mechanical behaviours. However, experimental studies on drained shear behaviour considering the whole range of fines content and different particle size ratios are scarce in the literature. For this purpose, we performed a series of drained triaxial compression tests on dense binary silica sand mixtures with 4 different particle size ratios to systematically investigate the effects of fines content and particle size ratio on the drained shear behaviours. Based on these tests, the strength-dilation behaviour and critical state behaviour were examined. It was observed that both fines content and particle size ratio have significant influence on the stress-strain response, the critical state void ratio, the critical state friction angle, the maximum dilation angle, the peak friction angle, and the strength–dilatancy relation. The underlying mechanism for the effects of fines content and particle size ratio was discussed from the perspective of the kinematic movements at particle level.
  5. Context. The carbon monoxide (CO) molecular line at around 46655 Å in solar infrared spectra is often used to investigate the dynamic behavior of the cold heart of the solar atmosphere, i.e., sunspot oscillation, especially at the sunspot umbra. Aims. We investigated sunspot oscillation at Doppler velocities of the CO 7-6 R67 and 3-2 R14 lines that were measured by the Cryogenic Infrared Spectrograph (CYRA), as well as the line profile of Mg  II k line that was detected by the Interface Region Imaging Spectrograph (IRIS). Methods. A single Gaussian function is applied to each CO line profile to extract the line shift, while the moment analysis method is used for the Mg  II k line. Then the sunspot oscillation can be found in the time–distance image of Doppler velocities, and the quasi-periodicity at the sunspot umbra are determined from the wavelet power spectrum. Finally, the cross-correlation method is used to analyze the phase relation between different atmospheric levels. Results. At the sunspot umbra, a periodicity of roughly 5 min is detected at the Doppler velocity range of the CO 7-6 R67 line that formed in the photosphere, while a periodicity of around 3 min is discovered at the Dopplermore »velocities of CO 3-2 R14 and Mg  II k lines that formed in the upper photosphere or the temperature minimum region and the chromosphere. A time delay of about 2 min is measured between the strong CO 3-2 R14 line and the Mg  II k line. Conclusions. Based on the spectroscopic observations from the CYRA and IRIS, the 3 min sunspot oscillation can be spatially resolved in the Doppler shifts. It may come from the upper photosphere or the temperature minimum region and then propagate to the chromosphere, which might be regarded as a propagating slow magnetoacoustic wave.« less