An official website of the United States government
Deep neural networks have been shown to be effective adaptive beamformers for ultrasound imaging. However, when training with traditional L p norm loss functions, model selection is difficult because lower loss values are not always associated with higher image quality. This ultimately limits the maximum achievable image quality with this approach and raises concerns about the optimization objective. In an effort to align the optimization objective with the image quality metrics of interest, we implemented a novel ultrasound-specific loss function based on the spatial lag-one coherence and signal-to-noise ratio of the delayed channel data in the short-time Fourier domain. We employed the R-Adam optimizer with look ahead and cyclical learning rate to make the training more robust to initialization and local minima, leading to better model performance and more reliable convergence. With our custom loss function and optimization scheme, we achieved higher contrast-to-noise-ratio, higher speckle signal-to-noise-ratio, and more accurate contrast ratio reconstruction than with previous deep learning and delay-and-sum beamforming approaches.
more »
« less
This content will become publicly available on July 10, 2026
Improving the Adam optimizer using time delays
Abstract One of the more popular optimization methods in current use is the Adam optimizer. This is due, at least in part, to its effectiveness as a training algorithm for deep neural networks, which are associated with many machine learning tasks. In this paper, we introduce time delays into the Adam optimizer. Time delays typically have an adverse effect on dynamical systems, including optimizers, slowing the system’s rate of convergence and potentially causing instabilities. However, our numerical experiments indicate that introducing time-delays into the Adam optimizer can significantly improve its performance, resulting in an often much smaller loss-value. Perhaps more surprising is that this improvement often scales with dimension-the higher the dimension the greater the advantage of using time delays in improving loss-values. Along with describing these results we show that, for the time-delays we consider, the temporal complexity of the delayed Adam optimizer remains the same as the undelayed optimizer and that the algorithm’s spatial complexity scales linearly in the length of the largest time-delay. Last, we extend the theory of intrinsic stability to give a criterion under which the minima, either local or global, associated with the delayed Adam optimizer are stable.
more »
« less
- Award ID(s):
- 2205837
- PAR ID:
- 10627601
- Publisher / Repository:
- London Mathematical Society; IOP Publishing
- Date Published:
- Journal Name:
- Nonlinearity
- Volume:
- 38
- Issue:
- 7
- ISSN:
- 0951-7715
- Page Range / eLocation ID:
- 075030
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
IntroductionAs robot teleoperation increasingly becomes integral in executing tasks in distant, hazardous, or inaccessible environments, operational delays remain a significant obstacle. These delays, inherent in signal transmission and processing, adversely affect operator performance, particularly in tasks requiring precision and timeliness. While current research has made strides in mitigating these delays through advanced control strategies and training methods, a crucial gap persists in understanding the neurofunctional impacts of these delays and the efficacy of countermeasures from a cognitive perspective. MethodsThis study addresses the gap by leveraging functional Near-Infrared Spectroscopy (fNIRS) to examine the neurofunctional implications of simulated haptic feedback on cognitive activity and motor coordination under delayed conditions. In a human-subject experiment (N= 41), sensory feedback was manipulated to observe its influences on various brain regions of interest (ROIs) during teleoperation tasks. The fNIRS data provided a detailed assessment of cerebral activity, particularly in ROIs implicated in time perception and the execution of precise movements. ResultsOur results reveal that the anchoring condition, which provided immediate simulated haptic feedback with a delayed visual cue, significantly optimized neural functions related to time perception and motor coordination. This condition also improved motor performance compared to the asynchronous condition, where visual and haptic feedback were misaligned. DiscussionThese findings provide empirical evidence about the neurofunctional basis of the enhanced motor performance with simulated synthetic force feedback in the presence of teleoperation delays. The study highlights the potential for immediate haptic feedback to mitigate the adverse effects of operational delays, thereby improving the efficacy of teleoperation in critical applications.more » « less
-
We describe an analysis of speech during time-critical, team-based medical work and its potential to indicate process delays. We analyzed speech intention and sentence types during 39 trauma resuscitations with delays in one of three major lifesaving interventions: intravenous/intraosseous (IV/IO) line insertion, cardiopulmonary and resuscitation (CPR), and intubation. We found a significant difference in patterns of speech during delays vs. speech during non-delayed work. The speech intention during CPR delays, however, differed from the other LSIs, suggesting that context of speech must be considered. These findings will inform the design of a clinical decision support system (CDSS) that will use multiple sensor modalities to alert medical teams to delays in real time. We conclude with design implications and challenges associated with speech-based activity recognition in complex medical processes.more » « less
-
We describe an analysis of speech during time-critical, team-based medical work and its potential to indicate process delays. We analyzed speech intention and sentence types during 39 trauma resuscitations with delays in one of three major lifesaving interventions: intravenous/intraosseous (IV/IO) line insertion, cardiopulmonary and resuscitation (CPR), and intubation. We found a significant difference in patterns of speech during delays vs. speech during non-delayed work. The speech intention during CPR delays, however, differed from the other LSIs, suggesting that context of speech must be considered. These findings will inform the design of a clinical decision support system (CDSS) that will use multiple sensor modalities to alert medical teams to delays in real time. We conclude with design implications and challenges associated with speech-based activity recognition in complex medical processes.more » « less
-
Scalable Computation of Controlled Invariant Sets for Discrete-Time Linear Systems with Input Delaysnull (Ed.)In this paper, we first propose a method that can efficiently compute the maximal robust controlled invariant set for discrete-time linear systems with pure delay in input. The key to this method is to construct an auxiliary linear system (without delay) with the same state-space dimension of the original system in consideration and to relate the maximal invariant set of the auxiliary system to that of the original system. When the system is subject to disturbances, guaranteeing safety is harder for systems with input delays. Ability to incorporate any additional information about the disturbance becomes more critical in these cases. Motivated by this observation, in the second part of the paper, we generalize the proposed method to take into account additional preview information on the disturbances, while maintaining computational efficiency. Compared with the naive approach of constructing a higher dimensional system by appending the state-space with the delayed inputs and previewed disturbances, the proposed approach is demonstrated to scale much better with the increasing delay time.more » « less
