More Like this

1. Speech Enhancement Using Forked Generative Adversarial Networks with Spectral Subtraction

   https://doi.org/10.21437/Interspeech.2019-2954  

   Lin, Ju; Niu, Sufeng; Wei, Zice; Lan, Xiang; Wijngaarden, Adriaan J.; Smith, Melissa C.; Wang, Kuang-Ching (September 2019, Proceedings of Interspeech 2019)

   null (Ed.)

   Speech enhancement techniques that use a generative adversarial network (GAN) can effectively suppress noise while allowing models to be trained end-to-end. However, such techniques directly operate on time-domain waveforms, which are often highly-dimensional and require extensive computation. This paper proposes a novel GAN-based speech enhancement method, referred to as S-ForkGAN, that operates on log-power spectra rather than on time-domain speech waveforms, and uses a forked GAN structure to extract both speech and noise information. By operating on log-power spectra, one can seamlessly include conventional spectral subtraction techniques, and the parameter space typically has a lower dimension. The performance of S-ForkGAN is assessed for automatic speech recognition (ASR) using the TIMIT data set and a wide range of noise conditions. It is shown that S-ForkGAN outperforms existing GAN-based techniques and that it has a lower complexity. 

   more » « less
2. Dual-Path Minimum-Phase and All-Pass Decomposition Network for Single Channel Speech Dereverberation

   https://doi.org/10.1109/ICASSP48485.2024.10446719  

   Liu, Xi; Chen, Szu-Jui; Hansen, John_H L (April 2024, IEEE)

   With the development of deep neural networks (DNN), many DNN-based speech dereverberation approaches have been proposed to achieve significant improvement over the traditional methods. However, most deep learning-based dereverberation methods solely focus on suppressing time-frequency domain reverberations without utilizing cepstral domain features which are potentially useful for dereverberation. In this paper, we propose a dual-path neural network structure to separately process minimum-phase and all-pass components of single channel speech. First, we decompose speech signal into minimum-phase and all-pass components in cepstral domain, then Conformer embedded U-Net is used to remove reverberations of both components. Finally, we combine these two processed components together to synthesize the enhanced output. The performance of proposed method is tested on REVERB-Challenge evaluation dataset in terms of commonly used objective metrics. Experimental results demonstrate that our method outperforms other compared methods. 

   more » « less
3. Enhancing Bayesian Inference-Based Damage Diagnostics Through Domain Translation With Application to Miter Gates

   https://doi.org/10.1115/1.4067719  

   Zeng, Yichao; Zhao, Zhao; Qian, Guofeng; Todd, Michael D; Hu, Zhen (June 2025, Journal of Mechanical Design)

   Abstract Bayesian inference based on computational simulations plays a crucial role in model-informed damage diagnostics and the design of reliable engineering systems, such as the miter gates studied in this article. While Bayesian inference for damage diagnostics has shown success in some applications, the current method relies on monitoring data from solely the asset of interest and may be affected by imperfections in the computational simulation model. To address these limitations, this article introduces a novel approach called Bayesian inference-based damage diagnostics enhanced through domain translation (BiEDT). The proposed BiEDT framework incorporates historical damage inspection and monitoring data from similar yet different miter gates, aiming to provide alternative data-driven methods for damage diagnostics. The proposed framework first translates observations from different miter gates into a unified analysis domain using two domain translation techniques, namely, cycle-consistent generative adversarial network (CycleGAN) and domain-adversarial neural network (DANN). Following the domain translation, a conditional invertible neural network (cINN) is employed to estimate the damage state, with uncertainty quantified in a Bayesian manner. Additionally, a Bayesian model averaging and selection method is developed to integrate the posterior distributions from different methods and select the best model for decision-making. A practical miter gate structural system is employed to demonstrate the efficacy of the BiEDT framework. Results indicate that the alternative damage diagnostics approaches based on domain translation can effectively enhance the performance of Bayesian inference-based damage diagnostics using computational simulations. 

   more » « less
4. Bloom-Net: Blockwise Optimization for Masking Networks Toward Scalable and Efficient Speech Enhancement

   https://doi.org/10.1109/ICASSP43922.2022.9746767  

   Kim, Sunwoo; Kim, Minje (May 2022, IEEE International Conference on Acoustics, Speech and Signal Processing)

   In this paper, we present a blockwise optimization method for masking-based networks (BLOOM-Net) for training scalable speech enhancement networks. Here, we design our network with a residual learning scheme and train the internal separator blocks sequentially to obtain a scalable masking-based deep neural network for speech enhancement. Its scalability lets it dynamically adjust the run-time complexity depending on the test time environment. To this end, we modularize our models in that they can flexibly accommodate varying needs for enhancement performance and constraints on the resources, incurring minimal memory or training overhead due to the added scalability. Our experiments on speech enhancement demonstrate that the proposed blockwise optimization method achieves the desired scalability with only a slight performance degradation compared to corresponding models trained end-to-end. 

   more » « less
5. Dynamic-GAN: Learning Spatial-Temporal Attention for Dynamic Object Removal in Feature Dense Environments

   https://doi.org/10.1109/IROS47612.2022.9981102  

   Trombley, Christopher M; Kumar Das, Sumit; Popa, Dan O (October 2022, Dynamic-GAN: Learning Spatial-Temporal Attention for Dynamic Object Removal in Feature Dense Environments)

   This paper presents an attention-based, deep learning framework that converts robot camera frames with dynamic content into static frames to more easily apply simultaneous localization and mapping (SLAM) algorithms. The vast majority of SLAM methods have difficulty in the presence of dynamic objects appearing in the environment and occluding the area being captured by the camera. Despite past attempts to deal with dynamic objects, challenges remain to reconstruct large, occluded areas with complex backgrounds. Our proposed Dynamic-GAN framework employs a generative adversarial network to remove dynamic objects from a scene and inpaint a static image free of dynamic objects. The Dynamic-GAN framework utilizes spatial-temporal transformers, and a novel spatial-temporal loss function. The evaluation of Dynamic-GAN was comprehensively conducted both quantitatively and qualitatively by testing it on benchmark datasets, and on a mobile robot in indoor navigation environments. As people appeared dynamically in close proximity to the robot, results showed that large, feature-rich occluded areas can be accurately reconstructed with our attention-based deep learning framework for dynamic object removal. Through experiments we demonstrate that our proposed algorithm has up to 25% better performance on average as compared to the standard benchmark algorithms. 

   more » « less