More Like this

1. A DNN-HMM-DNN Hybrid Model for Discovering Word-Like Units from Spoken Captions and Image Regions

   https://doi.org/10.21437/Interspeech.2020-1148  

   Wang, Liming; Hasegawa-Johnson, Mark (October 2020, Interspeech)

   null (Ed.)

   Discovering word-like units without textual transcriptions is an important step in low-resource speech technology. In this work,we demonstrate a model inspired by statistical machine translation and hidden Markov model/deep neural network (HMM-DNN) hybrid systems. Our learning algorithm is capable of discovering the visual and acoustic correlates of distinct words in an unknown language by simultaneously learning the map-ping from image regions to concepts (the first DNN), the map-ping from acoustic feature vectors to phones (the second DNN),and the optimum alignment between the two (the HMM). In the simulated low-resource setting using MSCOCO and Speech-COCO datasets, our model achieves 62.4 % alignment accuracy and outperforms the audio-only segmental embedded GMM approach on standard word discovery evaluation metrics. 

   more » « less
2. DNSS2 : Improved ab initio protein secondary structure prediction using advanced deep learning architectures

   https://doi.org/10.1002/prot.26007  

   Guo, Zhiye; Hou, Jie; Cheng, Jianlin (September 2020, Proteins: Structure, Function, and Bioinformatics)

   Abstract Accurate prediction of protein secondary structure (alpha‐helix, beta‐strand and coil) is a crucial step for protein inter‐residue contact prediction and ab initio tertiary structure prediction. In a previous study, we developed a deep belief network‐based protein secondary structure method (DNSS1) and successfully advanced the prediction accuracy beyond 80%. In this work, we developed multiple advanced deep learning architectures (DNSS2) to further improve secondary structure prediction. The major improvements over the DNSS1 method include (a) designing and integrating six advanced one‐dimensional deep convolutional/recurrent/residual/memory/fractal/inception networks to predict 3‐state and 8‐state secondary structure, and (b) using more sensitive profile features inferred from Hidden Markov model (HMM) and multiple sequence alignment (MSA). Most of the deep learning architectures are novel for protein secondary structure prediction. DNSS2 was systematically benchmarked on independent test data sets with eight state‐of‐art tools and consistently ranked as one of the best methods. Particularly, DNSS2 was tested on the protein targets of 2018 CASP13 experiment and achieved the Q3 score of 81.62%, SOV score of 72.19%, and Q8 score of 73.28%. DNSS2 is freely available at:https://github.com/multicom-toolbox/DNSS2. 

   more » « less
3. Efficient distributed state estimation of hidden Markov Models over unreliable networks

   https://doi.org/10.1109/MRS.2017.8250939  

   Tamjidi, Amirhossein; Oftadeh, Reza; Chakravorty, Suman; Shell, Dylan (December 2017, 2017 International Symposium on Multi-Robot and Multi-Agent Systems (MRS))

   This paper presents a new recursive Hybrid consensus filter for distributed state estimation on a Hidden Markov Model (HMM), which is well suited to multirobot applications and settings. The proposed algorithm is scalable, robust to network failure and capable of handling non-Gaussian transition and observation models and is, therefore, quite general. No global knowledge of the communication network is assumed. Iterative Conservative Fusion (ICF) is used to reach consensus over potentially correlated priors, while consensus over likelihoods is handled using weights based on a Metropolis Hastings Markov Chain (MHMC). The proposed method is evaluated in a multi-agent tracking problem and a high-dimensional HMM and it is shown that its performance surpasses the competing algorithms. 

   more » « less
4. Diagnosing Storm Mode with Deep Learning in Convection-Allowing Models

   https://doi.org/10.1175/MWR-D-22-0342.1  

   Sobash, Ryan A.; Gagne, David John; Becker, Charlie L.; Ahijevych, David; Gantos, Gabrielle N.; Schwartz, Craig S. (May 2023, Monthly Weather Review)

   Abstract While convective storm mode is explicitly depicted in convection-allowing model (CAM) output, subjectively diagnosing mode in large volumes of CAM forecasts can be burdensome. In this work, four machine learning (ML) models were trained to probabilistically classify CAM storms into one of three modes: supercells, quasi-linear convective systems, and disorganized convection. The four ML models included a dense neural network (DNN), logistic regression (LR), a convolutional neural network (CNN) and semi-supervised CNN-Gaussian mixture model (GMM). The DNN, CNN, and LR were trained with a set of hand-labeled CAM storms, while the semi-supervised GMM used updraft helicity and storm size to generate clusters which were then hand labeled. When evaluated using storms withheld from training, the four classifiers had similar ability to discriminate between modes, but the GMM had worse calibration. The DNN and LR had similar objective performance to the CNN, suggesting that CNN-based methods may not be needed for mode classification tasks. The mode classifications from all four classifiers successfully approximated the known climatology of modes in the U.S., including a maximum in supercell occurrence in the U.S. Central Plains. Further, the modes also occurred in environments recognized to support the three different storm morphologies. Finally, storm mode provided useful information about hazard type, e.g., storm reports were most likely with supercells, further supporting the efficacy of the classifiers. Future applications, including the use of objective CAM mode classifications as a novel predictor in ML systems, could potentially lead to improved forecasts of convective hazards. 

   more » « less
5. Latent State Models of Training Dynamics

   Hu, Michael; Chen, Angelica; Saphra, Naomi; Cho, Kyunghyun (November 2023, Transactions on Machine Learning Research (TMLR) 2023)

   The impact of randomness on model training is poorly understood. How do differences in data order and initialization actually manifest in the model, such that some training runs outperform others or converge faster? Furthermore, how can we interpret the resulting training dynamics and the phase transitions that characterize different trajectories? To understand the effect of randomness on the dynamics and outcomes of neural network training, we train models multiple times with different random seeds and compute a variety of metrics throughout training, such as the norm, mean, and variance of the neural network's weights. We then fit a hidden Markov model (HMM) over the resulting sequences of metrics. The HMM represents training as a stochastic process of transitions between latent states, providing an intuitive overview of significant changes during training. Using our method, we produce a low-dimensional, discrete representation of training dynamics on grokking tasks, image classification, and masked language modeling. We use the HMM representation to study phase transitions and identify latent "detour" states that slow down convergence. 

   more » « less