More Like this

1. Clustering on Sparse Data in Non-overlapping Feature Space with Applications to Cancer Subtyping

   Kang, Tianyu; Zarringhalam, Kourosh; Kuijjer, Marieke; Chen, Ping; Quackenbush, John; Ding, Wei (November 2018, 2018 IEEE International Conference on Data Mining (ICDM))

   This paper presents a new algorithm, Reinforced and Informed Network-based Clustering (RINC), for finding unknown groups of similar data objects in sparse and largely non-overlapping feature space where a network structure among features can be observed. Sparse and non-overlapping unlabeled data become increasingly common and available especially in text mining and biomedical data mining. RINC inserts a domain informed model into a modelless neural network. In particular, our approach integrates physically meaningful feature dependencies into the neural network architecture and soft computational constraint. Our learning algorithm efficiently clusters sparse data through integrated smoothing and sparse auto-encoder learning. The informed design requires fewer samples for training and at least part of the model becomes explainable. The architecture of the reinforced network layers smooths sparse data over the network dependency in the feature space. Most importantly, through back-propagation, the weights of the reinforced smoothing layers are simultaneously constrained by the remaining sparse auto-encoder layers that set the target values to be equal to the raw inputs. Empirical results demonstrate that RINC achieves improved accuracy and renders physically meaningful clustering results. 

   more » « less
2. BEAN: Interpretable and Efficient Learning With Biologically-Enhanced Artificial Neuronal Assembly Regularization

   https://doi.org/10.3389/fnbot.2021.567482  

   Gao, Yuyang; Ascoli, Giorgio A.; Zhao, Liang (June 2021, Frontiers in Neurorobotics)

   null (Ed.)

   Deep neural networks (DNNs) are known for extracting useful information from large amounts of data. However, the representations learned in DNNs are typically hard to interpret, especially in dense layers. One crucial issue of the classical DNN model such as multilayer perceptron (MLP) is that neurons in the same layer of DNNs are conditionally independent of each other, which makes co-training and emergence of higher modularity difficult. In contrast to DNNs, biological neurons in mammalian brains display substantial dependency patterns. Specifically, biological neural networks encode representations by so-called neuronal assemblies: groups of neurons interconnected by strong synaptic interactions and sharing joint semantic content. The resulting population coding is essential for human cognitive and mnemonic processes. Here, we propose a novel Biologically Enhanced Artificial Neuronal assembly (BEAN) regularization 1 to model neuronal correlations and dependencies, inspired by cell assembly theory from neuroscience. Experimental results show that BEAN enables the formation of interpretable neuronal functional clusters and consequently promotes a sparse, memory/computation-efficient network without loss of model performance. Moreover, our few-shot learning experiments demonstrate that BEAN could also enhance the generalizability of the model when training samples are extremely limited. 

   more » « less
3. RAPTA: A Hierarchical Representation Learning Solution For Real-Time Prediction of Path-Based Static Timing Analysis

   https://doi.org/10.1145/3526241.3530831  

   Chowdhury, Tanmoy; Vakil, Ashkan; Saber Latibari, Banafsheh; Aresh Beheshti Shirazi, Seyed; Mirzaeian, Ali; Guo, Xiaojie; Manoj P D, Sai; Homayoun, Houman; Savidis, Ioannis; Zhao, Liang; et al (June 2022, GLSVLSI '22: Proceedings of the Great Lakes Symposium on VLSI)

   This paper presents RAPTA, a customized Representation-learning Architecture for automation of feature engineering and predicting the result of Path-based Timing-Analysis early in the physical design cycle. RAPTA offers multiple advantages compared to prior work: 1) It has superior accuracy with errors std ranges 3.9ps~16.05ps in 32nm technology. 2) RAPTA's architecture does not change with feature-set size, 3) RAPTA does not require manual input feature engineering. To the best of our knowledge, this is the first work, in which Bidirectional Long Short-Term Memory (Bi-LSTM) representation learning is used to digest raw information for feature engineering, where generation of latent features and Multilayer Perceptron (MLP) based regression for timing prediction can be trained end-to-end. 

   more » « less
4. What Makes Freezing Layers in Deep Neural Networks Effective? A Linear Separability Perspective

   Coil, Collin; Cheney, Nick (June 2025, AutoML Conference 2025)

   Freezing layers in deep neural networks has been shown to enhance generalization and accelerate training, yet the underlying mechanisms remain unclear. This paper investigates the impact of frozen layers from the perspective of linear separability, examining how untrained, randomly initialized layers influence feature representations and model performance. Using multilayer perceptrons trained on MNIST, CIFAR-10, and CIFAR-100, we systematically analyze the effects freezing layers and network architecture. While prior work attributes the benefits of frozen layers to Cover’s theorem, which suggests that nonlinear transformations improve linear separability, we find that this explanation is insufficient. Instead, our results indicate that the observed improvements in generalization and convergence stem from other mechanisms. We hypothesize that freezing may have similar effects to other regularization techniques and that it may smooth the loss landscape to facilitate training. Furthermore, we identify key architectural factors---such as network overparameterization and use of skip connections---that modulate the effectiveness of frozen layers. These findings offer new insights into the conditions under which freezing layers can optimize deep learning performance, informing future work on neural architecture search. 

   more » « less
5. Discovering functionality of urban regions by learning low-dimensional representations of a spatial multiplex network

   Hajiseyedjavadi, Seyedsaeed; Lin, Yu-Ru; Pelechrinis, Konstantinos (August 2018, Proceedings of the Third Mining Urban Data Workshop (MUD 2018))

   The complex relationships in an urban environment can be captured through multiple interrelated sources of data. These relationships form multilayer networks, that are also spatially embedded in an area, could be used to identify latent patterns. In this work, we propose a low-dimensional representation learning approach that considers multiple layers of a multiplex network simultaneously and is able to encode similarities between nodes across different layers. In particular, we introduce a novel neural network architecture to jointly learn low-dimensional representations of each network node from multiple layers of a network. This process simultaneously fuses knowledge of various data sources to better capture the characteristics of the nodes. To showcase the proposed method we focus on the problem of identifying the functionality of an urban region. Using a variety of public data sources for New York City, we design a multilayer network and evaluate our approach. Our results indicate that our proposed approach can improve the accuracy of traditional approaches in an unsupervised task. 

   more » « less