More Like this

1. Network properties determine neural network performance

   https://doi.org/10.1038/s41467-024-48069-8  

   Jiang, Chunheng ; Huang, Zhenhan ; Pedapati, Tejaswini ; Chen, Pin-Yu ; Sun, Yizhou ; Gao, Jianxi ( July 2024 , Nature Communications)

   Machine learning influences numerous aspects of modern society, empowers new technologies, from Alphago to ChatGPT, and increasingly materializes in consumer products such as smartphones and self-driving cars. Despite the vital role and broad applications of artificial neural networks, we lack systematic approaches, such as network science, to understand their underlying mechanism. The difficulty is rooted in many possible model configurations, each with different hyper-parameters and weighted architectures determined by noisy data. We bridge the gap by developing a mathematical framework that maps the neural network’s performance to the network characters of the line graph governed by the edge dynamics of stochastic gradient descent differential equations. This framework enables us to derive a neural capacitance metric to universally capture a model’s generalization capability on a downstream task and predict model performance using only early training results. The numerical results on 17 pre-trained ImageNet models across five benchmark datasets and one NAS benchmark indicate that our neural capacitance metric is a powerful indicator for model selection based only on early training results and is more efficient than state-of-the-art methods.

    

   more » « less
2. Network Schema Preserving Heterogeneous Information Network Embedding

   https://doi.org/10.24963/ijcai.2020/190  

   Zhao, Jianan ; Wang, Xiao ; Shi, Chuan ; Liu, Zekuan ; Ye, Yanfang ( July 2020 , International Joint Conference on Artificial Intelligence (IJCAI))

   null (Ed.)

   As heterogeneous networks have become increasingly ubiquitous, Heterogeneous Information Network (HIN) embedding, aiming to project nodes into a low-dimensional space while preserving the heterogeneous structure, has drawn increasing attention in recent years. Many of the existing HIN embedding methods adopt meta-path guided random walk to retain both the semantics and structural correlations between different types of nodes. However, the selection of meta-paths is still an open problem, which either depends on domain knowledge or is learned from label information. As a uniform blueprint of HIN, the network schema comprehensively embraces the high-order structure and contains rich semantics. In this paper, we make the first attempt to study network schema preserving HIN embedding, and propose a novel model named NSHE. In NSHE, a network schema sampling method is first proposed to generate sub-graphs (i.e., schema instances), and then multi-task learning task is built to preserve the heterogeneous structure of each schema instance. Besides preserving pairwise structure information, NSHE is able to retain high-order structure (i.e., network schema). Extensive experiments on three real-world datasets demonstrate that our proposed model NSHE significantly outperforms the state-of-the-art methods.

    

   more » « less
3. Tensor Network Contraction For Network Reliability Estimates

   Shepherd, K. ; Dueñas-Osorio, L. ( September 2022 , The 13th International Conference on Structural Safety and Reliability (ICOSSAR 2021-2022))

   Li, J. ; Spanos, P. D. ; Chen, J.-B. ; Peng, Y.-B. (Ed.)

   Quantifying network reliability is a hard problem, proven to be #P-complete [1]. For real-world network planning and decision making, approximations for the network reliability problem are necessary. This study shows that tensor network contraction (TNC) methods can quickly estimate an upper bound of All Terminal Reliability, RelATR(G), by solving a superset of the network reliability problem: the edge cover problem, EC(G). In addition, these tensor contraction methods can exactly solve source-terminal (S-T) reliability for the class of directed acyclic networks, RelS−T (G). The computational complexity of TNC methods is parameterized by treewidth, significantly benefitting from recent advancements in approximate tree decomposition algorithms [2]. This parameterization does not rely on the reliability of the graph, which means these tensor contraction methods can determine reliability faster than Monte Carlo methods on highly reliable networks, while also providing exact answers or guaranteed upper bound estimates. These tensor contraction methods are applied to grid graphs, random cubic graphs, and a selection of 58 power transmission networks [3], demonstrating computational efficiency and effective approximation using EC(G). 

   more » « less
4. Network Schema Preserved Heterogeneous Information Network Embedding

   Zhao, Jianan ; Wang, Xiao ; Shi, Chuan ; Liu, Zekuan ; Ye, Yanfang. ( January 2020 , 29th International Joint Conference on Artificial Intelligence (IJCAI))
5. Network Schema Preserved Heterogeneous Information Network Embedding

   Zhao, Jianan ; Wang, Xiao ; Shi, Chuan ; Liu, Zekuan ; Ye, Yanfang ( January 2020 , 29th International Joint Conference on Artificial Intelligence (IJCAI))