More Like this

1. Open Graph Benchmark: Datasets for Machine Learning on Graphs

   Hu, Weihua; Fey, Matthias; Zitnik, Marinka; Dong, Yuxiao; Ren, Hongyu; Liu, Bowen; Catasta, Michele; Leskovec, Jure. (January 2020, Neural Information Processing Systems (NeurIPS))

   We present the OPEN GRAPH BENCHMARK (OGB), a diverse set of challenging and realistic benchmark datasets to facilitate scalable, robust, and reproducible graph machine learning (ML) research. OGB datasets are large-scale, encompass multiple important graph ML tasks, and cover a diverse range of domains, ranging from social and information networks to biological networks, molecular graphs, source code ASTs, and knowledge graphs. For each dataset, we provide a unified evaluation protocol using meaningful application-specific data splits and evaluation metrics. In addition to building the datasets, we also perform extensive benchmark experiments for each dataset. Our experiments suggest that OGB datasets present significant challenges of scalability to large-scale graphs and out-of-distribution generalization under realistic data splits, indicating fruitful opportunities for future research. Finally, OGB provides an automated end-to-end graph ML pipeline that simplifies and standardizes the process of graph data loading, experimental setup, and model evaluation. OGB will be regularly updated and welcomes inputs from the community. OGB datasets as well as data loaders, evaluation scripts, baseline code, and leaderboards are publicly available at https://ogb.stanford.edu. 

   more » « less
2. PROXI: Challenging the GNNs for Link Prediction

   Tola, Astrit; Myrick, Jack; Coskunuzer, Baris (February 2025, Transactions on machine learning research)

   Over the past decade, Graph Neural Networks (GNNs) have transformed graph representation learning. In the widely adopted message-passing GNN framework, nodes refine their representations by aggregating information from neighboring nodes iteratively. While GNNs excel in various domains, recent theoretical studies have raised concerns about their capabilities. GNNs aim to address various graph-related tasks by utilizing such node representations, however, this one-size-fits-all approach proves suboptimal for diverse tasks. Motivated by these observations, we conduct empirical tests to compare the performance of current GNN models with more conventional and direct methods in link prediction tasks. Introducing our model, PROXI, which leverages proximity information of node pairs in both graph and attribute spaces, we find that standard machine learning (ML) models perform competitively, even outperforming cutting-edge GNN models when applied to these proximity metrics derived from node neighborhoods and attributes. This holds true across both homophilic and heterophilic networks, as well as small and large benchmark datasets, including those from the Open Graph Benchmark (OGB). Moreover, we show that augmenting traditional GNNs with PROXI significantly boosts their link prediction performance. Our empirical findings corroborate the previously mentioned theoretical observations and imply that there exists ample room for enhancement in current GNN models to reach their potential. 

   more » « less
3. Reconstruction for Powerful Graph Representations

   Cotta, Leonardo; Morris, Christopher; Ribeiro, Bruno (October 2021, Advances in neural information processing systems)

   Graph neural networks (GNNs) have limited expressive power, failing to represent many graph classes correctly. While more expressive graph representation learning (GRL) alternatives can distinguish some of these classes, they are significantly harder to implement, may not scale well, and have not been shown to outperform well-tuned GNNs in real-world tasks. Thus, devising simple, scalable, and expressive GRL architectures that also achieve real-world improvements remains an open challenge. In this work, we show the extent to which graph reconstruction---reconstructing a graph from its subgraphs---can mitigate the theoretical and practical problems currently faced by GRL architectures. First, we leverage graph reconstruction to build two new classes of expressive graph representations. Secondly, we show how graph reconstruction boosts the expressive power of any GNN architecture while being a (provably) powerful inductive bias for invariances to vertex removals. Empirically, we show how reconstruction can boost GNN's expressive power---while maintaining its invariance to permutations of the vertices---by solving seven graph property tasks not solvable by the original GNN. Further, we demonstrate how it boosts state-of-the-art GNN's performance across nine real-world benchmark datasets. 

   more » « less
4. Improving deep learning model performance under parametric constraints for materials informatics applications

   https://doi.org/10.1038/s41598-023-36336-5  

   Gupta, Vishu; Peltekian, Alec; Liao, Wei-keng; Choudhary, Alok; Agrawal, Ankit (December 2023, Scientific Reports)

   Abstract Modern machine learning (ML) and deep learning (DL) techniques using high-dimensional data representations have helped accelerate the materials discovery process by efficiently detecting hidden patterns in existing datasets and linking input representations to output properties for a better understanding of the scientific phenomenon. While a deep neural network comprised of fully connected layers has been widely used for materials property prediction, simply creating a deeper model with a large number of layers often faces with vanishing gradient problem, causing a degradation in the performance, thereby limiting usage. In this paper, we study and propose architectural principles to address the question of improving the performance of model training and inference under fixed parametric constraints. Here, we present a general deep-learning framework based on branched residual learning (BRNet) with fully connected layers that can work with any numerical vector-based representation as input to build accurate models to predict materials properties. We perform model training for materials properties using numerical vectors representing different composition-based attributes of the respective materials and compare the performance of the proposed models against traditional ML and existing DL architectures. We find that the proposed models are significantly more accurate than the ML/DL models for all data sizes by using different composition-based attributes as input. Further, branched learning requires fewer parameters and results in faster model training due to better convergence during the training phase than existing neural networks, thereby efficiently building accurate models for predicting materials properties. 

   more » « less
5. Adaptive Joint Spatio-Temporal Graph Learning Network for Traffic Data Forecasting

   https://doi.org/10.1145/3634913  

   Wang, Tianyi; Chen, Shu-Ching (September 2024, ACM Transactions on Spatial Algorithms and Systems)

   Traffic data forecasting has become an integral part of the intelligent traffic system. Great efforts are spent developing tools and techniques to estimate traffic flow patterns. Many existing approaches lack the ability to model the complex and dynamic spatio-temporal relations in the traffic data, which are crucial in capturing the traffic dynamic. In this work, we propose AJSTGL, a novel adaptive joint spatio-temporal graph learning network for traffic data forecasting. The proposed model utilizes static and adaptive graph learning modules to capture the static and dynamic spatial traffic patterns and optimize the graph learning process. A sequence-to-sequence fusion model is proposed to learn the temporal correlation and combine the output of multiple parallelized encoders. We also develop a spatio-temporal graph transformer module to complement the sequence-to-sequence fusion module by dynamically capturing the time-evolving node relations in long-term intervals. Experiments on three large-scale traffic flow datasets demonstrate that our model could outperform other state-of-the-art baseline methods. 

   more » « less