More Like this

1. BiTe-GCN: A New GCN Architecture via Bidirectional Convolution of Topology and Features on Text-Rich Networks

   https://doi.org/10.1145/3437963.3441774  

   Jin, Di ; Song, Xiangchen ; Yu, Zhizhi ; Liu, Ziyang ; Zhang, Heling ; Cheng, Zhaomeng ; Han, Jiawei ( March 2021 , WSDM'21, The Fourteenth ACM International Conference on Web Search and Data Mining, March 2021)

   Liane Lewin-Eytan, David Carmel (Ed.)

   Graph convolutional networks (GCNs), aiming to obtain node embeddings by integrating high-order neighborhood information through stacked graph convolution layers, have demonstrated great power in many network analysis tasks such as node classification and link prediction. However, a fundamental weakness of GCNs, that is, topological limitations, including over-smoothing and local homophily of topology, limits their ability to represent networks. Existing studies for solving these topological limitations typically focus only on the convolution of features on network topology, which inevitably relies heavily on network structures. Moreover, most networks are text-rich, so it is important to integrate not only document-level information, but also the local text information which is particularly significant while often ignored by the existing methods. To solve these limitations, we propose BiTe-GCN, a novel GCN architecture modeling via bidirectional convolution of topology and features on text-rich networks. Specifically, we first transform the original text-rich network into an augmented bi-typed heterogeneous network, capturing both the global document-level information and the local text-sequence information from texts.We then introduce discriminative convolution mechanisms, which performs convolution on this augmented bi-typed network, realizing the convolutions of topology and features altogether in the same system, and learning different contributions of these two parts (i.e., network part and text part), automatically for the given learning objectives. Extensive experiments on text-rich networks demonstrate that our new architecture outperforms the state-of-the-arts by a breakout improvement. Moreover, this architecture can also be applied to several e-commerce search scenes such as JD searching, and experiments on JD dataset show the superiority of the proposed architecture over the baseline methods. 

   more » « less
2. Single serotonergic axons: From a convolutional neural network for trajectory analysis to a neuroscience-inspired dropout in machine learning

   Madinei, P. ; Mays, Kasie C. ; Janusonis, Skirmantas ( November 2022 , Abstracts Society for Neuroscience)

   The brain serotonergic axons (fibers) are quintessential “stochastic” axons in the sense that their individual trajectories are best described as sample paths of a spatial stochastic process. These fibers are present in high densities in virtually all regions of vertebrate brains; more generally, they appear to be an obligatory component of all nervous systems on this planet (from the dominating arthropods to such small phyla as the kinorhynchs). In mammals, serotonergic fibers are nearly unique in their ability to robustly regenerate in the adult brain, and they have been strongly associated with neural plasticity. We have recently developed several experimental approaches to trace individual serotonergic fibers in the mouse brain (Mays et al., 2022). To further advance the theoretical analyses of their stochastic properties (e.g., the increment covariance structure), we developed a convolutional neural network (CNN) that performs high-throughput analysis of experimental data collected with sub-micrometer resolution. In contrast to a recently developed mesoscale platform that can separate large-caliber fiber segments from the background on the whole-brain scale (Friedmann et al., 2020), our microscale model prioritizes the accuracy and continuity of individual fiber trajectories, an essential element in downstream stochastic analyses. In particular, it seamlessly integrates information about the physical properties of serotonergic fibers and high-resolution experimental data to achieve reliable, fully-automated tracing of trajectories in brain regions with different fiber densities. This 3D-spatial information supports our current theoretical frameworks based on step-wise random walks (Janusonis & Detering, 2019) and continuous-time processes (Janusonis et al., 2020). In a complementary approach, we also investigated whether the structure of the serotonergic fibers may provide useful information for machine learning architectures. Specifically, we studied whether dropout, a standard regularization technique in artificial neural networks, can be matched or improved by virtual serotonergic fibers moving through CNN layers (endowed with the Euclidean metric) and triggering spatially correlated dropout events. This research was funded by NSF CRCNS (#1822517 and #2112862), NIMH (#MH117488), and the California NanoSystems Institute. 

   more » « less
3. Self-supervised Feature Learning by Cross-modality and Cross-view Correspondences

   Jing, L ; Zhang, L ; Tian, Y ( June 2021 , The 4th Multimodal Learning and Applications (MULA) Workshop in conjunction with CVPR 2021)

   null (Ed.)

   The success of supervised learning requires large-scale ground truth labels which are very expensive, time- consuming, or may need special skills to annotate. To address this issue, many self- or un-supervised methods are developed. Unlike most existing self-supervised methods to learn only 2D image features or only 3D point cloud features, this paper presents a novel and effective self-supervised learning approach to jointly learn both 2D image features and 3D point cloud features by exploiting cross-modality and cross-view correspondences without using any human annotated labels. Specifically, 2D image features of rendered images from different views are extracted by a 2D convolutional neural network, and 3D point cloud features are extracted by a graph convolution neural network. Two types of features are fed into a two-layer fully connected neural network to estimate the cross-modality correspondence. The three networks are jointly trained (i.e. cross-modality) by verifying whether two sampled data of different modalities belong to the same object, meanwhile, the 2D convolutional neural network is additionally optimized through minimizing intra-object distance while maximizing inter-object distance of rendered images in different views (i.e. cross-view). The effectiveness of the learned 2D and 3D features is evaluated by transferring them on five different tasks including multi-view 2D shape recognition, 3D shape recognition, multi-view 2D shape retrieval, 3D shape retrieval, and 3D part-segmentation. Extensive evaluations on all the five different tasks across different datasets demonstrate strong generalization and effectiveness of the learned 2D and 3D features by the proposed self-supervised method. 

   more » « less
4. PointConv: Deep Convolutional Networks on 3D Point Clouds

   https://doi.org/10.1109/CVPR.2019.00985  

   Wu, Wenxuan ; Qi, Zhongang ; Fuxin, Li ( June 2019 , 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR))

   Unlike images which are represented in regular dense grids, 3D point clouds are irregular and unordered, hence applying convolution on them can be difficult. In this paper, we extend the dynamic filter to a new convolution operation, named PointConv. PointConv can be applied on point clouds to build deep convolutional networks. We treat convolution kernels as nonlinear functions of the local coordinates of 3D points comprised of weight and density functions. With respect to a given point, the weight functions are learned with multi-layer perceptron networks and the density functions through kernel density estimation. A novel reformulation is proposed for efficiently computing the weight functions, which allowed us to dramatically scale up the network and significantly improve its performance. The learned convolution kernel can be used to compute translation-invariant and permutation-invariant convolution on any point set in the 3D space. Besides, PointConv can also be used as deconvolution operators to propagate features from a subsampled point cloud back to its original resolution. Experiments on ModelNet40, ShapeNet, and ScanNet show that deep convolutional neural networks built on PointConv are able to achieve state-of-the-art on challenging semantic segmentation benchmarks on 3D point clouds. Besides, our experiments converting CIFAR-10 into a point cloud showed that networks built on PointConv can match the performance of convolutional networks in 2D images of a similar structure. 

   more » « less
5. TEDIC: Neural Modeling of Behavioral Patterns in Dynamic Social Interaction Networks

   https://doi.org/10.1145/3442381.3450096  

   Wang, Yanbang ; Li, Pan ; Bai, Chongyang ; Leskovec, Jure ( January 2021 , WWW '21: Proceedings of the Web Conference 2021)

   null (Ed.)

   Dynamic social interaction networks are an important abstraction to model time-stamped social interactions such as eye contact, speaking and listening between people. These networks typically contain informative while subtle patterns that reflect people’s social characters and relationship, and therefore attract the attentions of a lot of social scientists and computer scientists. Previous approaches on extracting those patterns primarily rely on sophisticated expert knowledge of psychology and social science, and the obtained features are often overly task-specific. More generic models based on representation learning of dynamic networks may be applied, but the unique properties of social interactions cause severe model mismatch and degenerate the quality of the obtained representations. Here we fill this gap by proposing a novel framework, termed TEmporal network-DIffusion Convolutional networks (TEDIC), for generic representation learning on dynamic social interaction networks. We make TEDIC a good fit by designing two components: 1) Adopt diffusion of node attributes over a combination of the original network and its complement to capture long-hop interactive patterns embedded in the behaviors of people making or avoiding contact; 2) Leverage temporal convolution networks with hierarchical set-pooling operation to flexibly extract patterns from different-length interactions scattered over a long time span. The design also endows TEDIC with certain self-explaining power. We evaluate TEDIC over five real datasets for four different social character prediction tasks including deception detection, dominance identification, nervousness detection and community detection. TEDIC not only consistently outperforms previous SOTA’s, but also provides two important pieces of social insight. In addition, it exhibits favorable societal characteristics by remaining unbiased to people from different regions. Our project website is: http://snap.stanford.edu/tedic/. 

   more » « less