Title: Representing Joint Hierarchies with Box Embeddings
Learning representations for hierarchical and multi-relational knowledge has emerged as an active area of research. Box Embeddings [Vilnis et al., 2018, Li et al., 2019] represent concepts with hyperrectangles in -dimensional space and are shown to be capable of modeling tree-like structures efficiently by training on a large subset of the transitive closure of the WordNet hypernym graph. In this work, we evaluate the capability of box embeddings to learn the transitive closure of a tree-like hierarchical relation graph with far fewer edges from the transitive closure. Box embeddings are not restricted to tree-like structures, however, and we demonstrate this by modeling the WordNet meronym graph, where nodes may have multiple parents. We further propose a method for modeling multiple relations jointly in a single embedding space using box embeddings. In all cases, our proposed method outperforms or is at par with all other embedding methods. more »« less
Chanpuriya, Sudhanshu; Musco, Cameron; Sotiropoulos, Konstantinos; Tsourakakis, Charalampos E.(
, International Conference on Machine Learning (ICML))
null
(Ed.)
Low-dimensional node embeddings play a key role in analyzing graph datasets. However, little work studies exactly what information is encoded by popular embedding methods, and how this information correlates with performance in downstream machine learning tasks. We tackle this question by studying whether embeddings can be inverted to (approximately) recover the graph used to generate them. Focusing on a variant of the popular DeepWalk method (Perozzi et al., 2014; Qiu et al., 2018), we present algorithms for accurate embedding inversion - i.e., from the low-dimensional embedding of a graph G, we can find a graph H with a very similar embedding. We perform numerous experiments on real-world networks, observing that significant information about G, such as specific edges and bulk properties like triangle density, is often lost in H. However, community structure is often preserved or even enhanced. Our findings are a step towards a more rigorous understanding of exactly what information embeddings encode about the input graph, and why this information is useful for learning tasks.
Hyperbolic neural networks have been popular in the re- cent past due to their ability to represent hierarchical data sets effectively and efficiently. The challenge in develop- ing these networks lies in the nonlinearity of the embed- ding space namely, the Hyperbolic space. Hyperbolic space is a homogeneous Riemannian manifold of the Lorentz group which is a semi-Riemannian manifold, i.e. a mani- fold equipped with an indefinite metric. Most existing meth- ods (with some exceptions) use local linearization to de- fine a variety of operations paralleling those used in tra- ditional deep neural networks in Euclidean spaces. In this paper, we present a novel fully hyperbolic neural network which uses the concept of projections (embeddings) fol- lowed by an intrinsic aggregation and a nonlinearity all within the hyperbolic space. The novelty here lies in the projection which is designed to project data on to a lower- dimensional embedded hyperbolic space and hence leads to a nested hyperbolic space representation independently useful for dimensionality reduction. The main theoretical contribution is that the proposed embedding is proved to be isometric and equivariant under the Lorentz transforma- tions, which are the natural isometric transformations in hyperbolic spaces. This projection is computationally effi- cient since it can be expressed by simple linear operations, and, due to the aforementioned equivariance property, it al- lows for weight sharing. The nested hyperbolic space rep- resentation is the core component of our network and there- fore, we first compare this representation – independent of the network – with other dimensionality reduction methods such as tangent PCA, principal geodesic analysis (PGA) and HoroPCA. Based on this equivariant embedding, we develop a novel fully hyperbolic graph convolutional neural network architecture to learn the parameters of the projec- tion. Finally, we present experiments demonstrating com- parative performance of our network on several publicly available data sets.
All-solid-state batteries (ASSBs) have garnered increasing attention due to the enhanced safety, featuring nonflammable solid electrolytes as well as the potential to achieve high energy density. 1 The advancement of the ASSBs is expected to provide, arguably, the most straightforward path towards practical, high-energy, and rechargeable batteries based on metallic anodes. 1 However, the sluggish ion transmission at the cathode-electrolyte (solid/solid) interface would result in the high resistant at the contact and limit the practical implementation of these all solid-state materials in real world batteries. 2 Several methods were suggested to enhance the kinetic condition of the ion migration between the cathode and the solid electrolyte (SE). 3 A composite strategy that mixes active materials and SEs for the cathode is a general way to decrease the ion transmission barrier at the cathode-electrolyte interface. 3 The active material concentration in the cathode is reduced as much as the SE portion increases by which the energy density of the ASSB is restricted. In addition, the mixing approach generally accompanies lattice mismatches between the cathode active materials and the SE, thus providing only limited improvements, which is imputed by random contacts between the cathode active materials and the SE during the mixing process. Implementing high-pressure for the electrode and electrolyte of ASSB in the assembling process has been verified is a but effective way to boost the ion transmission ability between the cathode active materials and the SE by decreasing the grain boundary impedance. Whereas the short-circuit of the battery would be induced by the mechanical deformation of the electrolyte under high pressure. 4 Herein, we demonstrate a novel way to address the ion transmission problem at the cathode-electrolyte interface in ASSBs. Starting from the cathode configuration, the finite element method (FEM) was employed to evaluate the current concentration and the distribution of the space charge layer at the cathode-electrolyte interface. Hierarchical three-dimensional (HTD) structures are found to have a higher Li + transfer number (t Li+ ), fewer free anions, and the weaker space-charge layer at the cathode-electrolyte interface in the resulting FEM simulation. To take advantage of the HTD structure, stereolithography is adopted as a manufacturing technique and single-crystalline Ni-rich (SCN) materials are selected as the active materials. Next, the manufactured HTD cathode is sintered at 600 °C in an N 2 atmosphere for the carbonization of the resin, which induces sufficient electronic conductivity for the cathode. Then, the gel-like Li 1.4 Al 0.4 Ti 1.6 (PO 4 ) 3 (LATP) precursor is synthesized and filled into the voids of the HTD structure cathode sufficiently. And the filled HTD structure cathodes are sintered at 900 °C to achieve the crystallization of the LATP gel. Scanning transmission electron microscopy (STEM) is used to unveil the morphology of the cathode-electrolyte interface between the sintered HTD cathode and the in-situ generated electrolyte (LATP). A transient phase has been found generated at the interface and matched with both lattices of the SCN and the SE, accelerating the transmission of the Li-ions, which is further verified by density functional theory calculations. In addition, Electron Energy Loss Spectroscopy demonstrates the preserved interface between HTD cathode and SEs. Atomic force microscopy is employed to measure the potential image of the cross-sectional interface by the peak force tapping mode. The average potential of modified samples is lower than the sample that mix SCN and SEs simply in the 2D planar structure, which confirms a weakened space charge layer by the enhanced contact capability as well as the ion transmission ability. To see if the demonstrated method is universally applicable, LiNi 0.8 Co 0.1 Mn 0.1 O 2 (NCM811) is selected as the cathode active material and manufactured in the same way as the SCN. The HTD cathode based on NCM811 exhibits higher electrochemical performance compared with the reference sample based on the 2D planar mixing-type cathode. We believe such a demonstrated universal strategy provides a new guideline to engineer the cathode/electrolyte interface by revolutionizing electrode structures that can be applicable to all-solid-state batteries. Figure 1. Schematic of comparing of traditional 2D planar cathode and HTD cathode in ASSB Tikekar, M. D. , et al. , Nature Energy (2016) 1 (9), 16114 Banerjee, A. , et al. , Chem Rev (2020) 120 (14), 6878 Chen, R. , et al. , Chem Rev (2020) 120 (14), 6820 Cheng, X. , et al. , Advanced Energy Materials (2018) 8 (7) Figure 1
A wide variety of machine learning tasks such as knowledge base completion, ontology alignment, and multi-label classification can benefit from incorporating into learning differentiable representations of graphs or taxonomies. While vectors in Euclidean space can theoretically represent any graph, much recent work shows that alternatives such as complex, hyperbolic, order, or box embeddings have geometric properties better suited to modeling real-world graphs. Experimentally these gains are seen only in lower dimensions, however, with performance benefits diminishing in higher dimensions. In this work, we introduce a novel variant of box embeddings that uses a learned smoothing parameter to achieve better representational capacity than vector models in low dimensions, while also avoiding performance saturation common to other geometric models in high dimensions. Further, we present theoretical results that prove box embeddings can represent any DAG. We perform rigorous empirical evaluations of vector, hyperbolic, and region-based geometric representations on several families of synthetic and real-world directed graphs. Analysis of these results exposes correlations between different families of graphs, graph characteristics, model size, and embedding geometry, providing useful insights into the inductive biases of various differentiable graph representations.
Bai, J; Ying, R; Ren, H; Leskovec, J(
, Advances in neural information processing systems)
Hierarchical relations are prevalent and indispensable for organizing human knowledge captured by a knowledge graph (KG). The key property of hierarchical relations is that they induce a partial ordering over the entities, which needs to be modeled in order to allow for hierarchical reasoning. However, current KG embeddings can model only a single global hierarchy (single global partial ordering) and fail to model multiple heterogeneous hierarchies that exist in a single KG. Here we present ConE (Cone Embedding), a KG embedding model that is able to simultaneously model multiple hierarchical as well as non-hierarchical relations in a knowledge graph. ConE embeds entities into hyperbolic cones and models relations as transformations between the cones. In particular, ConE uses cone containment constraints in different subspaces of the hyperbolic embedding space to capture multiple heterogeneous hierarchies. Experiments on standard knowledge graph benchmarks show that ConE obtains state-of-the-art performance on hierarchical reasoning tasks as well as knowledge graph completion task on hierarchical graphs. In particular, our approach yields new state-of-the-art Hits@1 of 45.3% on WN18RR and 16.1% on DDB14 (0.231 MRR). As for hierarchical reasoning task, our approach outperforms previous best results by an average of 20% across the three datasets.
Dhruvesh Patel, Shib Sankar. Representing Joint Hierarchies with Box Embeddings. Retrieved from https://par.nsf.gov/biblio/10188900. Automated Knowledge Base Construction .
Dhruvesh Patel, Shib Sankar.
"Representing Joint Hierarchies with Box Embeddings". Automated Knowledge Base Construction (). Country unknown/Code not available. https://par.nsf.gov/biblio/10188900.
@article{osti_10188900,
place = {Country unknown/Code not available},
title = {Representing Joint Hierarchies with Box Embeddings},
url = {https://par.nsf.gov/biblio/10188900},
abstractNote = {Learning representations for hierarchical and multi-relational knowledge has emerged as an active area of research. Box Embeddings [Vilnis et al., 2018, Li et al., 2019] represent concepts with hyperrectangles in -dimensional space and are shown to be capable of modeling tree-like structures efficiently by training on a large subset of the transitive closure of the WordNet hypernym graph. In this work, we evaluate the capability of box embeddings to learn the transitive closure of a tree-like hierarchical relation graph with far fewer edges from the transitive closure. Box embeddings are not restricted to tree-like structures, however, and we demonstrate this by modeling the WordNet meronym graph, where nodes may have multiple parents. We further propose a method for modeling multiple relations jointly in a single embedding space using box embeddings. In all cases, our proposed method outperforms or is at par with all other embedding methods.},
journal = {Automated Knowledge Base Construction},
author = {Dhruvesh Patel, Shib Sankar},
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
