Graph Neural Networks with Learnable Structural & Positional Representations | Vijay Prakash Dwivedi скачать в хорошем качестве

Graph Neural Networks with Learnable Structural & Positional Representations | Vijay Prakash Dwivedi

Join the Learning on Graphs and Geometry Reading Group: https://hannes-stark.com/logag-readin... Paper "Graph Neural Networks with Learnable Structural and Positional Representations": https://arxiv.org/abs/2110.07875 Abstract: Graph neural networks (GNNs) have become the standard learning architectures for graphs. GNNs have been applied to numerous domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural language processing. A major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish e.g. isomorphic nodes and other graph symmetries. An approach to tackle this issue is to introduce Positional Encoding (PE) of nodes, and inject it into the input layer, like in Transformers. Possible graph PE are Laplacian eigenvectors. In this work, we propose to decouple structural and positional representations to make easy for the network to learn these two essential properties. We introduce a novel generic architecture which we call LSPE (Learnable Structural and Positional Encodings). We investigate several sparse and fully-connected (Transformer-like) GNNs, and observe a performance increase for molecular datasets, from 2.87% up to 64.14% when considering learnable PE for both GNN classes. Authors: Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, Xavier Bresson Twitter Hannes:   / hannesstaerk   Twitter Dominique:   / dom_beaini   Twitter Valence Discovery:   / valence_ai   Reading Group Slack: https://logag.slack.com/join/shared_i... ~ 00:00 Intro 00:18 Graph Neural Networks with Learnable Structural and Positional Presentations 03:00 Motivation 05:18 Background 21:01 Learnable Structural and Positional Encodings 57:18 Numerical Evaluations 01:09:20 Conclusion 01:10:22 Q&A