Fast prototyping of graph neural networks with the ignnition framework
Graphs are a fundamental data type that enables to represent in a well-structured manner many objects and problems of real-life; particularly those involving a set of elements that interact in some way with each other (i.e., relational information). Thus, they are widely used in fields where informa...
| Autor: | |
|---|---|
| Formato: | tesis de maestría |
| Fecha de publicación: | 2021 |
| País: | España |
| Recursos: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/353506 |
| Acesso em linha: | https://hdl.handle.net/2117/353506 |
| Access Level: | acceso abierto |
| Palavra-chave: | Machine learning Neural networks (Computer science) Artificial intelligence aprenentatge automàtic gnn grafs ignnition transmissió de missatges xarxa neuronal de transmissió de missatges xarxes neuronals sobre grafs machine learning message passing message passing neural network neural networks Aprenentatge automàtic Xarxes neuronals (Informàtica) Intel·ligència artificial Àrees temàtiques de la UPC::Informàtica |
| Resumo: | Graphs are a fundamental data type that enables to represent in a well-structured manner many objects and problems of real-life; particularly those involving a set of elements that interact in some way with each other (i.e., relational information). Thus, they are widely used in fields where information is mainly relational, such as communication networks, chemistry, physics, biology, or recommendation systems. In order to efficiently build statistical models that can understand and process graph-structured information, a new paradigm of models leveraging Deep Neural Networks arises, called Graph Neural Networks (GNN), with proven efficiency in previous works. The current state of the art of GNN, however, is based on domain-specific architectures and implementations. To unify research on the field and democratize the use of GNNs, in this thesis we present the novel iGNNition framework, recently built by a group of researchers of Universitat Politècnica de Catalunya (UPC-Barcelona Tech) in the context of a European project. This framework is specifically intended to enable fast prototyping of GNNs, taking as a reference a general description of GNNs called Message Passing Neural Networks (MPNN). We describe both the general design, with some popular applications in the literature, as well as the framework's working principles; including several contributions made to this open-source project in the context of this Master's Thesis. To emphasize the benefits of the framework, we build an alternative custom GNN library through direct TensorFlow implementation to discuss the intricacies and problematics of GNN model building, comparing the performance of native TensorFlow implementations to that of the implementations produced by iGNNition, by implementing two use cases of increased difficulty: one simpler supervised model applied to quantum chemistry, formulated as a graph-level regression problem; and a harder self-supervised model applied to radio resource management in wireless networks, formulated as a node-level regression problem. We conclude that both types of implementations (native TensorFlow, and iGNNition) are equivalent for the simpler model, while the harder model shows how the iGNNition framework captures more accurately the relations in the graph samples, due to the better default hyper-parameter selection of iGNNition. Moreover, we theorize a bottleneck in the data pipeline of iGNNition, which makes it considerably slower when dealing with large input graphs, compared to native TensorFlow implementations. This raises a relevant issue to address in the future development of this framework. Finally, we propose a change over the design of the use cases presented and evaluate the improvement of the models, obtaining a more accurate model over the simpler use case, and a better model training over the harder. |
|---|