Abstract

Link to google meet: https://meet.google.com/wtz-cxne-shb to watch online.

Graph Neural Networks are becoming very popular as a way to model relational data with many applications. Nonetheless, there are two problems that frequently appear when dealing with GNNs: they perform poorly on heterophilic data and they exhibit over- smoothing behaviour. The first problem emerges because most models assume homophily (similar nodes tend to be connected) and the second one arises from deep GNNs tending to produce features too smooth in order to be useful.

Sheaf Neural Networks were proposed to address the problems described above. These equip each node and edge with a vector space and a linear application between these spaces for each incident edge-node pair. This gives the graph a non-trivial diffusion operator that may not have the aforementioned issues. Nonetheless, the way this extra structure is computed leaves little room for interpretation of the overall model.

Consequently, in this talk we will explore sheaf neural networks and other GNN ’s through the lens of opinion dynamics, namely the study of the evolution of opinions through (mostly) ODEs. This theoretical framework will allow us to interpret the sheaf in a very natural way. We will also explore the direct incorporation of the opinion dynamics diffusion processes into new neural network architectures such as Joint diffusion Sheaf Neural Networks and Rotation invariant Sheaf Neural Networks. On top of that, new data generation methods to evaluate sheaf based models will be proposed. Overall we show that our new SNN variants may have a more fitting inductive bias towards heterophilic data as well as the ability to detect and account for long range correlations between nodes