Weisfeiler and Lehman use Simplicial Sets: Psuedotop Vertex Neural Networks
Date:
Abstract: Graph neural networks are paradigms of computation that yield powerful results for structured data based on binary relationships. However, they are limited in their expressivity by the Weisfeiler-Lehman test of for graph isomorphism. The core idea behind machine learning community's circumvention of this limitation relies on identifying (and working with) higher relationships within the data. In this talk, we put forward an architecture closely based on the identification of such higher relationships via Kan Extensions of structured data built on binary relations. We will talk about its theoretical underpinnings based on the combinatorics of simplicial sets, and based off of it, introduce the notion of a pseudotop vertex as a proxy for these higher relations. We talk about how this choice respects the variance and bias trade off necessary for generalizability of the architecture.