Abstract

Abstract reasoning, particularly in the visual domain, is a complex human ability, but it remains a challenging problem for artificial neural learning systems. In this work we propose MXG Net, a multilayer graph neural network for multi-panel diagrammatic reasoning tasks. MXG Net combines three powerful concepts, namely, object-level representation, graph neural networks and multiplex graphs, for solving visual reasoning tasks. MXG Net first extracts object-level representations for each element in all panels of the diagrams, and then forms a multi-layer multiplex graph capturing multiple relations between objects across different diagram panels. MXG Net summarises the multiple graphs extracted from the diagrams of the task, and uses this summarisation to pick the most probable answer from the given candidates. We have tested MXG Net on two types of diagrammatic reasoning tasks, namely Diagram Syllogisms and Raven Progressive Matrices (RPM). For an Euler Diagram Syllogism task MXG Net achieves state-of-the-art accuracy of 99.8%. For PGM and RAVEN , two comprehensive datasets for RPM reasoning, MXG Net outperforms the state-of-the-art models by a considerable margin.

This work will be presented at ICLR .