Abstract

Matrix factorization approaches to relation extraction provide several attractive features: they support distant supervision, handle open schemas, and leverage unlabeled data. Such models learn dense fixed-length vector representations (embeddings) of binary relations and entity-pairs. Inference of an unseen factual statement amounts to a simple efficient dot product between the corresponding relation and entity-pair vector, making these models highly scalable. However, it is unclear to what extent models based on distributed representations support complex reasoning as enabled, for instance, by symbolic representations such as first-order logic. Moreover, distributed representations are hard to debug and it is not clear how symbolic background knowledge can be incorporated into such models.

Rule-based extractors, on the other hand, can be easily extended to novel relations and improved for existing but inaccurate relations, through first-order formulae that capture auxiliary domain knowledge. However, usually a large set of such formulae is necessary to achieve generalization.

In this talk, I will introduce a paradigm for learning low-dimensional embeddings of entity-pairs and relations that combine the advantages of matrix factorization with first-order logic domain knowledge. Specifically, I will introduce a novel training algorithm to jointly optimize over factual and first-order logic information and talk about our ongoing research in this direction.