Abstract

Many problems across computer vision and the natural sciences require the analysis of spherical data. For example, the cosmic microwave background (CMB) relic radiation from the Big Bang is acquired on the sphere, as are 360° photos and videos that are prevalent in virtual reality. To leverage the potential of deep learning in these areas and others, geometric deep learning techniques defined natively on the sphere are required. In this talk I will discuss connections between physics and deep learning, and in particular the role of symmetry in representation learning. I will highlight the importance of encoding the symmetries and geometric properties of the sphere in spherical deep learning constructions, in order to capture rotational equivariance so that representations may be learned effectively. I will discuss various frameworks for constructing spherical convolutional neural networks (CNNs), including continuous, discrete and hybrid approaches, and their pros and cons. I will concisely review our recent works on Efficient Generalized Spherical CNNs ( https://arxiv.org/abs/2010.11661), Scattering Networks on the Sphere ( https://arxiv.org/abs/2102.02828), and Scalable and Equivariant Spherical CNNs (yet to be published). I will present the application of our frameworks to numerous spherical deep learning benchmark tasks, on all of which we achieve the state-of-the-art performance.