Abstract

Many neural networks (NNs) are drawn from Gaussian processes in an appropriate infinite N limit. At large-but-finite N, the associated non-Gaussian processes may be treated using techniques from quantum field theory (QFT). One fundamental aspect of both NN and QFT distributions are their symmetries, which in this talk I will study via a duality between parameter space and function space. As in physics, via duality we can utilize one perspective to gain knowledge of the other. In this case, we can study NN correlation functions computed in parameter space to determine symmetries of the function space distribution, even when it is not known. Central results are obvious in the GP limit of IID -parameter neural networks, but the duality allows us to extend the symmetry results to all values of N and some independence-breaking schemes.