Abstract

Would Kepler have discovered his laws if machine learning had been around in 1609? Or would he have been satisfied with the accuracy of some black box regression model, leaving Newton without the inspiration to find the law of gravitation? In this talk I will present a quick introduction to machine learning and two major issues facing its application in the sciences: (1) a lack of interpretability, and (2) no physical priors. I will then present new potential solutions to each of these. For (1), I will introduce a method for translating a neural network into a symbolic model, using symbolic regression techniques such as PySR (github.com/MilesCranmer/PySR). Then, for (2), I will discuss the idea behind “foundation models,” which are large, general models pre-trained on vast amounts of data, endowing them with strong general priors – such as ChatGPT and Stable Diffusion. I will present “Polymathic AI,” a new research collaboration (polymathic-ai.org) which aims to build an analogous foundation model for scientific data, and describe our recent work.