Abstract

Topic 1: A Review of Automated Scientific Discovery with ML (Yi Gu): Automated physics discovery using machine learning is an exciting research direction. A core validation challenge lies in whether machine learning systems can autonomously reproduce historically famous physics discoveries under the condition of zero prior knowledge. I will review and discuss a series of research efforts that validate this feasibility. We will see the progress achieved by machine learning at multiple key levels, including the formation of concepts, the discovery of equations, and the identification of physical laws and symmetries.

Topic 2: Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks (Luca Muscarnera): Understanding whether a Neural Network will generalize or not is crucial in high-stakes applications, and assessing the reliability of such models is, unfortunately, a non-trivial challenge. This article explains a mechanistic interpretability-based technique to address this problem, analyzing the statistical properties of the weight matrices in very large neural networks, without accessing the training data. After developing a suitable theoretical framework, the authors corroborate their claims by studying the correlation between their proposed metrics and the test error of large neural networks. In a nutshell, the article tries to answer a fundamental question: can we interpret the quality of learning in NNs without testing it? Discussing the paper “Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks” C. Martin, M. Mahoney (2020) https://arxiv.org/abs/1901.08278