Abstract

The manifold hypothesis is a widely accepted tenet of machine learning which asserts that nominally high-dimensional data are in fact concentrated around a low-dimensional manifold. In this talk, I will show some real examples of manifold structure occurring in science and in AI (internal representations of LLMs), and discuss associated research questions, particularly around how observed topology and geometry might map to the real world or human perceptions. I will present a statistical model and associated theory which explains how complex hidden manifold structure might emerge from simple statistical assumptions (e.g. latent variables, correlation, stationarity), exposing different possible mathematical relationships between the manifold and the ground truth (e.g. homeomorphism, isometry), and elucidating the efficacy of popular combinations of tools for data exploration (e.g. PCA followed by t-SNE).

Papers: Nick Whiteley, Annie Gray, Patrick Rubin-Delanchy. “Statistical exploration of the Manifold Hypothesis”. JRSSB (with discussion), to appear. Alexander Modell, Patrick Rubin-Delanchy, Nick Whiteley. “The Origins of Representation Manifolds in Large Language Models”, arXiv:2505.18235