Abstract

Transformers excel at natural language tasks, which naturally raises the question of whether they can also learn the mathematical language of particle physics. In this talk, I will introduce BART -L, a transformer-based model designed to generate particle physics Lagrangians from the field content and their symmetry information. Trained on Lagrangians respecting the Standard Model gauge group SU(3)×SU(2)×U(1), BART -L achieves over 90 % accuracy for Lagrangians involving up to six matter fields. Embedding analyses show that the model internalizes concepts such as group representations and conjugation operations, despite not being explicitly trained for them. We further examine its out-of-distribution behavior to identify architectural constraints that limit generalization. Finally, we discuss how this framework provides an early indication of what a foundation model for theoretical high-energy physics might look like, along with its potential capabilities and inherent limitations.