University of Cambridge > Talks.cam > Language Technology Lab Seminars > Pretraining, Instruction Tuning, Alignment: Towards Building Large Language Models from First Principles

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹ️ Information

Pretraining, Instruction Tuning, Alignment: Towards Building Large Language Models from First Principles

Download to your calendar using vCal

If you have a question about this talk, please contact Panagiotis Fytas .

Recently, the field has been greatly impressed and inspired by Large Language Models (LLMs). LLMs’ multi-dimensional abilities are significantly beyond many AI researchers’ and practitioners’ expectations and thus reshaping the AI research paradigm. A natural question is how LLMs get there, and where these fantastic abilities come from. In this talk, we try to dissect the strong LLMs’ capabilities and trace them to their sources. We first review the generic recipe for building large language models from first principles. Then we discuss recipes for improving language models’ reasoning capabilities. Finally, we consider further improvements by complexity-based prompting, distilling chain-of-thought, and learning from AI feedback.

This talk is part of the Language Technology Lab Seminars series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

BAS Seminar Room 1; https://ukri.zoom.us/j/92110296123 Faculty of English, Room GR-06/07 Boys Smith Room, Fisher Building, St John's College

Other talks

Title to be confirmed CSAR lecture: In a post-Ozempic world, have we cured obesity? From lab to clinic: an introduction to translational research Talk TBC Online Causal Inference Seminar When Programmes Disagree About Programmes: Code Poetry and the Problem of Programme–Programme Communication
 

© 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity