University of Cambridge > Talks.cam > Language Technology Lab Seminars > Rethinking Benchmarking in AI

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Rethinking Benchmarking in AI

Download to your calendar using vCal

If you have a question about this talk, please contact Marinela Parovic .

The current benchmarking paradigm in AI has many issues: benchmarks saturate quickly, are susceptible to overfitting, contain exploitable annotator artifacts, have unclear or imperfect evaluation metrics, and do not measure what we really care about. I will talk about my work in trying to rethink the way we do benchmarking in AI, specifically in natural language processing, focusing mostly on the recently launched Dynabench platform.

This talk is part of the Language Technology Lab Seminars series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

SigProC meeting room (Kitchen, 3rd floor of Dept. of Engineering) Darwin College Island Bateman Auditorium, Gonville & Caius College, Cambridge, CB2 1TA

Other talks

John Kendrew Lecture - Title TBC Coming soon Sera Markoff Inaugural Lecture for the Plumian Professorship in Astronomy and Experimental Philosophy - 'Staring into the heart of darkness: from theory to the direct imaging of black holes' Machine Politics? Computers and AI in the History of Political Thought Online Causal Inference Seminar Proteomics: Expression Analysis and Subcellular Localisation
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity