University of Cambridge > Talks.cam > Applied and Computational Analysis > Feature Learning in Two-layer Neural Networks: The Effect of Data Covariance

  Browse A-Z

Log in

Googledetails Microsoftdetails

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Feature Learning in Two-layer Neural Networks: The Effect of Data Covariance

Download to your calendar using vCal

If you have a question about this talk, please contact Nicolas Boulle .

We study the effect of gradient-based optimization on feature learning in two-layer neural networks. We consider a setting where the number of samples is of the same order as the input dimension and show that, when the input data is isotropic, gradient descent always improves upon the initial random features model in terms of prediction risk, for a certain class of targets. Further leveraging the practical observation that data often contains additional structure, i.e., the input covariance has non-trivial alignment with the target, we prove that the class of learnable targets can be significantly extended, demonstrating a clear separation between kernel methods and two-layer neural networks in this regime.

This talk is part of the Applied and Computational Analysis series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

FW11, William Gates Building. Zoom link: https://cl-cam-ac-uk.zoom.us/j/4361570789?pwd=Nkl2T3ZLaTZwRm05bzRTOUUxY3Q4QT09&from=addon KH107, Keynes House, Judge Business School Magdalene College, Magdalene Street

Other talks

Animating urbanism: the demotic multiplicities of seeing cities digitally TBC Optimal metric and approximation of free boundary minimal surfaces LCLU Annual Talk: The Early Aqueous Environment of Mars Inferred From Mission Lifetime Results by the Curiosity Rover at Gale Crater Talk TBC The role of radiation in cancer care: a spotlight on cancers of the oesophagus, head and neck
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity