Bruno Loureiro, École Normale Supérieure

Abstract

Feature learning - or the capacity of neural networks to adapt to the data during training - is often quoted as one of the fundamental reasons behind their unreasonable effectiveness. Yet, making mathematical sense of this seemingly clear intuition is still a largely open question. In this talk, I will discuss a simple setting where we can precisely characterize how features are learned by a two-layer neural network during the very first few steps of training, and how these features are essential for the network to efficiently generalize under limited availability of data.

Based on the following works: https://arxiv.org/abs/2305.18270, https://arxiv.org/abs/2402.04980   

[Video]    [Slides]