September 18, 2019, 11:30 AM - 12:10 PM
Busch Campus Student Center
604 Bartholomew Rd
Click here for map.
Zhenyu Liao, University of Paris - Saclay
Modern neural networks are commonly trained with gradient-based methods. The understanding of the dynamics of gradient descent algorithm is one of the key issues for the theoretical comprehension of why deep neural nets work so well today. In this work, we introduce a random matrix-based framework to analyze the dynamics of a simple toy network model trained by gradient descent. This preliminary result opens the door for future studies of more elaborate structures and models appearing in today’s neural networks.