A Random Matrix Viewpoint of Learning with Gradient Descent

September 18, 2019, 11:30 AM - 12:10 PM


Center Hall

Rutgers University

Busch Campus Student Center

604 Bartholomew Rd

Piscataway NJ

Click here for map.

Zhenyu Liao, University of Paris - Saclay

Modern neural networks are commonly trained with gradient-based methods. The understanding of the dynamics of gradient descent algorithm is one of the key issues for the theoretical comprehension of why deep neural nets work so well today. In this work, we introduce a random matrix-based framework to analyze the dynamics of a simple toy network model trained by gradient descent. This preliminary result opens the door for future studies of more elaborate structures and models appearing in today’s neural networks.