Mathematics of Deep Learning Seminar: Francis Bach

Title: Towards a quantitative analysis of gradient descent for infinitely wide two-layer neural networks

Abstract: I will consider fully connected two-layer neural networks with a number of hidden neurons that tends to infinity. For these networks (and with additional technical conditions), it is known that gradient descent with infinitesimal steps on the risk will converge to its global optimum. However, current results are mostly qualitative, with no indication on the number of neurons to achieve the infinite-width limit, or the time to reach a global optimum. The main goal of the talk is to present this open problem. (Based on joint work with Lénaïc Chizat).