Speaker
Description
Gradient descent methods are a popular choice for minimizing the approximation error of different neural network models. Understanding their trajectories can be used to study implicit bias or the existence of spurious minima in the optimization process. In this work we study the polynomials defining algebraic varieties that contain the trajectories of the Gradient flow for deep linear neural networks. These polynomials are invariants of the gradient flow. We use combinatorial methods to compute the number of independent invariants that are satisfied by all flows and differentiable loss functions. This computation relies on an identification of the neural networks with quiver representations, and becomes tractable by the introduction of a double Poisson bracket related to the quadrupling of the underlying quiver.
