We describe mechanisms for the creation and annihilation of spurious minima in a student-teacher network (ReLU activation). Methods involve ideas from dynamical systems and bifurcation theory, real analysis and real analytic geometry, and the representation theory of the symmetric group. This talk will be an overview of some recent results - a background in dynamical systems would be helpful. The work presented is part of an ongoing collaboration with Yossi Arjevani (School of Engineering and Computer Science, Hebrew University, Israel).