Neural networks are highly non-linear functions often parametrized by a staggering number of parameters, called weights. They have been a subject of intense research due in large part to their remarkable success in a wide range of application areas. Miniaturizing these networks and implementing them in hardware is a direction of research that is fueled by a practical need, and at the same time connects to interesting mathematical problems. For example, by quantizing, or replacing the weights of a neural network with