We consider optimization with many parameters in the context of statistical mechanics. A direct analogy to thermodynamic systems from physics is not drawn, instead we use the fundamental idea of entropy and its relation to noise in the datasets to demonstrate a transition with an example dataset. As a result, we obtain a critical value of noise for which an image classification is possible. This is discussed together with finite size and finite time effects, i.e. a finite number of nodes in a network and a finite training time. The consequences may be relevant for many design questions and practical purposes, like the determination of a network size given a certain complexity of the AI problem to be solved.