We present Newton-like and quasi-Newton methods for both convex and non-convex optimization problems whose objective functions can be expressed as the sum of a huge number of functions in an extremely large number of variables. In particular, we propose new ways to use the BFGS and limited memory BFGS updating methods in this setting. We present numerical results on standard test problems from machine learning.