Variable projection is a popular technique for solving structured nonlinear optimization problems, where decision variables can be naturally partitioned into two blocks, and one block can be easily 'projected' out. We describe classic formulations and the approach, and then show that the idea extends nicely to big data and machine learning contexts, with numerous applications including PDE constrained optimization, nuisance parameter estimation, statistically robust techniques, multiple kernel learning, and nonsmooth regularizations of classic formulations.