Many applied convex optimization problems have favorable objective functions and complicated constraints which precludes first-order methods from being immediately applicable. In this talk we describe an approach that exchanges the roles of the objective and constraint functions, and instead approximately solves a sequence of parametric level-set problems. A zero-finding procedure, based on inexact function evaluations and possibly inexact derivative information, leads to an efficient solution scheme for the original optimization problem. We present the theoretical and practical properties of this approach for a broad range of problems, including conic optimization, gauge optimization, generalized linear models for inference, and robust regression.