This talk reviews results obtained in the last decade on algorithms for linear algebra that minimize communication and are thus suitable for current and emerging massively parallel computers. We will focus in particular on two operations, solving large sparse linear systems of equations and computing low rank matrix approximations. The talk will conclude with recent results obtained on computing tensor approximations and present many remaining open questions in this context.