The main aim of this talk is to introduce and review some of the recent developments in the theoretical and mathematical aspects of the non-equilibrium statistical mechanics of turbulent geophysical flows and climate dynamics. This field, at the intersection between statistical mechanics, turbulence, and climate applications is a wonderful new playground for theoretical and mathematical physicists. Path integrals, instanton theory, semiclassical approximations, large deviation theory, and diffusion Monte-Carlo algorithms are at the core of our approach. We will consider two classes of applications in climate dynamics for which, rare dynamical events may play a key role. A first class of problems are extreme events that have huge impacts, for instance extreme heat waves. We will apply large deviation algorithms to compute the probability of extreme heat waves. A second class of problems are rare trajectories that suddenly drive the complex dynamical system from one attractor to a completely different one, for instance abrupt climate changes. We will treat the example of the disappearance of one of Jupiter's jets during the period 1939-1940, as a paradigmatic example of a drastic climate change related to internal variability. We will demonstrate that quasi geostrophic turbulent models show this kind of ultra rare transitions, where turbulent jets suddenly appear or disappear on times scales tens of thousands times larger than the typical dynamical time scale. Those transitions will be studied using large deviation theory and non-equilibrium statistical mechanics.