We investigate the quantum Mpemba effect from the perspective of non-equilibrium quantum thermodynamics by studying relaxation dynamics described by Davies maps. Starting from a state with coherences in the energy eigenbasis, we demonstrate that an exponential speedup to equilibrium will always occur if the state is transformed to a diagonal state in the energy eigenbasis, provided that the spectral gap of the generator is defined by a complex eigenvalue. When the transformed state has a higher non-equilibrium free energy, we argue using thermodynamic reasoning that this is a genuine quantum Mpemba effect. Furthermore, we show how a unitary transformation on an initial state can always be constructed to yield the effect and demonstrate our findings by studying the dynamics of both the non-equilibrium free energy and the irreversible entropy production in single and multi-qubit examples.