Optimal Control in Quantum Thermodynamics using Reinforcement Learning
Quantum thermodynamics deals with understanding how the concepts of thermodynamics, originally developed for macroscopic systems, can be extended to the quantum regime. From a more practical point of view, a fundamental question is that of understanding how to optimally control quantum devices such as quantum thermal machines and quantum batteries. A quantum thermal machine is a micro- or nano-scale device that can perform useful thermal tasks, such as cooling or energy harvesting, and its performance is quantified in terms of its power, efficiency, and power fluctuations. A quantum battery is a quantum system used to rapidly store and deliver energy, and its performance is quantified in terms of the charging power and of the ergotropy, i.e. the extractable work. A crucial question is that of understanding whether quantum effects, such as entanglement and coherence, can provide an advantage in such quantum devices. However, the optimal control of quantum devices is an extremely challenging task involving many-body quantum dynamics, out-of-equilibrium systems, and open quantum system dynamics. Here we discuss how Reinforcement Learning provides a flexible framework to tackle a variety of optimal control problems emerging in quantum thermodynamics. After introducing a general approach, capable of optimizing one or multiple objectives, we showcase its flexibility optimizing the power [1] and finding Pareto-optimal tradeoffs between power, efficiency, and power fluctuations [2] of quantum thermal machines. We then show how we can also maximize the charging power and ergotropy of a quantum battery exhibiting a collective speedup of the charging power [3]. In all cases, our approach discovers novel control strategies that outperform previous proposals made in literature. We conclude discussing how the method can be potentially applied directly to experimental devices, even in the absence of direct knowledge of the systems' dynamics. REFERENCES: [1] P.A. Erdman and F. Noé, NPJ Quantum Inf. 8, 1 (2022). [2] P.A. Erdman, A. Rolandi, P. Abiuso, M. Perarnau-Llobet, and F. Noé, Phys. Rev. Res. 5, L022017 (2023). [3] P.A. Erdman, G. M. Andolina, V. Giovannetti, and F. Noé, arXiv:2212.12397 (2023). [4] P.A. Erdman and F. Noé, PNAS Nexus 2, pgad248 (2023).