We study issues of robustness in the context of Quantitative Risk Management. Depending on the underlying objectives, we develop a general methodology for determining whether a given risk measurement related optimization problem is robust. Motivated by practical issues from financial regulation, we give special attention to the two most widely used risk measures in the industry, Value-at-Risk (VaR) and Expected Shortfall (ES). We discover that for many simple representative optimization problems, VaR generally leads to non-robust optimizers whereas ES generally leads to robust ones. Our results thus shed light from a new angle on the ongoing discussion about the comparative advantages of VaR and ES in banking and insurance regulation.