The closed-form (CF) likelihood approximation of Ait-Sahalia (2002, 2008) is commonly used in financial modeling. Bayesian inference requires the use of MCMC and the (unnormalized) CF likelihood can become inaccurate when the parameters are far from the MLE; samplers can become stuck when (typically) in the tails of the posterior distribution. Auxiliary variables have been used in conjunction with MCMC to address intractable normalizers (see Moller et al. (2006)), but choosing such variables is not trivial. We propose a MCMC algorithm that addresses the intractable normalizers in the CF likelihood which 1) is easy to implement, 2) yields a sampler with the correct limiting distribution, and 3) greatly increases the stability of the sampler compared to using the unnormalized CF likelihood in a standard Metropolis-Hastings algorithm. The efficacy of our approach is demonstrated in a simulation study of the Cox-Ingersoll-Ross (CIR) and Heston models, and is applied to two well known real-world datasets.