We introduce a higher dimensional analog of the Kac-Moody vertex algebra in conformal field theory using the language of factorization algebras. After characterizing its central extensions, we proceed to discuss its relationship with higher dimensional suspersymmetric gauge theory. The relationship generalizes the correspondence between Chern-Simons theory on a three manifold with boundary and the WZW model on the boundary Riemann surface. We will focus on two classes of supersymmetric gauge theories in dimensions 5 and 7 that admit boundary conditions realizing the higher dimensional Kac-Moody algebras as the boundary operators. Time permitting we will discuss how this setup is part of an ongoing effort to formulate a case of the AdS/CFT correspondence based on ideas of Costello-Li.