We study the behavior of a stochastic variant of replicator dynamics in two-agent zero-sum games. We characterize the statistics of such systems by their invariant measures which can be shown to be entirely supported on the boundary of the space of mixed strategies. Thus, in the presence of stochastic perturbations, even in the most classic zero-sum settings, such as Matching Pennies, we observe a stark disagreement between the axiomatic prediction of mixed strategy Nash equilibrium and the evolutionary emergent behavior derived by an assumption of stochastically adaptive, learning agents.