Suppose that A is a finite symmetric subset of an abelian group G and write MG(A) for the magnitude of the largest negative Fourier coefficient of 1A. Chowla asked for a lower bound on MRZ(A)uniform in |A|. We consider the question for an arbitrary group in which case one has a dichotomy: either A is close to a union of subgroups of G, or MG(A) is large. We shall emphasize the case when G is finite and |A| = Ω(|G|), where we are able to achieve particularly strong lower bounds for MG(A).