We discuss the algorithms for group sparsity with overlap in which matrix-vector multiplications can be carried out efficiently. An accelerating technique to improve the performance of existing algorithms is proposed by solving a similar sub-problem. We further show that the new scheme can be formulated in the inner-outer regime by exploring the sparse structure. This can also be viewed from the dim.