Data integration is the process of extracting information from multiple sources and jointly analyzing different data sets. In this project, we propose to use the mixed $\ell_{2,1}$ regularized composite quasi likelihood function to perform multi-task feature learning with different types of responses including continuous and discrete responses. For high dimensional settings, our result establishes the sign recovery consistency and estimation error bounds of the penalized estimates under regularity conditions. Simulation studies and real data analysis examples are provided to illustrate the utility of the proposed method to combine correlated platforms with heterogeneous tasks and perform the joint sparse estimation.