This paper introduces a structural dynamic factor model (SDFM) for stock returns. Compared to standard linear factor models, structural modeling accounts for nonlinear effects of common factors when the distance-to-default is small. Such a SDFM has a rather complicated form, especially since the underlying factors are unobservable and have nonlinear effects. We develop appropriate methods to estimate such models and use them for prediction, filtering and smoothing purposes, such as indirect inference or Approximate Bayesian Computation (ABC) filtering and smoothing. These SDFM are applied to the analysis of systematic risk of financial institutions and to obtain their rating of default and speculative features.