[Research Snapshot] Evaluation of Mathematics Interventions: Propensity score analysis as an alternative to randomized control
Many first-year students find the transition to post-secondary mathematics education challenging. Toronto Metropolitan University (TMU), like many other institutions, has implemented several intervention programs to address the knowledge gap of first year students or provide support for them when they transition to university mathematics. However, rigorous evaluation of similar or related programmes have always been a challenge in mathematics education research due to ethical reasons or self selection bias. Randomized control trial, which is the gold standard for scientific research, is not always feasible for evaluating interventions such as those cited in the preceding paragraph. This talk is a preliminary undergraduate thesis report in which we argue that Propensity Score Analysis (PSA) (Thoemmes and Kim, 2011) is a statistical methodology that can be employed to address the issue of self selection bias inherent in the evaluation of mathematics education intervention programmes. We use simulated data to demonstrate how the PSA procedure can be used to evaluate mathematics education interventions such as ACES. We highlight the challenges involved in the implementation of PSA. We suggest some potential reasons why PSA is not well known and used in mathematics education research. We conclude that 1) PSA is the next best alternative to randomized controlled trial and 2) there is a need for collaborative research that employs the technique and offers workshops on the implementation of PSA.