Post Your Answer
1 year ago in Research Design , Statistical Analysis By Govind
What major threats affect internal validity in non-equivalent groups pre-post designs, and which modern statistical methods can enhance causal inference in such quasi-experiments?
My educational intervention study uses existing, non-randomly formed classrooms as treatment and control groups, with pre-test and post-test scores. I'm concerned about selection bias, maturation, and regression to the mean confounding my results. I know ANCOVA on post-test scores using the pre-test as a covariate is common, but I'm looking for more modern, robust methods to approximate causal inference.
Â
All Answers (1 Answers In All)
By Farah Answered 10 months ago
The core threat in your design is selection bias the groups likely differed before your intervention. While ANCOVA helps, I have seen it fail if the relationship between pre- and post-test isn't perfectly linear or homogeneous. I would recommend stronger modern approaches. First, consider Propensity Score Matching (PSM) to create a balanced synthetic control group from your pool of non-equivalent subjects. Alternatively, a Difference-in-Differences (DiD) model explicitly compares the change in each group, which can control for time-invariant group differences. For the strongest claim, use ANCOVA on a sample matched via PSM. Always test for parallel trends in a DiD framework.
Reply to Farah
Related Questions