The importance of understanding on-line tutoring impact has increased dramatically, especially after the COVID-19 pandemic. However, deep causal concepts about on-line tutoring are still lacking, especially on economically disadvantaged students. This paper is an observational study that targets low-income high school students in Saudi Arabia with high failing risk. The paper aims at (1) finding on-line math tutoring impact on needy students who already took tutoring, and (2) identifying and characterizing students that need tutoring to pass. We use observational data collected in a student registration process to build two models: (1) a Bayesian multi-level regression causal model, then (2) a counter-factual model. Results show that the models gave statistically significant estimates. In model 1, the average causal impact of maximum tutoring minutes on the math mark was +4.9 (out of 100). In model 2, the counter-factual maximum impact on tutored students was +5.3. We also estimate that only 1.9% of students needed the tutoring to avoid failing (2.8% of the enrolled), and we show their characteristics. © 2021 IEEE.
Counter-factual Analysis of On-Line Math Tutoring Impact on Low-income High School Students