This study presents a meta-analysis of the effects of educational technology use on mathematics learning. Researchers have established that the type and the quality of instruction, not the delivery mechanism, is what affects learning. Our research questions (RQs) were:

To what extent do educational technology interventions improve students’ mathematics achievement?
What are the effects of educational technology by mathematical domain (e.g., operations, word problems, fractions, algebra, general mathematics proficiency)?
What study and research features (e.g., design, grade level, publication status) moderate the effects of educational technology programs on students’ mathematics achievement?
What substantive and instructional program features (e.g., fidelity of implementation, type of feedback) moderate the effects of educational technology programs on students’ mathematics achievement?

Results showed that methodological features continue to moderate effects of educational technology applications, and considerations must be made regarding such aspects as the sample size, study design, and, perhaps most importantly, fidelity of implementation when evaluating educational technology interventions. Second, program developers should continue to integrate research-based strategies to positively influence learning, particularly in terms of personalization and how the educational technology application integrates into or replaces traditional instruction. Lastly, an immense number of studies were excluded from the present meta-analysis due to a lack of rigorous methodology. The quality of meta-analyses, and thus, the interpretation of the effectiveness of an intervention, depend on the existence of quality studies.

Technical Report:

Morrison, J. R., Borokhovski, E., Bernard, R. M., & Slavin, R. E. (2024). A quantitative synthesis of outcomes of educational technology approaches in K-12 mathematics. Center for Research and Reform in Education, The Johns Hopkins University.

This review examines research on the effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms. It applies rigorous, consistent inclusion standards to focus on studies that meet high methodological standards. Three key research questions are addressed:

• Do education technology applications improve mathematics achievement in K-12 classrooms as compared to traditional teaching methods without education technology?
• What study and research features moderate the effects of education technology applications on student mathematics achievement?
• Large-scale randomized studies by Dynarski and Campuzzano found near-zero effects of modern CAI programs on math achievement. Do other high-quality studies agree or disagree with these findings?

A total of 74 qualifying studies, with a total sample size of 56,886 K-12 students, are included in the final analysis. Three major categories of education technology are reviewed:

• Computer-managed learning, which included only Accelerated Math. This program uses computers to assess students’ mathematics levels, assign mathematics materials at appropriate levels, score tests on this material, and chart students’ progress.
• Comprehensive models, such as Cognitive Tutor and I Can Learn, use computer-assisted instruction along with non-computer activities as the students’ core approach to mathematics.
• Supplemental CAI technology, which consists of individualized computer-assisted instruction (CAI). Supplemental CAI programs, such as Jostens, PLATO, Larson Pre-Algebra, and SRA Drill and Practice, provide additional instruction at students’ assessed levels of need to supplement traditional classroom instruction.

Findings of the review indicate that educational technology applications produce a positive but small effect (ES=+0.16) on mathematics achievement. In particular, supplemental CAI had the largest effect, with an effect size of +0.19. The other two categories, computer-managed learning and comprehensive models, had a much smaller effect size, +0.09 and +0.06, respectively.

Technical Report

Published Report

Cheung, A., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88-113.