Assessing the DI Research Base

Hundreds of studies have examined different instructional programs.  In recent decades, scientists have developed techniques of sorting through these studies to summarize the research in an objective manner. One of the most widely used methods is meta-analysis, a technique that uses statistical techniques to combine the results of numerous studies that test the same hypothesis. Several researchers have examined how Direct Instruction affects students’ achievement through meta-analyses or other ways of assessing the research base.

One of the most common ways of summarizing results in meta-analyses is through “effect sizes.” Effect sizes are simple statistical computations that summarize the strength of the difference between two groups. It is calculated by dividing the difference between two means by the common standard deviation. In the education literature, effect sizes of .25 or larger have traditionally been considered “educationally significant.”

W. A, T. White reviewed studies of the effectiveness of Direct Instruction programs with special education students in an article published in 1988. The examination of 25 different articles found none that had results that favored the comparison groups over DI and effect sizes several times the level of “educationally significant.” The positive effects of Direct Instruction appeared across different handicapping conditions, age groups and skill areas.

Gary Adams conducted a review of research on Direct Instruction studies published between 1972 and 1996. He limited his analysis to experimental or quasi-experimental studies that included a comparison group and in which the pretest scores of the Direct Instruction and comparison groups were not significantly different. Adams found 37 separate research articles that met these criteria. The average effect size across these studies was .87, again far larger than the standard criterion of educational significance.

In 2002 Cheryl Schieffer and associates published a review of research studies on Reading Mastery. Of the 21 comparative studies that they found two-thirds (14) had significant results favoring Direct Instruction, while only 3 had results that favored other programs.

Two different reports have examined the body of research on Direct Instruction as a method of Comprehensive School Reform (CSR), the reorganization and revitalization of entire schools, rather than the implementation of single curricula: an article by Geoffrey Borman and associates (2003) and a report by the Comprehensive School Reform Quality Center (2006). Direct Instruction was one of only 3 models found to have the strongest evidence of effectiveness. The research base for DI was the largest of all models examined and the effect sizes associated with DI were stronger than for any other model.
The National Reading Panel published an assessment of the scientific literature on reading in 2000. Similar to other meta-analyses and reviews, they limited their review to studies that employed experimental or quasi-experimental designs.   Again, the results demonstrated the effectiveness of Direct Instruction.

The What Works Clearinghouse was established in 2002. It periodically issues reports that summarize the effectiveness of individual curricular programs, focusing on small scale studies of limited duration. While it was hoped that the WWC could expand the knowledge base developed through the meta-analytic tradition, it has not lived up to expectations and its work has encountered substantial criticism.

Click on the links below for more information on meta-analyses and reviews of educational research.

Gary Adams’ description of how he conducted his meta-analysis of Direct Instruction
Background and results of Adams’ DI meta-analysis
White’s (1988) meta-analysis of the effectiveness of Direct Instruction in special education
Schieffer et al’s analysis of studies of Reading Mastery
Borman and associates’ 2003 meta-analysis of Comprehensive School Reform and Achievement
The 2006 report of the Comprehensive School Reform Quality Center
The National Reading Panel report
The What Works Clearinghouse

Back to Research Entry Page