This report summarizes federal regulations regarding protections of human subjects and describes the policies and procedures that the National Institute for Direct Instruction (NIFDI) will use when gathering data from subjects and analyzing data provided from other sources. The intention of the procedures is to minimize any risk to subjects that may occur from the research process and to maximize possible benefits. The document details the federal guidelines regarding the protection of human subjects, procedures to be used with intact data sets, and procedures to be used when data are gathered by NIFDI personnel. Additionally, appendices are included to provide details on categories of review used by the U.S. Department of Health and Human Services as well as a sample of a NIFDI informed consent letter.
In this monograph Engelmann outlines his approach to teaching concepts to children and explains the reasoning behind it. He describes the principles behind the process of conceptual learning, showing how a teacher demonstrates the characteristics of a concept by presenting both instances and non-instances, and developing an appropriate test/task the student must perform to demonstrate an understanding of the concept. Engelmann also discusses the issues of programming efficient and appropriate routines and tasks in order to not only teach students a concept, but also how the concept is a sub-concept within a larger concept. He determined that a child must learn three things when being taught a specific task: the concept being taught, the rules for responding to the concept, and the rule that responding is worthwhile. Engelmann asserts that with adequate teaching, there should be virtually no instructional failures.
This short document reviews conclusions from the scholarly literature regarding Direct Instruction and contrasts these conclusions with those of the What Works Clearinghouse (WWC). It compares the procedures used by the WWC to those commonly used within the scholarly literature and provides information on numerous errors within the WWC reviews of DI programs.
Scholarly reviews are unanimous in concluding that Direct Instruction (DI) programs are highly effective. In contrast, the WWC has found very few studies of DI that meet its criteria for review and has concluded that there is little evidence to support the programs’ efficacy. This report analyzes why these conclusions differ so markedly from the scholarly literature. The first section discusses issues related to the WWC’s criteria regarding exclusion or inclusion of studies and their review procedures. Areas discussed include the ways in which reviews focus on narrow curricular programs, fail to examine or consider the characteristics of the programs, apply an arbitrary time limit to the included studies, and use standards for review that differ markedly from those generally used in the social sciences, excluding most field-based studies and those using advanced statistical methods. The second section examines studies of DI programs that the WWC has found to meet their inclusion criteria and documents serious errors in a large proportion of these decisions. The third and fourth sections analyze content of the WWC report on Reading Mastery and students with learning disabilities that was initially posted in July 2012, reviewing errors in inclusion and exclusion of studies. Over twenty research studies that could have been included in the WWC review are examined, detailing the design and conclusions of the studies, the effect size associated with their results, and reasons that the WWC might reject the study for inclusion. The effect sizes are statistically analyzed. There was no indication that the criteria used by the WWC to select or exclude studies from consideration were related to the reported results. The report concludes that the WWC procedures appear to result in a selective and inaccurate view of the DI literature. It suggests that consumers would be well advised to consult sources of summary material other than the WWC and, especially, the well-conducted and highly regarded meta-analysis literature.
