The What Works Clearinghouse Review Process: An Analysis of Errors in Two Recent Reports, Jean Stockard and Timothy W. Wood, Eugene, Oregon: National Institute for Direct Instruction, NIFDI Technical Report 2013-4.
The What Works Clearinghouse (WWC) is a federally funded program established in 2002 that evaluates educational interventions and publishes reports and summary ratings. The reports have received extensive criticism, including concerns such as examining only a small proportion of the available evidence, errors in the review process, and a lack of peer review and comparisons of results to related literature. Two WWC reports issued in July 2013 illustrate the severe problems that can permeate the process and result in the dissemination of erroneous conclusions. In one case, the WWC’s errors resulted in a positive rating for a program that has been determined, by more inclusive and careful reviews, to be ineffective and inefficient. In the other case the WWC’s errors resulted in a negative conclusion regarding a program that has been judged, by more inclusive and careful reviews, to be highly effective. This document describes these errors and preliminary steps needed to prevent their reoccurrence.