Dr Kerry Hempenstall, Senior Industry Fellow, School of Education, RMIT University, Melbourne, Australia.

All my blogs can be viewed on-line or downloaded as a Word file or PDF at https://www.dropbox.com/sh/olxpifutwcgvg8j/AABU8YNr4ZxiXPXzvHrrirR8a?dl=0

This paper refers/adds to a piece I wrote back in 2013. I checked what age some of the papers might be. There are some of the published documents down to 2001!

It seems to me that there may be less criticism on DI now – and more publicity.

The original 2013 document can be found below. 

Why does Direct Instruction evoke such rancour?

https://www.nifdi.org/resources/hempenstall-blog/389-why-does-direct-instruction-evoke-such-rancour.html

The newer issues DI substance does appear much more than in the past. Lots of print with views on DI. Lets see!

Direct Instruction Works. So Why Is It Controversial

“ … Direct Instruction remains controversial in Australia. In an influential 2014 essay, Allan Luke, Emeritus Professor at Queensland University of Technology and an expert in “multiliteracies,” took aim at Direct Instruction. While begrudgingly acknowledging that Direct Instruction “can generate some performance gains in conventionally-measured basic skills of early literacy and numeracy,” he listed “many” criticisms. These criticisms range from “focusing on teacher control” to the “deskilling of teachers” and an opposition to tracking—Direct Instruction programs often use placement tests to determine which program is most suitable for each child.

The effectiveness of Direct Instruction aligns with the wide body of evidence from educational psychology and from correlational studies of teacher effectiveness that favors explicit approaches where teachers fully explain concepts and model tasks before asking students to do the same. Direct Instruction is not the only way of teaching explicitly, but it is understandable that Pearson would turn to this solid bet when attempting to address the problems of disadvantage in remote Australian schools.”

Ashman, G. (2022). Direct Instruction Works. So Why Is It Controversial? Quillette.

https://quillette.com/2022/05/14/direct-instruction-works-so-why-is-it-controversial

A series of DI from the Research Hub

Direct instruction and how it helps novice learners get the basics  (2020)

“Research from Clark, Kirschner and Sweller suggests that direct forms of instruction are much more effective than approaches that rely on students finding things out for themselves and that novice learners cannot simply copy the behaviours of experts.

In the case of problem-solving, for example, novice learners are usually less successful because they do not have sufficient knowledge to draw on. Experts, on the other hand, have a reservoir of knowledge that informs their decisions – they are not better at problem-solving in an abstract sense, they just know more.

Rosenshine’s research is clear that more effective teachers are good at sequencing learning from an initial phase of direct instruction to modelling in small steps, guided practice and independent practice.”

Tom Sherrington and Sara Stafford 2020

https://my.chartered.college/research-hub/direct-instruction-and-how-it-helps-novice-learners-get-the-basics/#:~:text=Direct%20instructionXA%20method,prior%20knowledge%20and%20low%20confidence

A move towards reconceptualising direct instruction in sport coaching pedagogy. (2020)

Cope, E. & Cushion, C. (2020). A move towards reconceptualising direct instruction in sport coaching pedagogy. Impact, issue 10, pp. 70-73.

https://my.chartered.college/impact_article/a-move-towards-reconceptualising-direct-instruction-in-sport-coaching-pedagogy/ 

Utilising direct instruction to train primary school children in decision-making skills in the science classroom. (2021)

“A longstanding debate in educational and psychological research is the effectiveness of constructivist teaching methods over direct instruction. Although constructivism can take many forms (such as discovery learning, inquiry learning, etc.), one common assumption widely shared in the research community is that learning is constructed by the individuals, who are active sense-makers, rather than being just a reflection of external events (Mayer, 2004; Tobias, 2009).

However, advocates of direct instruction (e.g. Kirschner et al., 2006) have provided evidence that it is significantly more efficient than discovery learning (minimal guidance during instruction) in teaching scientific concepts and processes.

Moreover, through a review of the literature, Alfieri et al. (2011) illustrate that unassisted discovery generally does not generate beneficial learning outcomes, and emphasise the need for teachers to support learners with scaffolded tasks, feedback and worked examples.

In the case of novice learners in particular, most of the researchers on both sides of the argument indicate how important direct instruction is when dealing with new content (e.g. Paas and van Gog, 2006; Tobias, 2009).

Having reviewed different areas of research, our hypothesis was that direct instruction is more effective than discovery learning instruction for introducing primary school students to systematic decision-making, a topic on which students have limited prior knowledge or training.”

Tsapali, M. & Ellefson, M. (2021). Utilising direct instruction to train primary school children in decision-making skills in the science classroom. Impact. Journal of the Chartered College of Teaching. Issue 5.

https://educationhq.com/news/direct-instruction-flexible-literacy-program-a-costly-failure-researchers-76001/

‘Engaging conceptual development within science through the use of concept cartoons: Harmonising direct instruction and constructivist pedagogy. (2023)

“Within science, misconceptions can occur for a range of reasons (e.g. an opinion based on incorrect thinking); however, misconceptions can be resistant to change. The consistency of misconceptions has been specifically noticeable during transition from primary to secondary science (Driver et al., 2014). Consequently, an investigation was structured to assess the impact of an intervention comparing direct instruction and constructivist approaches through the use of concept cartoons. Between 1987 and 1990, the Science Processes and Concept Exploration (SPACE) project explored children’s scientific conceptual development and misconceptions (Black and Harlen, 1993; Harlen and Qualter, 2018; STEM Learning, 2016).”

Buckler, S. & Moore, H. (2023). Engaging conceptual developing within science through the use of concept cartoons: harmonising direct instruction and constructivist pedagogy. Impact: Journal of the Chartered College of Teaching, 18. https://my.chartered.college/impact_article/engaging-conceptual-development-within-science-through-the-use-of-concept-cartoons-harmonising-direct-instruction-and-constructivist-pedagogy/ 

The Case for Direct Instruction (2023)

“For over 50 years, the best way to educate children has been heatedly debated by those who favour what might be described as ‘teacher-directed instruction’ and those who favour ‘student-centred instruction’. In this article, we contend that Direct Instruction (DI), a teacher-directed approach, offers educators, by several important measures, the most effective approach to meeting the academic needs of underperforming students.”

The case for direct instruction. Marcy Stein and Kristen Rolf 2023

https://my.chartered.college/impact_article/the-case-for-direct-instruction/

The case for combining inquiry-based and direct instruction (2023)

“Many studies investigating inquiry learning in science domains have appeared over the years. Throughout this period, inquiry learning has been regularly criticized by scholars who favor direct instruction over inquiry learning. In this vein, Zhang, Kirschner, Cobern, and Sweller (2022) recently asserted that direct instruction is overall superior to inquiry-based instruction and reproached policy makers for ignoring this fact. In the current article we reply to this assertion and the premises on which it is based.

We review the evidence and argue that a more complete and correct interpretation of the literature demonstrates that inquiry-based instruction produces better overall results for acquiring conceptual knowledge than does direct instruction.

We show that this conclusion holds for controlled, correlational, and program-based studies. We subsequently argue that inquiry-based and direct instruction each have their specific virtues and disadvantages and that the effectiveness of each approach depends on moderating factors such as the learning goal, the domain involved, and students' prior knowledge and other student characteristics.

Furthermore, inquiry-based instruction is most effective when supplemented with guidance that can be personalized based on these moderating factors and can even involve providing direct instruction. Therefore, we posit that a combination of inquiry and direct instruction may often be the best approach to support student learning.

We conclude that policy makers rightfully advocate inquiry-based instruction, particularly when students’ investigations are supplemented with direct instruction at appropriate junctures.”

Ton de Jong, Ard W. Lazonder, Clark A. Chinn, Frank Fischer, Janice Gobert, Cindy E. Hmelo-Silver, Ken R. Koedinger, Joseph S. Krajcik, Eleni A. Kyza, Marcia C. Linn, Margus Pedaste, Katharina Scheiter, Zacharias C. Zacharia. Let's talk evidence – The case for combining inquiry-based and direct instruction. Educational Research Review, 39,2023, 100536,

ISSN 1747-938X,

https://doi.org/10.1016/j.edurev.2023.100536.

(https://www.sciencedirect.com/science/article/pii/S1747938X23000295)

Let's talk evidence – The case for combining inquiry-based and direct instruction. (2024)

In conclusion, while we welcome the acknowledgement of a need for explicit instruction by De Jong et al. (2023), we are bemused by their theory-free approach to inquiry learning with its limited evidence of effectiveness from randomised, controlled trials. They provide neither theory nor data to indicate when or how explicit instruction and inquiry learning should be used and combined.

Meta-analyses that are primarily concerned with program-based studies rather than randomised, controlled trials that examine one factor at a time reflect exactly the issues that Zhang et al. (2022) discussed in their article. De Jong et al. (2023) strongly object to the Zhang et al. (2022) conclusions, seemingly unaware that their own conclusions must inevitably flow from Zhang et al.’s (2022) analyses.

Data without theory is likely to appear random. Cognitive load theory provides an overarching theory that can be used to both generate data and organise it. The theory indicates both how and when explicit instruction and inquiry should be used.

One of the crowning glories of human evolution, our skill in rapidly communicating large amounts of information to each other, is a central principle of cognitive load theory: the borrowing and reorganising principle.

Cognitive load theory is concerned with how to organise instruction to maximally make use of this principle. It is a primary skill universally used by all humans including the readers and writers in the current exchange but is considered inappropriate by De Jong et al. (2023), under some undefined conditions, for learners in educational settings. Even worse, at times and for reasons that remain unclear, that skill is to be replaced by an inefficient procedure that we otherwise only use when communication from others is unavailable.

It remains a mystery to us why students, and only students, should, under unspecified conditions, be capriciously prevented from using what is arguably the most important evolved skill of our species, our unique ability to rapidly assimilate large amounts of novel information from other people.”

Sweller, John, Zhang, Lin, Ashman, Greg, Cobern, William and Kirschner, Paul A.. (2024). Response to De Jong et al.’s (2023) paper “Let's talk evidence – The case for combining inquiry-based and direct instruction”. Educational Research Review. 42, pp. 1-6. https://doi.org/10.1016/j.edurev.2023.100584

Direct/Explicit Instruction and Social Constructivist Practices in The Inclusive Classroom (2024)

“Effective educational practices play an instrumental role in student success. In the context of an inclusive classroom, it is crucial that educators use evidence-based practices to ensure all students to meet educational outcomes. This review focuses on two evidence-based pedagogies, namely direct/explicit instruction (DI/EI) and social constructive approaches, and their effects on the inclusive classroom. Special consideration is given to cooperative learning and concrete implementation guidelines are explored. Lastly, the complimentary effects of combining DI/EI and social constructivist practices are investigated to advance the argument for using a variety of evidenced-based practices within the inclusive classroom.”

Matthews, A. (2024). Direct/Explicit Instruction and Social Constructivist Practices in The Inclusive Classroom. ResearchGate, 29(1), 79-102. 10.37119/ojs2024.v29i1.738

Direct instruction 'Flexible Literacy' program a costly failure: researchers.(2020)

Geordie Little Published April 9, 2020

A $30 million direct instruction program aimed at boosting literacy results in very remote schools has failed, according to a new study.

Direct instruction 'Flexible Literacy' program a costly failure: Researchers. (2020)

Osborne said that the Flexible Literacy program has not only failed to achieve its goals, but that it has demonstrated potential harm in very remote schools.

The study, put out recently by the University of South Australia and the Batchelor Institute of Indigenous Tertiary Education, found that both literacy outcomes and attendance dropped at schools participating in the Flexible Literacy for Remote Primary Schools Program.

The program, introduced in 2015, aims to improve literacy outcomes in remote schools with primarily Indigenous students. It has been subject to two evaluation reports so far, with a final evaluation scheduled to come out this year.

The study’s co-author, Associate Director of Regional Engagement (APY Lands) at UniSA Dr Sam Osborne, said that the program’s lack of success was “obfuscated” by the Government.

“In late 2017, there was a parliamentary review that came out just before Christmas,” he said.

“Simon Birmingham was the minister at the time who cited ‘green shoots of hope’ as the reason to put another $4.1 million into the program. Now, he's used the language of 'green shoots of hope' because the data clearly wasn't there to support the kinds of outcomes that they were hoping for.”

Osborne said that the program has not only failed to achieve its goals, but that it has demonstrated potential harm in very remote schools.

“Schools who used direct instruction had lower results in the three years that we looked at the NAPLAN data compared to the three years prior,” he said.

“They also had lower results than other schools, similar schools in the area, who didn't use the program. A secondary outcome, which is of some concern, was that we also looked at student attendance rates for schools using the direct instruction program and those results were significantly lower, more than 7 per cent for the three years that they ran the direct instruction program, and significantly lower than other schools in the region, similar schools who didn't use the direct instruction model.

“So initially we're looking at, ‘Did the program achieve what it set out to do?’, and the answer is no. And secondly, the program was evaluated twice and the data was available. It's of some concern that these results weren't  put out front to say, ‘Actually, this isn't achieving what it set out to do’.”

Edith Cowan University’s Associate Professor Dr Lorraine Hammond was involved with the program’s initial rollout. Speaking to EducationHQ, she identified several issues with Osborne’s study.

“Normally in comparison studies, the schools in this case are assigned randomly to the control and the comparison groups and this is to make them as similar as possible, and that just didn't happen,” she said.

“So that raises some questions about the actual methodology. And then also, they didn't explain how they chose the actual schools that they ... put as non-intervention schools in the end. So that for me raised a few issues. I guess in terms of their own [identified] limitations, they said that some of the schools didn't report pre and post results, so that, for me, is a bit of an issue as well.”

Hammond said that the schools which adopted the Flexible Literacy for Remote Primary Schools Program were likely to be in a worse initial position than other comparable schools.

“It's interesting in the sense that the schools who agreed to be part of the intervention were probably the ones who weren't doing so well, possibly the ones who had the highest changeover of staff, which is a big issue in these schools as well.

“Yeah, so lots of unanswered questions for me, but disappointing in the sense that I saw the impact of that particular intervention and that there is a massive gap between research and practice.

“Direct instruction is really effective and there's lots of meta analyses out there to show that, and certainly in the schools that I've worked in, we've got good results using it, but implementation is another thing.”

Hammond is currently involved in WA’s Kimberley Schools Project, another program which uses direct instruction in remote Indigenous schools.

“What we use is an unscripted approach. Teachers are encouraged to teach reading in a very structured way,” she said.

“And so I provide the professional learning for that and it's not unlike the model that comes from the direct instruction reading approach, but it's adapted for teacher use. In every school … there might be some children who are coming in at a very low level, and so they might be using something that's [designed for four-year-olds], even though they might be maybe five or six years of age. So we can adapt it accordingly.

“It gives a lot more control for teachers and I think that might have been one of the issues around the direct instruction program.”

Osborne said that the money used to fund the Flexible Literacy program could have been spent on extra support, resources and training for teachers.

“I suppose a question that comes up from this study is, if you've got a program that's $30 million across 30 small schools and it's not running the schools, it's teaching English, which is a component of a school program, then I think questions probably do need to be asked about the accountability and monitoring and evaluation of the effectiveness of such a program and why we've continued with an approach [when] it's been apparent for some time that the numbers haven't been good.

“And it leads us to more interesting and more important questions. If not direct instruction, then what else?”

Little, G. (2020). Direct instruction 'Flexible Literacy' program a costly failure: researchers. EducationHQ. https://educationhq.com/news/direct-instruction-flexible-literacy-program-a-costly-failure-researchers-76001/

This next segment is a broader document, and includes earlier periods.

 In a previous post (http://www.nifdi.org/resources/news/hempenstall-blog/403-reviews-supporting-direct-instruction-program-effectiveness), I listed those reports, syntheses, reviews, and meta-analyses that have offered support to Direct Instruction as a genuine evidence-based approach to instruction.

In this post, I want to consider what is Direct Instruction, and what are the criticisms that have impeded the model from achieving the strong acceptance in education that it deserves. As an avid reader of research in education for many years, I’ve been regularly bemused to read studies employing a wide range of recently developed programs, some of which clearly influenced by DI. Rarely is DI evaluated or even discussed by independent researchers.

Research supports explicit instruction

How has DI been viewed by educators? Obviously, those still enamoured with Whole Language, or those whose pre-service training was conducted by WL protagonists, are likely to be critical of explicit instruction generally. DI being perhaps the prime example of explicit instruction, and having had a long history, may have been a lightning rod for those who do not consider explicit instruction as appropriate. In my education readings, and in my experience in offering electives to teachers-in-training, it is frequently evident that many critics have little understanding of DI. They just know (or have been told) that they don’t like it!

This position of being critical of DI because it is an exemplar of explicit instruction is yet another example of the disconnect between research and practice in education, given the acknowledgement of the explicit models’ evidence based superiority over other approaches (Alfieri, Brooks, Aldrich, & Tenenbaum, 2010).

"Research almost universally supports explicit instructional practices (Archer & Hughes, 2011; Kirschner, Sweller, & Clark, 2006; Klahr & Nigam, 2004; Marchand-Martella, Slocum, & Martella, 2004). Explicit instructional approaches are considered more effective and efficient as compared to discovery-based approaches (Alfieri, Brooks, Aldrich, & Tenenbaum, 2010; Ryder, Tunmer, & Greaney, 2008), particularly when students are naïve or struggling learners” (Marchand-Martella, Martella, Modderman, Petersen, & Pan, 2013, p.166).

Many teachers ignore the advantages for their students of explicit instruction, say, in reading. Yet, the strategies replacing them (such as the three cueing system (see http://www.adihome.org/adi-blog/entry/the-three-cueing-system-in-reading-will-it-ever-go-away) are not simply ineffective or neutral in effect, but create unnecessary obstacles to student success.

"The apparently unruly nature of the orthography, the existence of many words that do not follow straightforward one-to-one mapping of letter onto phoneme, may undermine the resolve of teachers to teach reading as if it were an exercise in alphabetic decoding. And teachers may not have such a resolve in the first place. We know that some do not because they have been trained to avoid explicit instruction in the alphabetic principle (Goodman, 1986; Shankweiler & Fowler, 2004). This in turn has been in part based on the conviction that reading cannot be done this way anyway, precisely because of the existence of irregular words like the, once, one, was, were, there … .So, we may have the beginnings of a perfect storm – children ill equipped to discover, all by themselves, the alphabetic nature of English writing, the same children well equipped, all by themselves, to discover its morphemic nature, and a teacher who advertently or inadvertently fosters the morphemic hypothesis and obscures the phonemic one, leading to children trapped in an initially successful strategy but one that will eventually leave them floundering (Byrne, Freebody, & Gates, 1992)" (Byrne, 2011, p. 182).

One issue raised by those who espouse constructivist approach is that higher order processing should be the priority, and it doesn’t arise from explicit teaching (it is claimed).

"The problem with that argument is that learning generally doesn’t work that way. As cognitive scientists like Daniel Willingham have shown, it’s all but impossible to have higher-order thinking without strongly established skills and lots of knowledge of facts. Cognitive leaps, intuition, inspiration--the stuff of vision--are facilitated by expending the smallest amount of processing capacity on lower-order aspects of a problem and reapplying it at higher levels. You leap over the more basic work by being able to do it without thinking much about it, not by ignoring it. This synergy between the rote and the creative is more commonly accepted in many nations in Asia. ‘Americans have developed a fine dichotomy between rote and critical thinking; one is good, the other is bad,’ write the authors of one study (2) of Japanese schools. But they find that many types of higher-order thinking are in fact founded on and require rote learning. Creativity often comes about because the mind has been set free in new and heretofore encumbered situations" (Lemov, Woolway, & Yezzi, 2012, p. 37-38).

One contributing factor may be related to the failure of teacher education to provide prospective teachers with the knowledge needed to appreciate research, and in particular, literacy research (Clark, Jones, Reutzel, & Andreasen, 2013; Greenberg, McKee, & Walsh, 2013; Leader-Janssen, & Rankin-Erickson, 2013).

"The results of these studies suggest that when teachers lack an understanding of research-based principles that allow effective adaptation, interventions may be prematurely discarded and practitioners may conclude that research has little relevance to their practice (Gersten, Vaughn, Deshler, & Schiller, 1997)” (Slocum, Spencer, & Detrich, 2012, p.172).

 There has been a burgeoning interest in brain-based learning (where else might learning occur, you ask?), and in the many programs purporting to address underlying neural structures. The evidence for these is generally slim to non-existent. So, it is interesting to read that approaches for which outcomes are strongly supportive of success are the same programs that indicate neural changes consequent upon such direct explicit teaching:

 "Focus, then, must be two-fold. First is the focus on ensuring appropriate environmental and nutritional conditions that stimulate dendritic growth in infancy and early childhood. But second must be emphasis on improving the strength of particular neural circuits, not simply on the overall growth of dendrites. Most interestingly, instructional activities such as memorization, mastery learning, and repetition-based activities appear to best strengthen and solidify the formation and maintenance of these circuits (Garrett, 2009; Freeberg, 2006). Data strongly support the use of precision teaching, mastery learning approaches, and programs such as DISTAR or direct instruction (Kirschner, Sweller, & Clark, 2006; Mills, Cole, Jenkins, & Dale, 2002; Ryder, Burton, & Silberg, 2006; Swanson & Sachse-Lee, 2000)” (Alferink & Farmer-Dougan, 2010, p. 46).

 More on the various criticisms later.

So, what is DI?

It is one of the most thoroughly researched educational models (DiMagliaro, Lockee, & Burton, 2005; Weir, 1990). There is ample evidence of its effectiveness for a wide range of student learning problems. It differs from Whole Language in its assumptions about the teaching process, about learner characteristics, and about the means of syllabus construction; in fact, it could be described as the antithesis of Whole Language.

"Although their [Whole Language] theories lack any academically acceptable research base they continue to dominate educational policy. Direct Instruction models are ignored notwithstanding the huge body of research that indicates that direct instruction is vastly superior if basic skills and knowledge are the goal" (Weir, 1990, p.30).

The Direct Instruction model lauded in Follow Through had its beginnings in the early 1960's through the work of Carl Bereiter and Siegfried Engelmann. The subsequent involvement of Wes Becker and Doug Carnine among others led to the publication of a number of teaching programs in 1969. The programs share a common teaching style readily observable to any classroom visitor. The instruction takes place in small groups with a teacher directing activities with the aid of a script, and students are actively involved in responding to a fast paced lesson during which they receive constant feedback. Programs are designed according to what, not whom, is to be taught. Thus, all children work through the same sequence of tasks directed by a teacher using the same teaching strategies. Individual differences are accommodated through different entry points, reinforcement, amounts of practice and correction strategies (Gregory, 1983).

Characteristics of the Direct Instruction Model

There are a number of important characteristics of Direct Instruction programs (Becker, 1977). It is assumed that all children can learn and be taught, thus failure to learn is viewed as failure to teach effectively (Engelmann, 1980). Children whose progress is restricted must be taught to learn faster through a focus on features of teaching designed to improve efficiency of instruction. These features derive from the design of instruction, and from process variables such as how the curriculum is implemented. Curriculum is designed with the goal of "faultless instruction" (Engelmann, 1980), that is, sequences or routines for which there is only one logical interpretation. The designer's brief is to avoid ambiguity in instruction - the focus is on logical-analysis principles. These principles allow the organisation of concepts according to their structure and the communication of them to the learner through the presentation of positive and negative examples.

Engelmann (1980) highlighted four design principles:

(i) Where possible teach a general case, that is, those skills which, when mastered, can be applied across a range of problems for which specific solutions have not been taught, for example, decoding regular words. These generalisations may be taught inductively, by examples only, or deductively, by providing a rule and a range of examples to define the rule's boundaries.

(ii) Teach the essentials. The essentials are determined by an analysis of the skills necessary to achieve the desired objective. There is an underlying assertion that, for reading, it is possible to achieve skilled reading by task analysis and the teaching of subskills within a cumulative framework. Advocates of a "Whole Language" perspective would disagree with the possibility or desirability of teaching in this manner.

(iii) Keep errors to a minimum. Direct Instruction designers consider errors counter-productive and time-wasting. For remedial learners a high success rate is useful in building and maintaining motivation lost through a history of failure. This low error rate is achieved by the use of the instructional design principles elucidated in Theory of Instruction (Englemann & Carnine, 1982) and by ensuring students have the pre-skills needed to commence any program (via a placement test).

(iv) Adequate practice. Direct Instruction programs include the requirement for mastery learning (usually above 90% mastery). Students continue to focus on a given task until that criterion is reached. The objective of this strategy is the achievement of retention without the requirement that all students complete the identical regimen. The practice schedule commences with massed practice, shifting to a spaced schedule. The amount of practice decreases as the relevant skill is incorporated into more complex skills. Advocates of Direct Instruction argue that this feature of instruction is particularly important for low-achieving students and is too often allowed scant regard (Engelmann, 1980). Whereas, this emphasis on practice may be unfashionable, there is considerable supporting research, and a number of effective schools are increasingly endorsing its importance (Rist, 1992; Thompson, Ransdell, & Rousseau, 2005). "The strategies that have fallen out of style, such as memorising, reciting and drilling, are what we need to do. They're simple - but fundamental - things that make complex thinking possible" (Rist, p. 19).

Roots of the Direct Instruction Model

It is these principles of instructional design that sets Direct Instruction apart from traditional and modern behavioural approaches to teaching. However, the model does share a number of features with other behavioural approaches (e.g., reinforcement, stimulus control, prompting, shaping, extinction, fading), and with the effective teaching movement (mastery learning, teacher presentation skills, academic engaged time, and correction procedures). These latter features have been researched thoroughly over the past 40 years, and have generally been accepted as comprising "direct instruction" (Gersten, Woodward, & Darch, 1986).

Rosenshine (1979) used the expression to describe a set of instructional variables relating teacher behaviour and classroom organisation to high levels of academic performance for primary school students. High levels of achievement were related to the amount of content covered and mastered. Hence the pacing of a lesson can be controlled to enhance learning. Academic engaged time refers to the percentage of the allotted time for a subject during which students are actively engaged. A range of studies (Rosenshine & Berliner, 1978) has highlighted the reduction in engagement that occurs when students work alone as opposed to working with a teacher in a small group or as a whole class. The choral responding typical of DI programs is one way of ensuring high student engagement. The author once counted 300 responses in the 10 minutes of teacher directed decoding activity in a Year 7 reading group (Hempenstall, 1990).

A strong focus on the academic was found to be characteristic of effective teachers. Non-academic activities, while perhaps enjoyable or directed at other educational goals, were consistently negatively correlated with achievement. Yet, in Rosenshine's (1980) review of studies it was clear that an academic focus rather than an affective emphasis produced classrooms with high student self-esteem and a warm atmosphere. Less structured programs and teachers with an affective focus had students with lower self esteem. Teacher centred rather than student centred classrooms had higher achievement levels. Analogously, teachers who were strong leaders and did not base their teaching around student choice of activities were more successful. Solomon and Kendall (1976) cited in Rosenshine (1980) indicated that permissiveness, spontaneity and lack of classroom control were " … negatively related, not only achievement gain, but also to positive growth in creativity, inquiry, writing ability, and self esteem for the students in those classrooms” (p. 18).

The instructional procedure called demonstration-practice-feedback (sometimes model-lead-test) has strong research support (Rosenshine, 1980). This deceptively simple strategy combines three elements of teaching strongly related to achievement in one general model. It comprises an invariant sequence in which a short demonstration of the skill or material is followed by guided practice during which feedback is provided to the student (and further demonstration if necessary). The second phase usually involves response to teacher questions about the material previously presented. It would appear that the overlearning this phase induces is particularly valuable. The third phase, that of independent practice, is evaluated by the teacher.

Medley's (1982) review indicated the efficacy for low SES students of a controlled practice strategy involving low cognitive level questions, a high success rate (above 80%), and infrequent criticism. Thus, the popularity among teachers of high cognitive level question implicit in discovery learning models is difficult to justify empirically. These high level questions require students to manipulate concepts without having been shown how to do so. Research on discovery approaches has indicated a negative relationship with student achievement. Winnie's (1979) review of 19 experimental studies on higher order questions made this point very strongly, as does Yates (1988).

To summarise the findings of research into teacher variables with a positive impact on student learning, Rosenshine and Berliner (1978) provide a definition for direct instruction (little di), a concept providing part of the theoretical basis for Direct Instruction.

"Direct instruction (di) pertains to a set of teaching behaviours focused on academic matters where goals are clear to students; time allocated for instruction is sufficient and continuous; content coverage is extensive; student performance is monitored; questions are at a low cognitive level and produce many correct responses; and feedback to students is immediate and academically oriented. In direct instruction, the teacher controls the instructional goals, chooses material appropriate for the student's ability level, and paces the instructional episode" (p. 7).

Big DI

Direct Instruction has developed into a comprehensive system of instruction covering many skill areas: reading, mathematics, language, spelling, microcomputing, writing, reasoning, and a variety of other school subjects including chemistry, critical reading, social studies, and history. Thus, the approach that initially restricted its emphasis to basic skills then expanded into higher order skills (Kinder & Carnine, 1991), has a long research base, and continues to have unfulfilled promise as part of a solution to the problems of illiteracy in our community.

Evaluation of the Direct Instruction Model

A very large national evaluation of different approaches to teaching basic skills was entitled Operation Follow Through. This evaluation showed that the Direct Instruction approach was particularly effective. For a discussion of FT, see Adams (1996), Becker & Gersten (1982), Engelmann, Becker, Carnine, & Gersten (1988), Grossen (1995), and Watkins (1996).

Additional to the Follow Through data, there were numerous evaluations of Direct Instruction programs from the early days, but as with much educational research, relatively few studies met the criteria for acceptability that are demanded today. Fabre (1984) compiled an annotated bibliography of almost 200 studies completed prior to 1984. For the most part, research findings were impressive, given the caveat of limited research design quality. Notable positive reviews of outcome research were provided by Gersten, 1985; Gregory, 1983; Kinder and Carnine, 1991; Lockery and Maggs, 1982; White, 1988. See later for contrary views.

Whereas, Direct Instruction was originally designed to assist disadvantaged students, its emphasis on task characteristics and effective teaching principles transcends learner characteristics, offering value across a range of learners. Willingham and Daniel (2012) made a similar point in noting that … “Research shows that instruction geared to common learning characteristics can be more effective than instruction focused on individual differences” (p.16).

Lockery and Maggs (1982) reviewed research indicating success with average children, those with mild, moderate or severe skill deficits, those in resource rooms, withdrawal classes and special classes in regular schools, disadvantaged students (including indigenous and those whose first language is not English), students in special facilities with varying degrees of intellectual disability, and physical disabilities.

Gersten (1985) in his review of studies involving students with a range of disabilities concluded that Direct Instruction tended to produce higher academic gains than traditional approaches. He also suggested that the mastery criterion (in excess of 90%) may be particularly important for special education students, and called for more formative evaluation where only one instructional variable is manipulated, and also, for more instructional dimensions research to highlight those variables alone or in company that are associated with academic gains. Gersten referred to the Leinhardt, Zigmond, and Cooley (1981) study with 105 learning disabled students. The authors noted that three teaching behaviours were strongly associated with student progress in reading - the use of reinforcers, academic focus, and a teacher instruction variable involving demonstration, practice and feedback. Each of these is critical to the definition of direct instruction (Rosenshine, 1979) and supports the assertion that there are teacher behaviours that transcend student characteristics. This study was the first to demonstrate that specific direct instruction principles have value for learning disabled students.

White's (1988) meta-analysis of studies involving learning disabled, intellectually disabled, and reading disabled students restricted its focus to those studies employing equivalent experimental and comparison groups. White reported an effect size of 0.84 standard deviation units for DI over comparison treatments. This is markedly above the 0.25-0.33 standard for educational significance of an educational treatment effect (Stebbins, St. Pierre, Proper, Anderson, & Cerva, 1977). White concluded that " ... instruction grounded in Direct Instruction theory (Engelmann & Carnine, 1982) is efficacious for both mildly and moderately/severely handicapped learners, and in all skill areas on which research has been conducted" (p. 372).

Further support for the explicit approach came from Kavale (1990). His summary of research into direct instruction and effective teaching concludes that they are five to ten times more effective for learning disabled students than are practices aimed at altering unobservable learning processes such as perception. Binder and Watkins (1990) described Direct Instruction (along with Precision Teaching) as the approaches best supported by research to address the problems of teaching found in the English-speaking world.

So, evaluations of DI extend back in time. For other, more recent evaluations, see http://www.adihome.org/adi-blog/entry/reviews-supporting-direct-instruction-program-effectiveness.

Note in particular this recent summary:

"One of the common criticisms is that Direct Instruction works with very low-level or specific skills, and with lower ability and the youngest students. These are not the findings from the meta-analyses. The effects of Direct Instruction are similar for regular (d=0.99), and special education and lower ability students (d=0.86), higher for reading (d=0.89) than for mathematics (d=0.50), similar for the more low-level word attack (d=0.64) and also for high-level comprehension (d=0.54), and similar for elementary and high school students. The messages of these meta-analyses on Direction Instruction underline the power of stating the learning intentions and success criteria, and then engaging students in moving towards these". Summarised from (Hattie, 2009, p. 206-7).

So what are these criticisms of Direct Instruction?

 Despite the long history of empirical support for Direct Instruction, unsurprisingly there have also been criticisms. Surely, no other approach has so polarised educators as has DI. The criticisms have been based on a number of different grounds. Some are fanciful, some shallow, some purely emotional, and many result from ideologically based beliefs regarding learning.

(a) Conspiracy theories

•  DI is an IBM/ McGraw-Hill conspiracy to oppress the masses/profiteer (Kohn, 2002; Nicholls, 1980).

• DI is a Christian right wing conspiracy (Berliner, 1996).

• “Critics contend that tack [ accepting DI] threatens to mandate the rote teaching style favored by religious conservatives and back-to-basics zealots” (Learn, 1998).

• It is designed to fuel a “global workforce training agenda” (Iserbyt, 1999, p.150). It’s to indoctrinate students into submitting to a life in the unskilled workforce (Shannon, 2007).

• It’s really about indoctrination: When I returned to the United States I realized that America’s transition from a sovereign constitutional republic to a socialist democracy would not come about through warfare (bullets and tanks) but through the implementation and installation of the “system” in all areas of government—federal, state and local. The brainwashing for acceptance of the “system’s” control would take place in the school—through indoctrination and the use of behavior modification, which comes under so many labels: the most recent labels being Outcome-Based Education, Skinnerian Mastery Learning or Direct Instruction (Iserbyt, 1999, p. XV).

• There is a conspiracy among researchers, publishers, and policy makers (Goodman, 2002; Zemelman, Daniels, & Bizar, 1999). "The research evidence is being distorted and purposefully misrepresented in ideologically consistent ways, in politically consistent ways, in reliably profitable ways" (Allington, 2002).

(b) It has negative side effects

•  “DI is a teaching method that bypasses the brain and causes an unnatural reflex that is controlled and programmed. This manipulation causes some students to become so stressed that they actually become ill and/or develop nervous tics” (Hayes, 1999).

• "It really does damage kids: socially, morally, as well as intellectually. It’s just too narrow and constrictive" Rheta DeVries, professor of curriculum and instruction, Regent’s Center for Early Development and Education, University of Northern Iowa.

• “Tullis makes the claim that "early exposure to academics" has the potential "to psychologically damage developing brains," and can lead to physical health problems, including (but presumably not limited to) "depression, anxiety disorders--even cardiovascular disease and diabetes." (Engelmann, 2012, p.1).

• DI produces more felony arrests, and more time in special education for emotional impairment, lower level school completion, and fewer living with their spouses. (HighScope) http://www.highscope.org/file/Research/high_scope_curriculum/preschool_validity.pdf

• DI students can’t think "When they have to think for themselves, they’re waiting to be told how." Psychologist Rebecca Marcon, www.titlei.com/samples/direct.htm

• It's “fact accumulation at the expense of thinking skill development” (Edwards, 1981) 

• DI damages students, causing delinquency (Schweinhart, Weikart, & Larner, 1986. Further, its "side effects may be lethal" (Boomer, 1988, p. 12). “It (direct instruction) is a scripted pedagogy for producing compliant, conformist, competitive students and adults.”

• "It's extremely authoritarian," observes Larry Schweinhart of the High Scope/Perry Research Project in Ypsilanti, Mich., and can lead children to "dependency on adults and resentment" (Duffrin, 1996, p.4) (cited in Coles, 1998). See also McKeen et al., 1972.

• “Direct Instruction has become today's federally-sanctioned child abuse for poor children” (Horn, 2007).

• “Clearly, the aid with the most strings of dependence for both teachers and students comes from the University of Oregon's Direct Instruction Model because it makes the teacher rely completely on curriculum programmers; it ignores the experience and knowledge of students altogether with its standardization of methods, making students dependent on lessons to learn to read.” (Shannon, 1988, pp. 36-37)

(c) Its view of the reading process is wrong (Gollash, 1980).

•  DI focusses on phonics, which is a bad approach (Meyer, 2003).

• It focusses on sight words: “Directed Instruction, although it gives lip service (pardon the pun) to phonics, seems to weight the instruction unavoidably in the direction of sight reading merely by virtue of group oral recitation from the text” (Fritzer & Herbst, 1999, p. 46).

• It emphasises phonics at the expense of comprehension (Jordan, Green, & Tuyay, 2005).

• It’s rote learning only, and doesn’t lead to conceptual understanding and problem solving (Ewing, 2011).

• "It's rote, it's memorization, it's not good solid practice," Karen Smith, associate director of the National Council of Teachers of English. "It goes against everything we think."(Duffrin, p.1)

• DI produces "A Nation of Rote Readers" (Coles, 2001).

(d) It is incompatible with other more important principles:

•  Normalisation (Penney, 1988).

• The wholistic nature of reading (Goodman, 1986; Giffen, 1980)

• A naturalistic educational paradigm (Heshusius, 1991).

• Flexible reciprocal child-teacher interaction (Ashman & Elkins, 1990).

• Teacher professionalism and creativity - the scripts deskill teachers. (Denise, 2008; McFaul, 1983). “the proletarianization of teacher work” (Giroux, 1985, p. 376).

• Constructivism which asserts that students create their own knowledge rather than simply absorbing information presented by others, like teachers. DI is antithetical to the constructivist attitude that there are multiple representations of reality, none of which is automatically nor necessarily superior or inferior to the others. Duffy (2009) reflects how ―”the direct-instruction researchers have focused on research in which variables are manipulated in tightly controlled experiments.…[whereas] the constructivist approach is to study rich learning environments, examining the variables in the context of those environments” (p. 354-355). You thus can’t compare the two approaches - like comparing apples with oranges. “Each relies on intellectual biases that would leave the other at a disadvantage were we to compare results” (Jonassen, 2009, p. 29). Confrey (1990) puts the constructivist position as being incompatible with Direct Instruction “We can have no direct or unmediated knowledge of any external or objective reality. We construct our understanding through our experiences, and the character of our experience is influenced profoundly by our cognitive lenses (p.108).

(e) The success of DI is illusory:

•  It is based on tests that do not measure real reading (Cambourne, 1979, Kohn, 1999; Nicholls, 1989).

• The apparent research support is not persuasive because empirical research can’t answer questions of superiority of methods (Weaver, 1988). “We don’t have an approach, we have a philosophy” (Horsch Erikson Institute, cited in Duffrin, 1996, p.6)

• It can’t work because it’s wrong: “ … in education, a priori beliefs about the way children ought to learn or about the relative value of different kinds of knowledge seem to have tremendous force in shaping judgments about effectiveness” (Traub, 2002).

• Operation Follow Through did not prove DI was effective (Kohn, 1999).

(f) Other approaches are more effective, for example,

•  Whole Language (Weaver, 1991),

• Discovery learning (Bay, Staver, Bryan, & Hale, 1992);

• As effective as DI (Kuder, 1990; O’Connor et al., 1993).

 (g) It may be inappropriate for certain sub groups.

•  Those in special education (Heshusius, 1991; Kuder, 1991; Penney, 1988).

• Those with certain learning styles, for example, those with an internal locus of control (McFaul, 1983; Peterson, 1979).

• Learning disabled students: “The failure of Direct Instruction to teach learning disabled children to read seems to be related to bad instructional design” (Allington, 2003).

• Those of high ability (Peterson, 1979).

• It’s not appropriate for indigenous students (Ewing, 2011; Sarra, 2011).
• It's only for the poor and at-risk (Eppley, 2011)

(h)       Its use is best restricted to basic skill development (Peterson, 1979).

 (i)           It is best used in conjunction with other approaches (Delpit, 1988; Gettinger, 1993; Harper, Mallette, Maheady, & Brennan, 1993; Spiegel, 1992; Stevens, Slavin, & Farnish, 1991).

(j)        Students might not find it acceptable (Reetz & Hoover, 1992).

• It destroys motivation by having students practise too much. “Heavy doses of practice with exercises that seem pointless to children further deaden interest and thinking” (Baroody & Ginsburg, 1990, p.58).

(k)       Relationships, not instruction, are what evoke learning (Sarra, 2011; Smith, 2003).

(l)       A lack of basic humanity.

•  Aspects of the programs, such as prescribed curriculum materials and instructions, are viewed as dehumanizing because they are centred in teaching materials rather than in people (Goodman, 1998).

• Siegfried Engelmann’s DISTAR (Reading Mastery) and ECRI are both based on the very sick philosophical world view that considers man nothing but an animal” (Iserbyt, 1999, p.212).

• DI renders learners passive (Becher, 1980; Johnson, 2004). “Indeed, it is often regarded as offensive to students, assuming they can only learn from a script; and offensive to educators, assuming they can only teach from a script; and both scripts are written by some old guy in the US” (Sara, 2011). It's "pouring of information from one container, the teacher’s head, to another container, the student’s head" (Brown & Campione, 1990, p. 112).

•  It's too regimented (Borko & Wildman, 1986)   

 (m)      It’s simply old-fashioned teaching:

• “ … the heart of Direct Instruction is group chanting (while following a text) in response to the teacher's scripted hand signals, analogous to the old "blab schools" of the 19th century, in which students recited in groups to memorize and feed back material” (Fritzer, & Herbst, 1999, p.45). … “lock step focus on drill and rote learning” (Fogarty & Schwab, 2012).

 (n)       It’s just Skinnerian behaviourism.

•  … instructional approaches now being imposed are something that most in the audience wouldn’t want their own children to suffer. These approaches have, he said, more to do with teaching rats than humans. He urged his audience to reclaim good instruction with attention to the lessons of social constructionism instead of treating students with a behaviorist approach in which, as B.F. Skinner proved, even pigeons can be taught to play ping-pong … DI is a steroidal scripted behaviorist methodology very popular with urban school policymakers and the Reading First thugs who make their curricular choices for them in Title I schools. No middle class suburban parent would ever permit this kind of cognitive decapitation of their children” (Horn, 2007).
• “Engelmann’s DI claims to be scientific as it rests upon the outmoded behaviourism of B.F. Skinner, an approach buried by Chomsky in his review of verbal behaviour in way back in 1967” (Sarra, 2011, p.1).

(o)      It only looks good because it’s old.

 "One of the problems is that to have proven programs, you have to have old programs," adds Richard L. Allington, the chairman of the reading department at the State University of New York at Albany. "Most of these Direct Instruction programs have been around 25 or 26 years, which is why there's more 'research' on them." If Direct Instruction looks good, Mr. Allington and others say, it may be because there is a dearth of effectiveness data on anything else” (Viadero, 1999).

(p)      It ignores higher order thinking, and, further, stifles it (Doyle, Sanford, & Emmer, 1983).

• Teaching is didactic, so students don’t learn how to have discourse among themselves (Ewing, 2011).

(q)      Zig shouldn’t be taken seriously:

• “an obscure educationist named Engelmann” (Rundle, 2009, p.1).

• “written by some old guy in the USA” (Sarra, 2011).

• “Engelmann DI advocates are not like most quality educators. They are zealots convinced they have the one true faith and the rest of us are heretics.”

  Sarra, C. (2011). Not the only way to teach Indigenous students. National Indigenous Times. Retrieved from https://chrissarra.wordpress.com/2011/05/26/not-the-only-way-to-teach-indigenous-students/

(r)        The effects may be short lived:

 The Coalition for Evidence-Based Policy (2012) does not include Direct Instruction among its list of evidence-based approaches because of their perception of a lack of long term effect studies.

This from a personal communication (July 11, 2012) from a Coalition spokesperson.

"We have reviewed the evidence supporting Direct Instruction and our overall thought is that, while a number of studies have found promising short-term effects of the model, more rigorous evaluations with longer-term follow-ups are needed to determine whether it produces sustained effects on important academic and behavioral outcomes. The reason we look for evidence of sustained effects is to rule out the possibility that any observed short-term effects quickly fade away, a phenomenon which is unfortunately quite common in education. There have been a handful of such long-term studies of Direct Instruction, but they’ve tended to suffer from key limitations that make it difficult to draw firm conclusions about its sustained effectiveness (e.g., because studies had very small sample sizes or Direct Instruction was combined with other interventions when evaluated)" (para 1, 2).

Jean Stockard has performed the huge task of compiling a DI research database to enable those interested to study the research themselves and make decisions about program evidence. Find it here http://nifdi.org/docs/doc_download/205-di-bibliography-1412. See also Cristy Couglin’s: Research on the effectiveness of Direct Instruction programs: An updated meta-analysis at http://nifdi.org/di-research-database?controller=publications&task=show&id=142

  (s)       A lack of methodological soundness in the research

 The What Works Clearinghouse rejects most of the Direct Instruction studies as not meeting their criteria for methodological soundness, and ignores those older than 20 years or so. There has been much criticism over the last 5 years (Briggs, 2008; Carter & Wheldall, 2008; Greene, 2010; Engelmann, 2008; McArthur, 2008; Reynolds, Wheldall, & Madelaine, 2009; Slavin, 2008; Stockard, 2008, 2010, 2013; Stockard & Wood, 2012, 2013). This criticism has included the criteria used and the inconsistent application of those criteria. For a detailed analysis as applied to WWC determinations about DI programs, see Jean Stockard’s analysis at http://www.nifdi.org/documents-library/doc_download/270-2013-1-examining-the-what-works-clearinghouse-and-its-reviews-of-direct-instruction-programs

Most of the criticisms described above have been ably dealt with by Adams, (2004), Adams and Engelmann (1996), Adams and Slocum (2004), Barnes 91095), Carnine (1992, 1994), Ellis and Fouts (1997), Engelmann (2002), Kozloff (2009), and Tarver (1995, 1998).

Of the literature critical of the DI model, much is based on philosophical issues concerning reality and power; on theoretical issues such as the nature of the learning process, the role of teaching, or issues of measurement. Of the few studies in which alternative approaches have proved equivalent or superior, issues of treatment fidelity have arisen. It is rarely made clear whether the model described is the Direct Instruction model or a direct instruction clone of unknown rigour. Nor is it usually specified whether the teachers of any Direct Instruction program have been provided with the training required to ensure the programs are presented according to the presentation protocols.

A surprising feature of much of the criticism is the degree of venom present. It appears that in many of the papers, a great antipathy underpins their criticism. There is little pretence of objectivity, and the language is often emotional. One can’t help but wonder what it is about this model that evokes such ire. DI is a “harsh, inflexible, and depersonalizing approach” … I’d “like to see a stake driven in the heart of DISTAR” (Jalongo, 1999, p. 139).

The prevailing subtext seems to be that the writer doesn’t approve of the system because it contradicts the philosophy/beliefs of the writer. It must be wrongheaded because constructivism is right, and this system doesn’t fit with constructivism. In logic, this error is called begging the question. "It's rote, it's memorization, it's not good solid practice," says Karen Smith, associate director of the National Council of Teachers of English. "It goes against everything we think” (Duffrin, 1996, p.4).

Perhaps the most egregious aspect of the criticisms is that relatively few dispute the effectiveness of the approach. It appears that, for most, the outcomes are not in dispute, but the process is not one with which many teachers feel comfortable. Thus the dismissal of DI appears to place teacher comfort before student success.

Scripts and human error

 Recently, I was watching a program called Life, Death And Mistakes, which focussed upon human error in various performance fields, and what steps different occupations are beginning to adopt to reduce the impact of these human factors on performance. It gave me pause to think about instructional scripts that are such an important part of DI programs.

I was struck by the manner in which other professions, far from being offended by protocols and checklists have learned to rely on (and benefit from) them. The program showed examples of airplane pilots in cockpit emergencies, surgical teams in patient crisis situations, fire personnel in dangerous settings, post-operative transfer medicos, and Formula One pit crews – all making use of these strategies to reduce human error, save lives (both their own and those of their charges), and increase their efficiency. There was no suggestion from the various individuals that their creativity was stifled, or that their work became demeaning. In fact, job satisfaction was elevated and stress more easily managed when they knew that they didn’t have to “wing it”. Interestingly, protocols and checklists are commonly employed in those schools and districts that have adopted Response to Intervention as their framework for preventing and ameliorating student failure.

Perhaps this is a signpost to a future in education too. Like all occupations, teachers are fallible, and thus prone to human error. The issue is how to reduce this error across organisations generally, not solely for managing emergency situations. Such as those involved in education. Variability in instruction was once seen as a quality to be promoted. I recall in Victoria, in the early stage of the whole language domination of curriculum, that there were Innovation Grants available to teachers who could devise a plan that involved them doing something different. As a visiting psychologist in the school system, I saw some of those innovations in practice. They were mostly an embarrassment to education, as pet theories reigned supreme without any requirement for the evaluation of subsequent student performance. I had applied for a grant to use Corrective Reading in a local high school, as it had never been used in the region at that time. However, the application was rejected because the WL panel considered DI to be discredited, and inappropriate for students.

A comment in the Life, Death And Mistakes TV program stood out for me, and my paraphrase of it is: “It’s the idea that you standardise everything you can, and only in those circumstances that are unforeseeable do you need to improvise. That surety is what makes our work better.” Consider the contrast between that model and the (once?) popular whole language edict that you teach “in the moment”, responding continuously in an ad hoc (but invariably brilliant) manner to each student’s needs.

 EBP in medicine and psychology

 During the 1990s, while evidence-based practice (EBP) in medicine was being discussed, the American Psychological Association (Chambless & Ollendick, 2001) introduced the term empirically supported treatments as a means of highlighting differential psychotherapy effectiveness. Prior to that time, many psychologists saw themselves as practising a craft in which competence arises through a combination of personal qualities, intuition, and experience. The result was extreme variability of effectiveness among practitioners, a problem also evident in education. The proposal was to devise a means of rating therapies for various psychological problems, and for practitioners to use these ratings as a guide to practice. The criteria for a treatment to be considered well established required efficacy to be established through two controlled clinical outcomes studies or a large series of controlled single case design studies. It also insisted on treatment manuals to ensure treatment fidelity, and the provision of clearly specified client characteristics for the study in question. A second level involved criteria for probably efficacious treatments. These criteria required fewer studies, and/or a lesser standard of rigour. The third category comprised experimental treatments – those without sufficient evidence to achieve probably efficacious status.

There are obvious similarities between these therapy requirements and the criteria for acceptability for studies demanded by evaluation bodies such as the What Works Clearinghouse.

There was significant resistance displayed by practitioners towards the adoption of EBP in the fields of medicine and psychology. However, as the principles have been espoused in these professions since the early nineties, a new generation of practitioners has been exposed in their training and generally accepted EBP as the normal standard for practice. This has occurred among most young practitioners because their training has emphasized the centrality of evidence in competent practice. The notion of manualised treatments is one with which they feel very comfortable. That is not to say that the principles of EBP are always adhered to in practice. The older brigade as a group are less accepting of change - "We often know something doesn't work, but out there are thousands and thousands of doctors who have been taught certain procedures and that's all they do … changing of clinician beliefs and behaviour, even in the face of credible evidence, remains highly challenging (p.5)" (Medew, 2012).

There is evidence that many teachers feel that they have “wing it” in their approaches to the teaching of reading (Cunningham, Perry, Stanovich, & Stanovich, 2004; Leader-Janssen & Rankin-Erickson, 2013; Spear-Swerling, Brucker, & Alfano, 2005), because neither their pre-service nor their in-service training has equipped them adequately for the task. In education, the equivalent of manualised treatment is scripted instruction, and it has been derided by many in the education profession, as outlined earlier. As we’ve seen, this is in stark contrast to other fields in which the benefits have become apparent, and have outweighed the understandable resistance to changing one’s practice.

Changed attitudes

 It is of interest that many DI teachers have altered their former discomfort when they perceive the effectiveness of the approach. Thus their attitude was changed by the experience of their and their students’ success according to numerous surveys (Bessellieu, Kozloff, & Rice, 2001; Cossairt, Jacobs, & Shade, 1990; Gersten, Carnine, & Cronin, 1986; Gervase, 2005; Hands, 1993; Proctor, 1989).

For example:

"Gersten et al. (1986) evaluated perceptions of teachers and paraprofessionals with regard to a Direct Instruction program. Teachers were interviewed toward the end of the first and second year of implementation. Initially, teachers were concerned with the high degree of structure leaving little room for fun activities and felt that scripted lessons were overly mechanical. At least half of the teachers believed that their teaching philosophy conflicted with that of Direct Instruction. By mid year, Gersten et al. found that teachers and paraprofessionals generally came to accept the program. By the end of the first year, attitudes had improved along with student achievement. Gersten et al. found that by the end of the second year of implementation, all but one teacher agreed with the main objectives of Direct Instruction as a program for educationally disadvantaged students" (Gervase, 2005, p.26-27).

What’s the future for systematic reviews like WWC?

 The issue of systematic reviews like WWC only considering gold standard research has created a new problem for education, and especially for evidence-based education. There are so few studies that meet criteria at present, that external validity has become a complicating issue, even when studies are gold standard.

The stumbling block that only the large scale, methodologically sophisticated studies are worthwhile somehow needs to be resolved. There are some alternatives: a single study involving a small number of schools or classes may not be conclusive in itself, but many such studies, preferably done by many researchers in a variety of locations, can add some confidence that a program's effects are valid (Slavin, 2003). If one obtains similar positive benefits from an intervention across different settings and personnel, there is added reason to prioritise the intervention for a large gold-standard study. There is a huge body of data out there that is no longer considered fit for human consumption. It seems such a waste that there are not currently analysis methods capable of making use of these studies.

It is important that issues of validity and reliability of the systematic reviews are continuously examined, and this process has been gathering momentum. The criticisms have been several: of the criteria for what constitutes acceptable research; of slowness in producing evaluations; of inconsistency in applying standards for what constitutes acceptable research; of the inclusion of studies that have not been peer reviewed; and of a failure to attend to fidelity of implementation issues in the WWC analyses. This latter criticism can be subsumed under a broader criticism of ignoring external validity or generalisation in the reviews.

The focus of syntheses must be on what has worked, that is, programs for which there is evidence of an aggregate effect that is internally valid. I would argue that such evidence, although certainly important, is necessary but not sufficient for those stakeholders enacting educational policies. What the superintendent of a school district wants to know is not so much what has worked but what will work. To be relevant, a good synthesis should give policy makers explicit guidance about program effectiveness that can be tailored to specific educational contexts: When and where will a given program work? For whom will it work? Under what conditions will it work the best? For causal inferences to be truly valid, both causal estimation and generalization should at the very least be given equal weight (Briggs, 2008, p.20).

The point here is that whilst RCT may provide the best bulwark against threats to internal validity, the acceptance of small scale and brief RCTs creates a strong threat to external validity. Thus, the large scale reviews have their own issues to deal with before they can be unquestioningly accepted as the royal road to truth. Further, it may also be quite some time before gold-standard research reaches critical mass to make decisions about practice easier.

It has also been queried whether educational research can ever have randomised control trials as the norm, however desirable that may appear to be. One issue is that the high cost of such research is not matched by the available funding. For example, The US D.O.E. spends about $80 million annually in educational research; whereas, the US Department of Health and Human Services provides about $33 billion for health research. In Australia, whilst the budgets for the provision of health and education services are roughly similar, the funding for health research is about 16 times that for educational research (Australian Bureau of Statistics, 2010). Another issue concerns the limitations on methodological purity in educational research. Students in schools cannot be routinely randomly selected for intervention as can occur in other settings, such as individual therapies in medicine and psychology. Thus, RCTs are arguably unlikely to ever form the great part of educational research.

Perhaps as a response to this dilemma, attention is now being paid to single case research as a possible valid and workable adjunct to RCTs in attempting to document what works in education settings. “The addition of statistical models for analysis of single-case research, especially measurement of effect size, offers significant potential for increasing the use of single-case research in documentation of empirically-supported treatments (Parker et al., 2007; Van den Noortagate & Onghena, 2003).” (Horner, Swaminathan, Sugai, & Smolkowski, 2012, p.271). In recognition of this potential, WWC released a document on single-case designs, providing and initial WWC standards for assessing any such single case studies (Kratochwill et al., 2010). In a generally supportive response Wolery (2013) offered a number of suggestions for improvement on this initial attempt.

Interestingly, early in 2013, the WWC agreed to reconsider their policies and procedures, and requested that interested groups/individuals make submissions. No announcement has yet been forthcoming.

So, where does that leave us? At least two perspectives that have been put forward are worthy of follow-up. O’Keefe et al. (2012) recognise that the current system needs improvement, but have a sense of optimism:

"Empirically supported treatment is still a relatively new innovation in education and the methods for conducting effective reviews to identify ESTs are in their infancy. Based on our experience in comparing these review systems we believe that review methods can and should continue to develop. There is still a great need to build review methods that can take advantage of a larger range of studies yet can provide recommendations in which practitioner can place a high degree of confidence" (p.362).

In contrast, Greene (2010) views the whole review enterprise with a more jaundiced eye, and urges consumers to rely upon their own resources.

"We have no alternative to sorting through the evidence and trying to figure these things out ourselves. We may rely upon the expertise of others in helping us sort out competing claims, but we should always do so with caution, since those experts may be mistaken or even deceptive" (Greene, 2010, para 15).

Given what has transpired thus far, it would seem premature for teachers, schools, and education policy makers to base their decision-making entirely on the results of systematic reviews such as those from WWC. The question “Are there any immediate shortcuts to discerning the gold from the dross?” appears to remain unresolved for the immediate future. This is an unfortunate situation as there does appear to be increasing acknowledgement of evidence-based practice as a beginning to produce a new direction for education.

So DI continues, and with greater organisational support than in the early years: the National Institute for Direct Instruction (NIFDI, http://www.nifdi.org/), and the Association for Direct Instruction (ADI, http://www.adihome.org/) in particular. It has grown from a small series of programs for basic skill development up to a wide ranging series of programs that include higher order skills, for example, literary analysis, logic, chemistry, critical reading, geometry and social studies. Use has been made of technology through computer-assisted instruction, low cost networking and videodisc courseware, and, employing the model for languages other than English.

There seems little doubt that DI will continue to be a viable and productive model, although there still remains a question mark over the extent of adoption by the school system. The major hurdle continues to be its lack of attractiveness for many educators, and resultant absence of adoption into classrooms. More than thirty years ago, Maggs and White (1982) wrote despairingly, "Few professions are more steeped in mythology and less open to empirical findings than are teachers" (p. 131). In the same decade, Ruddell and Sperling (1988) expressed a general concern at the gulf between literacy research findings and teachers' practice. They call for research aimed at discovering why empirically proven practices are "thwarted, undermined, or ignored in the classroom" (p. 319). It is easy to find the same sentiments expressed today. For more on evidence-based practice, see http://www.adihome.org/adi-blog/entry/first-blog

The concern about the gulf between research and practice has been expressed for a long time. Roger's (1983, cited in Ruddell & Sperling) asserted that there is often a period of 25 to 35 years between a research discovery and its serious implementation. Surely that time is arriving any day now?

References

Adams, G.L. (1996). Project Follow Through: In-depth and beyond. In Adams, G., & Engelmann, S. (Eds.). Research on Direct Instruction. Seattle, WA: Educational Achievement Systems. Retrieved from http://pages.uoregon.edu/adiep/ft/adams.htm

Adams, G.L. (2004). The need for an independent review of the study conducted by Dr. Randall Ryder. Education News, March 04. Retrieved fromhttp://www.educationnews.org/need-for-an-independent-review.htm

Adams, G.L., & Engelmann, S. (1996). Research on Direct Instruction: 20 years beyond DISTAR. Seattle, WA: Educational Achievement Systems.

Adams, G.L., & Slocum, T.A. (2004). A critical review of Randall Ryder’s report of Direct Instruction Reading in two Wisconsin school districts. Journal of Direct Instruction, 4(2), 111–127.

Alferink, L.A., & Farmer-Dougan, V. (2010): Brain-(not) based education: Dangers of misunderstanding and misapplication of neuroscience research, Exceptionality, 18(1), 42-52.

Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2010). Does discovery-based instruction enhance learning? Journal of Educational Psychology, 103(1), 1-18.

Allington, R.L. (2002). Big brother and the national reading curriculum: How ideology trumped evidence. Portsmouth, NH: Heinemann.

Allington, R.L. (2003). In Leslie Poynor and Paula Wolfe (Eds.). Marketing fear in America's public schools: The real war on literacy (p.58).

Ashman, A., & Elkins, J. (1990). Educating children with special needs. New York: Prentice Hall.

Australian Bureau of Statistics. (2010). ABS Research and Experimental Development, All Sector Summary, Australia, 2008-09. Retrieved from http://www.abs.gov.au/AUSSTATS/abs@.nsf/DetailsPage/8112.02008-09?OpenDocument

Barnes, D. (1985). Why not direct instruction? The Australian Educational & Developmental Psychologist, 1, 59-62.

Baroody, A.J., & Ginsburg, H.P. (1990). Children's mathematical learning: A cognitive view. Journal for Research in Mathematics Education. Monograph, Vol. 4, Constructivist views on the teaching and learning of mathematics, 51-64, 195-210.

Bay, M., Staver, J. R., Bryan, T., Hale, J. B. (1992). Science instruction for the mildly handicapped: Direct instruction versus discovery teaching. Journal of Research in Science Teaching, 29, 555-570.

Becker, W. C. (1977). Teaching reading and language to the disadvantaged. What we have learned from field research. Harvard Educational Review, 47, 518-543.

Becker, W. C., & Gersten, R. (1982). A follow-up to Follow Through: The later effects of the direct instruction model on children in fifth and sixth grades. American Educational Research Journal, 19, 75-92.

Berliner, D.C. (1996). Educational psychology meets the Christian right: Differing views of children, schooling, teaching, and learning. Retrieved from http://courses.ed.asu.edu/berliner/readings/differingh.htm

Bessellieu, F.B., Kozloff, M.A., & Rice, J.S. (2001,Spring). Teachers’ perceptions of direct instruction teaching. Direct Instruction News, 14-18. Retrieved from http://people.uncw.edu/kozloffm/teacherperceptdi.html

Binder, C., & Watkins, C. L., (1990). Precision teaching and Direct Instruction: Measurably superior instructional technology in schools. Performance Improvement Quarterly, 3, 74-96.

Boomer, G. (1988). Standards & literacy. Two hundred years on the road to literacy: Where to from here? Directions: Literacy. Supplement to Education Victoria., June. Melbourne, Australia: State Board of Education.

Briggs, D.C. (2008). Synthesizing causal inferences. Educational Researcher, 37(1), 15-22.

Brown, A. L., & Campione, J. C. (1990). Interactive learning environments and the teaching of science and mathematics. In M. Gardner et al. (Eds.), Toward a scientific prac­tice of science education. Hillsdale, NJ: Erlbaum.

Byrne, B. (2011). Evaluating the role of phonological factors in early literacy development. In S. Brady, D. Braze, & C.A. Fowler (Eds.), Explaining individual differences in reading: Theory and evidence. New York: Psychology Press.

Cambourne, B. (1979). How important is theory to the reading teacher? Australian Journal of Reading, 2, 78-90.

Carnine, D. (1992). Introduction. In D. Carnine and E. J. Kameenui (Eds.), Higher order thinking: Designing curriculum for mainstreamed students. Austin, TX: Pro-ed

Carnine, D. (1994). Introduction to the mini-series: Educational tools for diverse learners. School Psychology Review, 23(3), 341-350.

Carter, M., & Wheldall, K. (2008). Why can't a teacher be more like a scientist? Science, pseudoscience and the art of teaching. Australasian Journal of Special Education, 32(1), 5-21.

Chambless, D. L. & Ollendick, T. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685- 716.

Clark, S.K., Jones, C.D., Reutzel, R., & Andreasen, L. (2013). An examination of the influences of a teacher preparation program on beginning teachers' reading instruction. Literacy Research and Instruction, 52(2), 87-105.

Coles, G. (1998, Dec. 2). No end to the reading wars. Education Week Retrieved from http://www.edweek.org/ew/vol-18/14coles.h18

Coles, G. (2001). Bush's 'scientific' vision: A nation of rote readers. Newsday, June 23. Retrieved from http://www.newsday.com/culture-watch-bush-s-scientific-vision-a-nation-of-rote-readers-1.801819

Confrey, J. (1990). What constructivism implies for teaching. Journal for Research in Mathematics Education Monograph, Vol. 4, Constructivist views on the teaching and learning of mathematics, 107-122, 195-210.

Cossairt, A., Jacobs, J., & Shade, R. (1990). Incorporating direct instruction skills throughout the undergraduate teacher training process: A training and research direction for the future. Teacher Education and Special Education, 13, 167-171.

Coughlin, C. (2011). Research on the effectiveness of Direct Instruction programs: An updated meta-analysis. Paper Presented at the Annual Meetings of the Association for Behavior Analysis International, May, 2011. Retrieved from http://nifdi.org/di-research-database?controller=publications&task=show&id=142

Cunningham, A. E., Perry, K. E., Stanovich, K. E., & Stanovich, P. J. (2004). Disciplinary knowledge of K–3 teachers and their knowledge calibration in the domain of early literacy. Annals of Dyslexia, 54(1), 139–167.

Delpit, L. D. (1988). The silenced dialogue: Power and pedagogy in educating other people's children. Harvard Educational Review, 58, 280-298.

Denise, G. (2008). Scripted curriculum: Scourge or salvation? Educational Leadership, 65, 80.

DiMagliaro, S., Lockee, B., & Burton, J. (2005). Direct instruction revisited: A key model for instructional technology. Educational Technology Research & Development, 53(4), 41-55. Retrieved from http://projects.ict.usc.edu/itw/materials/Lockee/DI%20Revisited.pdf

Doyle, W., Sanford, J. & Emmer, E. (1983). Managing academic tasks in junior high school: Background, design and methodology (Report No. 6185). Austin: University of Texas, Research and Development Center for Teacher Education.

Duffrin, E. (1996, Sep 1)). Direct Instruction making waves. Catalyst, 8(1), 1-7. Retrieved from http://www.catalyst-chicago.org/issues/1996/09/direct-instruction

Duffy, T. M. (2009). Building lines of communication and a research agenda. In S. Tobias & T. M. Duffy (Eds.), Constructivist Instruction: Success or failure? (pp. 351-367). New York: Routledge.

Ellis, A. K., & Fouts, J. T. (1997). Research on educational innovations. Larchmont, New York: Eye on Education, Inc

Engelmann, O. (2012). Letter to the editor of Scientific American Mind. The premature death of preschool. The National Institute for Direct Instruction. Retrieved from http://www.nifdi.org/documents-library/doc_download/243-the-death-of-preschool-letter-to-editor

Engelmann, S. & Carnine, D. (1982). Theory of instruction. New York: Irvington.

Engelmann, S. (1980). Toward the design of faultless instruction: The theoretical basis of concept analysis. Educational Technology, Feb. 80, 28-36.

Engelmann, S. (2002). A response to Allington. Education News, Mar 2. Retrieved from http://www.educationnews.org/articles/allington-leveled-serious-allegations-against-direct-instruction.html

Engelmann, S. (2008). Machinations of What Works Clearinghouse. Retrieved from http://zigsite.com/PDFs/MachinationsWWC(V4).pdf.

Engelmann, S., Becker, W. C., Carnine, D., & Gersten, R. (1988). The Direct Instruction Follow Through model: Design and outcomes. Education and Treatment of Children, 11(4), 303-317.

Eppley. K. (2011). Reading Mastery as pedagogy of erasure. Journal of Research in Rural Education, 26(13). Retrieved from http://jrre.psu.edu/articles/26-13.pdf

Ewing, B. (2011). Direct Instruction in mathematics: Issues for schools with high indigenous enrolments: A literature review. Australian Journal of Teacher Education, 36(5), 64-91.

Fabre, T. (1984). The application of direct instruction in special education. An annotated bibliography. Unpublished manuscript, University of Oregon.

Fogarty, W., & Schwab, R.G. (2012). Indigenous education: Experiential learning and learning through country. Working Paper No. 80/2012. Centre for Aboriginal Economic Policy, The Australian National University.

Fritzer, P., & Herbst, P. (1999). A cautionary tale: Directed instruction reading. Contemporary Education, 70(2), 45- 47.

Garrison, J., & MacMillan, C. (1994). Process-product research on teaching: Ten years later. Educational Theory, 44(4). Retrieved November 2009, from http://www.ed.uiuc.edu/EPS/Educational-Theory/Contents/44_4_Garrison.asp

Gersten, R. M. (1985). Direct instruction with special education students: A review of evaluation research. Journal of Special Education, 19(1), 41-58.

Gersten, R., Carnine, D., & Cronin D. (1986). A multifaceted study of change in seven inner-city schools. Elementary School Journal, 86, 257-276.

Gersten, R.M., Woodward, J., & Darch, C. (1986). Direct Instruction: A research based approach to curriculum design and teaching. Exceptional Children, 53(1), 17-31.

Gervase, S. (2005). Reading Mastery: A descriptive study of teachers attitudes and perceptions towards Direct Instruction. (Electronic Thesis or Dissertation). Retrieved from http://rave.ohiolink.edu/etdc/view?acc_num=bgsu1120001760

Gettinger, M. (1993). Effects of invented spelling and direct instruction on spelling performance of second-grade boys. Journal of Applied Behaviour Analysis, 26, 281-291.

Giffen, D. (1980). A letter-a-reply - an invitation to respond. Reading Around, 8, 84-88.

Giroux, H. (1985). Teachers as intellectuals. Social Education, 49(15), 376-379.

Gollash, F. (1980). A review of Distar Reading I from a psycholinguistic viewpoint. Reading Education, 5, 78-86.

Goodman, K. (2002). When the fail proof reading programs fail, blow up the colleges of education. Retrieved from http://tlc.ousd.k12.ca.us/~acody/goodman.html

Goodman, K. S. (1986). What's whole in whole language. Richmond Hill, Ontario: Scholastic.

Goodman, K. S. (1998). In defense of good teaching: What teachers need to know about the reading wars. Urbana, IL: National Council of Teachers of English.

Greenberg, J., McKee. A., & Walsh, K. (2013). NCTQ Teacher Prep Review. National Council of Teacher Quality. Retrieved from http://www.nctq.org/dmsStage/Teacher_Prep_Review_2013_Report

Greene, J.P. (2010). What doesn’t work clearinghouse. Education Next. Retrieved from http://educationnext.org/what-doesnt-work-clearinghouse/

Gregory, R.P. (1983). Direct Instruction, disadvantaged and handicapped children: A review of the literature and some practical implications. Parts 1 & 2. Remedial Education, 18(3), 108-114, 130-136.

Grossen, B. (Winter 1995-6). Overview: The story behind project Follow Through. Effective School Practices, 15, Retrieved from http://darkwing.uoregon.edu/~adiep/ft/grossen.htm

Hands, B.P. (1993). Measurement of teacher attitude to direct instruction. Perth, Edith Cowan University. Retrieved fromhttp://trove.nla.gov.au/work/153169822

Harper, G. F., Mallette, B., Maheady, L., & Brennan, G. (1993). Classwide student tutoring teams and Direct Instruction as a combined instructional program to teach generalizable strategies for mathematics word problems. Education & Treatment of Children, 16, 115-134.

Hattie, J. A.C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London and New York: Routledge.

Hayes, T.J. (1999). Traditional education vs. Direct Instruction. What's the difference between these teaching methods? Education Reporter, 156. Retrieved from http://www.eagleforum.org/educate/1999/jan99/focus.html

Hempenstall, K. (1990). Direct Instruction: Experiences of an educational psychologist. Paper presented at the 13th National Conference of the Australian Behaviour Modification Association. Melbourne, Australia.

Heshusius, L. (1991). Curriculum based assessment and direct instruction: Critical reflections on fundamental assumptions. Exceptional Children, 57, 315-328.

Horn, J. (2007). Direct Instruction: Learning for rats, applied to children. Schools Matter, July 29. Retrieved from http://www.schoolsmatter.info/2007/07/direct-instruction-learning-for-rats.html

Horner, R.H., Swaminathan, K., Sugai, G. & Smolkowski, K. (2012). Considerations for the systematic analysis and use of single-case research. Education and Treatment of Children, 35(2), 269-290.

Iserbyt, C.T. (1999). The deliberate dumbing down of America. Ravenna, Ohio: Conscience Press. Retrieved from www.deliberatedumbingdown.com/MomsPDFs/DDDoA.sml.pdf

Jalongo, M.R. (1999). Editorial: On behalf of children. Early Childhood Education Journal, 26(3), 139–141.

Johnson, G.M. (2004). Constructivist remediation: Correction in context. International Journal of Special Education, 19(1), 72-88.

Jonassen, D. (2009). Reconciling a human cogntivite architecture. In S. Tobias & T. M. Duffy (Eds.), Constructivist instruction: Success or failure? (pp. 13-33). New York: Routledge.

Jordan, N.L., Green, J., & Tuyay, S. (2005). Basal readers and reading as socialization: What are children learning? Language Arts, 82(3), 204-213.

Kavale, K.A. (1990). Variances & verities in learning disability interventions. In T. Scruggs & B. Wong (Eds.), Intervention research in learning disabilities (pp.3-33). New York: Springer Verlag.

Kinder, D., & Carnine, D. (1991). Direct Instruction: What it is and what it is becoming. Journal of Behavioural Education, 1(2), 193-213.

Kohn, A. (1999). Early childhood education: The case against Direct Instruction of academic skills. In Alfie Kohn (Ed.), The schools our children deserve. Boston: Houghton Mifflin. Retrieved from http://www.alfiekohn.org/teaching/ece.htm

Kohn, A. (2002, October). The 500-pound gorilla. Phi Delta Kappan, 84(2),112-119.

Kratochwill, T.R., Hitchcock, J., Horner, R.H., Levin, J. R., Odom, S.L., Rindskoph, D.M., & Shadish, W.R. (2010). Single-case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf

Kuder, S. J. (1990). Effectiveness of the DISTAR reading program for children with learning disabilities. Journal of Learning Disabilities, 23(1), 69-71.

Kuder, S. J. (1991). Language abilities and progress in a Direct Instruction reading program for students with learning disabilities. Journal of Learning Disabilities, 24, 124-127.

Leader-Janssen, E.M., & Rankin-Erickson, J.L. (2013). Preservice teachers' content knowledge and self-efficacy for teaching reading. Literacy Research and Instruction, 52(3), 204-229.

Learn, S. (1998). Educators put reading to the test The Oregonian. Retrieved fom http://www.shearonforschools.com/direct_instruction.htm

Lemov, D., Woolway, E., & Yezzi, K. (2012). Practice perfect: 42 rules for getting better at getting better. San Francisco: Jossey-Bass.

Lockery, L. & Maggs, A. (1982). Direct instruction research in Australia: A ten year analysis. Educational Psychology, 2, 263-288.

Maggs, A. & White, R. (1982). The educational psychologist: Facing a new era. Psychology in the Schools, 19, 129-134.

Marchand-Martella, N.E., Martella, R.C., Modderman, S.L., Petersen, H.M., & Pan, S. (2013). Key areas of effective adolescent literacy programs. Education and Treatment of Children, 36(1), 161-184.

McArthur, G. (2008). Does What Works Clearinghouse work? A brief review of Fast ForWord®. Australasian Journal of Special Education, 32(1), 101-107.

McFaul, S. A. (1983, April). An examination of Direct Instruction. Educational Leadership, 67-69.

Medew, J. (2012). Push for tougher line on surgery. The Age, October 4, p.5.

Medley, D. M. (1982). Teacher effectiveness. In H. E. Mitzel (Ed.), Encyclopaedia of Educational Research (5th ed., Vol. 4). (pp. 1894-1903). New York: The Free Press.

Meyer, R. (2003). Captives of the script: Killing us softly with phonics. Rethinking Schools Online, 17(4). Retrieved from http://www.rethinkingschools.org/special_reports/bushplan/capt174.shtml

Nicholls, C. (1980). Kentucky Fried reading scheme. Reading Education, 6, 18-22.

O’Keeffe, B.V., Slocum, T.A., Burlingame, C., Snyder, K., & Bundock, K. (2012). Comparing results of systematic reviews: Parallel reviews of research on repeated reading. Education & Treatment of Children, 35(2), 333 – 366.ning

O'Connor, R. E., Jenkins, J. R., Cole, K. N., & Mills, P. E. (1993). Two approaches to reading instruction with children with disabilities: Does program design make a difference? Exceptional Children, 59, 312-323.

Penney, R. K. (1988, March). Compatability [sic] of early intervention programmes with normalization. Bulletin of the Australian Psychological Society, 28-29.

Peterson, P. L. (1979, Oct.). Direct instruction: Effective for what and for whom? Educational Leadership, 46-48.

Proctor, T. J. (1989). Attitudes toward direct instruction. Teacher Education and Special Education, 12, 40-45.

Reetz, L. J., & Hoover, J. H. (1992). The acceptability and utility of five reading approaches as judged by middle school LD students. Learning Disabilities, Research & Practice, 7, 11-15.

Reynolds, M., Wheldall, K., & Madelaine, A. (2009). The devil is in the detail regarding the efficacy of Reading Recovery: A rejoinder to Schwartz, Hobsbaum, Briggs, and Scull. International Journal of Disability, Development and Education, 56, 17-35.

Rist, M. C. (1992). Learning by heart. The Executive Educator, November, 12-19.

Rosenshine, B. V. (1979). Content, time and direct instruction. In P. L. Peterson, & H. J. Walbert (Eds.), Research on teaching: Concepts, findings and implications. (pp 28-57). Berkeley, CA: McCutchan.

Rosenshine, B. V. (1980). Direct instruction for skill mastery. Paper presented to the School of Education, University of Milwaukee, Wisconsin.

Rosenshine, B. V., & Berliner, D. C. (1978). Academic engaged time. British Journal of Teacher Education, 4, 3-16.

Ruddell, R. B. & Sperling, M. (1988). Factors influencing the use of literacy research by the classroom teacher: Research review and new directions. In J.E. Readence (Ed.), Dialogues in literacy research (pp. 319-329). Chicago: National Reading Conference Inc.

Rundle, G. (2009, Oct 29). Review: Noel Pearson’s Radical Hope. Retrieved from http://www.crikey.com.au/2009/10/23/rundle-review-noel-pearsons-radical-hope/

Sarra, C. (2011). Not the only way to teach Indigenous students. Retrieved from http://chrissarra.wordpress.com/2011/05/26/not-the-only-way-to-teach-indigenous-students/

Schweinhart, L. J., Weikart, D. P., & Larner, W. B. (1986). Consequences of three pre-school curriculum models through age 15. Early Childhood Research Quarterly, 1(1), 15-45.

     Shannon, P. (1988). Can we directly instruct students to be independent in reading? In J. R. Readence & S. Baldwin (Eds.), Dialogues in literacy research: Thirty-seventh yearbook of the National Reading Conference, (pp. 36-39). Chicago: National Reading Conference.

Shannon, P. (2007). Reading against democracy: The broken promises of reading instruction. Portsmouth, NH: Heinemann.

Slavin, R. E. (2003). A reader's guide to scientifically based research. Educational Leadership, 60(5), 12-16. Retrieved from http://www.ascd.org/publications/ed_lead/200302/slavin.html

Slavin, R.E. (2008). Evidence-based reform in education: Which evidence counts? Educational Researcher, 37(1), 47-50.

Slocum, T. A., Spencer, T. D., & Detrich, R. (2012). Best available evidence: Three complementary approaches. Education and Treatment of Children, 35(2), 153 – 181.

Smith, F. (2003). Unspeakable acts, unnatural practices flaws and fallacies in "scientific" reading instruction. NH: Heinemann.

Spear-Swerling, L., Brucker, P.O., &Alfano, M. P. (2005). Teachers' literacy-related knowledge and self-perceptions in relation to preparation and experience. Annals of Dyslexia, 55(2), 266-296.

Spiegel, D. L. (1992). Blending whole language and systematic direct instruction. The Reading Teacher, 46 (1), 38-44.

Stebbins, L., St. Pierre, R. G., Proper, E. C., Anderson, R. B., & Cerva, T. R. (1977). Education as experimentation: A planned variation model: Vol.IV. Cambridge, MA: Abt. Associates.

Stevens, R. J., Slavin, R. E., & Farnish, A. M. (1991). The effects of co-operative learning and direct instruction in reading comprehension strategies on main idea identification. Journal of Educational Psychology, 83, 8-16.

Stockard, J. (2008). The What Works Clearinghouse Beginning Reading reports and rating of Reading Mastery: An evaluation and comment. Technical Report 2008-04. National Institute for Direct Instruction. Retrieved fromhttp://www.nifdi.org/research/what-works-clearinghouse

Stockard, J. (2010). An analysis of the fidelity implementation policies of the What Works Clearinghouse. Current Issues in Education, 13(4). Retrieved from http://cie.asu.edu/

Stockard, J. (Spring, 2013). Examining the What Works Clearinghouse and its reviews of Direct Instruction programs. Technical Report 2013-1, National Institute for Direct Instruction. Retrieved from http://www.nifdi.org/documents-library/doc_download/270-2013-1-examining-the-what-works-clearinghouse-and-its-reviews-of-direct-instruction-programs

Stockard, J., & Wood, T.W. (2012). Reading Mastery and learning disabled students: A comment on the What Works Clearinghouse review. National Institute for Direct Instruction. Retrieved from http://www.nifdi.org/documents-library/doc_download/245-response-to-wwc-rm-and-ld-july-2012

Stockard, J., & Wood, T.W. (2013). The WWC Review Process: An analysis of errors in two recent reports. Technical Report 2013-4. Office of Research and Evaluation, National Institute for Direct Instruction. Retrieved from http://www.nifdi.org/documents-library/doc_download/283-technical-report-2013-4-wwc

Taber, K. S. (2010 July 6). Constructivism and Direct Instruction as competing instructional paradigms: An essay review of Tobias and Duffy‘s Constructivist Instruction: Success or Failure? Education Review, 13(8). Retrieved from http://www.edrev.info/essays/v13n8index.html

Tarver, S. G. (1995). Direct Instruction. In W. Stainback & S. Stainback (Eds.), Controversial issues confronting special education: Divergent perspectives. (2nd ed). Boston: Allyn & Bacon Publishing Company.

Tarver, S. G. (1998). Myths and truths about Direct Instruction (Dl). In Phonics and beyond: Literacy in the 21st century, conference proceedings books from the Sixteenth Annual Dyslexia Conference of the Wisconsin Branch of the International Dyslexia Society. Reprinted in Effective School Practices, 17(1), 18-22.

The Coalition for Evidence-Based Policy (2012). Interventions-for-children-age-0-6. Retrieved from http://toptierevidence.org/programs-reviewed/interventions-for-children-age-0-6

Thompson, S., Ransdell, M., & Rousseau, C. (2005). Effective teachers in urban school settings: Linking teacher disposition and student performance on standardized tests. Journal of Authentic Learning, 2(1), 22-36.

Traub, J. (2002). No Child Left Behind: Does it work? New York Times, Nov 10. Retrieved from http://www.nytimes.com/2002/11/10/education/no-child-left-behind-does-it-work.html?pagewanted=all&src=pm

Viadero, D. (1999). A direct challenge. Education Week, 18(27), 41-43. Retrieved from http://www.zigsite.com/DirectChallenge.htm

Watkins, C. L. (1996). Follow through: Why didn’t we? Effective School Practices, 15(1), 5.

Weaver, C. (1988). Reading: Progress and practice. Portsmouth, NJ: Heinemann.

Weaver, C. (1991). Weighing the claims about "Phonics First". The Education Digest, April, 19-22.

Weir, R. (1990). Philosophy, cultural beliefs and literacy. Interchange, 21(4), 24-33.

White, W. A. T. (1988). A meta-analysis of the effects of direct instruction in special education. Education & Treatment of Children, 11, 364-374.

Willingham, D.T., & Daniel, D. (2012). Teaching to what students have in common. Educational Leadership, 69(5), 16-21. Retrieved from http://www.ascd.org/publications/educational-leadership/feb12/vol69/num05/Teaching-to-What-Students-Have-in-Common.aspx

Winnie, P. (1979). Experiments relating teachers' use of higher cognitive questions to student achievement. Review of Educational Research, 49, 13-50.

Wolery, M. (2013). A commentary: single-case design technical document of the What Works Clearinghouse. Remedial and Special Education 34(1), 39-43.

Zemelman, S., Daniels, H., & Bizar, M. (1999, March). Sixty years of reading research -- But who's listening? Phi Delta Kappan. Retrieved from http://www.pdkintl.org/kappan/kzem9903.htm