Older students’ literacy problems: Who are they?
Fourth grade slump, and more.
Dr Kerry Hempenstall, Senior Industry Fellow, School of Education, RMIT University, Melbourne, Australia.
My blogs can be viewed on-line or downloaded as a Word file or PDF at https://www.dropbox.com/sh/olxpifutwcgvg8j/AABU8YNr4ZxiXPXzvHrrirR8a?dl=0
Reading interventions for struggling older students:
Is it hard? What does it take? What focus? How intense: How frequent? What duration? What's treatment fidelity?
What will I get for all the effort? How many will still make little progress?
Who are they?
“Older struggling readers fall into a wide range of developmental levels, presenting a unique set of circumstances not found in younger more homogeneous beginning readers (Biancarosa & Snow, 2004). These struggling adolescents readers generally belong to one of two categories, those provided with little or poor early reading instruction or those possibly provided with good early reading instruction, yet for unknown reasons were unable to acquire reading skills (Roberts, Torgesen, Boardman, & Sammacca, 2008). Additionally within these two categories, older struggling readers are extremely heterogeneous and complex in their remediation needs (Nation, Snowling, & Clarke, 2007; Torgesen et al., 2007)” (p.566).
Calhoon, M. B., & Prescher, Y. (2013). Individual and group sensitivity to remedial reading program design: Examining reading gains across three middle school reading projects. Reading and Writing: An Interdisciplinary Journal, 26, 565-592.
Fourth Grade Slump
Under the meaning centred approach to reading development, there is no systematic attention to ensuring children develop the alphabetic principle. Decoding is viewed as only one of several means of ascertaining the identity of a word – and it is denigrated as being the least effective identification method (behind contextual cues). In the early school years, books usually employ highly predictable language and usually offer pictures to aid word identification. This combination can provide an appearance of early literacy progress. The hope in this approach is that this form of multi-cue reading will beget skilled reading.
However, the problem of decoding unfamiliar words is merely postponed by such attractive crutches. It is anticipated in the meaning centred approach that a self-directed attention to word similarities will provide a generative strategy for these students. However, such expectations are all too frequently dashed – for many at-risk children progress comes to an abrupt halt around Year 3 or 4 when an overwhelming number of unfamiliar (in written form) words are rapidly introduced. This apparent stalling of progress became known as the fourth grade slump (Chall & Jacobs, 1983; Hirsch, 2003). The number of words a child requires to cope with grade level text in Year 2 was estimated by Carnine (1982) as between three and four hundred, and in Years 3 and 4 between three and four thousand. Share (1995) estimated that the average fifth year student encounters about ten thousand new words – an “orthographic avalanche” that overwhelms most of those without adequate decoding skills.
Strategies that rely upon memory-for-shapes of words, or picture-clues, or context-clues become unproductive (Spear-Swerling & Sternberg, 1994). This leaves a dependence largely on the students’ visual-recognition store of word shapes, and students too often have not developed any generative strategy for the decoding of these novel words. It is true that some children develop a working understanding of the alphabetic principle despite the absence of explicit instruction; however, those students who did not have the ‘Aha!’ experience tended to be left floundering without the structure necessary to progress (National Reading Panel, 2000). This circumstance often becomes apparent during fourth grade (though with appropriate assessment, the problem could have been uncovered in the first grade).
In the RMIT Psychology Clinic, I’ve lost count of the number of parents seeking assistance for their fourth grade child have lamented. “We often wondered if Jane’s reading progress was OK, but we were assured when we enquired each year that she was doing fine. Now this year’s report states that she’s way behind. How can this be?”
Some quotes on the fourth grade slump:
Some strugglers are simply not detected by their teachers (girls are especially vulnerable) until about Grade 4
“These results indicate that although the late emerging poor comprehenders showed below average performance for the first time in Grade 4, they had consistently lower performance on measures of word reading, nonword reading, and spelling across time. … Our findings are supported by other research, which has shown LERD prevalence rates between 2.8% and 13.4% (Catts et al., 2012; Compton et al., 2008; Kieffer, 2010). Together, these results suggest that earlier reports may have overestimated the number of children with late emerging reading problems and that the fourth grade slump is just that—a slump. … the majority, or 67%, of children with newly identified reading comprehension problems in Grade 4 were able to recover by the end of Grade 7. … Longitudinal studies have shown that children with LERD may have poorer phonological awareness, decoding efficiency, listening comprehension, vocabulary, grammar, and sentence imitation on kindergarten or Grade 1 measures (Catts et al., 2012; Compton et al., 2008). Our results indicate that working memory skills may be another indicator of late emerging difficulties in reading comprehension.” (p.33)
Etmanskie, J.M., Partanen, M., & Siegel, L.S. (2016). A longitudinal examination of the persistence of late emerging reading disabilities. Journal of Learning Disabilities, 49(1), 21–35.
“A longitudinal study of the 32 low income children (Snow, Barnes, Chandler, Goodman, & Hemphill, 1991) found that parents were not aware that their children were having problems. When they received report cards from the school with grades of B and C, they assumed that their children were making adequate progress. But teachers at the school gave such grades to children who were performing badly. Also, teachers were giving children credit for extent of improvement during the year rather than for actual achievement. These results indicate problems in communication between low-income parents and schools”.
Nicholson, T. (1998). Literacy, family and society. In G. B. Thompson & T. Nicholson (Eds.). Learning to read: Beyond phonics and whole language. New York: Teachers College Press.
“(In this study) the probability that a child who was initially a poor reader in first grade would be classified as a poor reader in the fourth grade was a depressingly high +0.88”.
Juel, C. (1988). Learning to read & write: A longitudinal study of 54 children from first through fourth grades. Journal of Educational Psychology, 80, 437-447.
“In a study of students from the Connecticut Longitudinal Study, Shaywitz et al found a research-identified incidence of reading disability of 8.7% of boys and 6.9% of girls. However, a teacher-identified incidence of the same population identified 13.6% of boys and only 3.2% of girls. The authors suggested that greater reports of behavioral difficulties among boys in the classroom may have led to this bias”.
Shaywitz, S.E., Shaywitz, B.A., Fletcher, J.M., & Escobar, M.D. (1990). Prevalence of reading disability in boys and girls. Journal of the American Medical Association, 264, 998-1002.
“’Meghan was doing fine in third grade. She had no trouble grasping third grade reading materials. But suddenly, in fourth grade, her rate of progress took a serious dive. Her teacher couldn't get her to move ahead. Meghan became increasingly frustrated. Her parents were worried. Meghan was suffering from what educators know as the infamous "fourth grade slump", the phenomenon that often occurs during the tricky transition students must make from learning to read to reading to learn. (p. 26)’ This quotation from a magazine article highlights the increased reading demands of the middle grades. These difficulties extend well into secondary school (Nicholson, 1984, 1988)”.
Nicholson, T. (1984). Experts and novices: A study of reading in the high school classroom. Reading Research Quarterly, 19, 436-458.
Nicholson, T. (1988). Reading and learning in the junior secondary school. Wellington, New Zealand: Department of Education.
Some strugglers have been taught strategies that break down around about Grade 4
“Juel (1991) suggested that some children may rely heavily on memorization of words, appear to be successful in beginning word reading, but struggle when such memorization becomes inefficient in the later grades.”
Juel, C. (1991). Beginning reading. In R. Barr, M. L. Kamil, P. B. Mosenthal, & P. D. Pearson (Eds.), Handbook of reading research (Vol. 2, pp. 759–788). New York, NY: Longman.
“The average student encounters about ten thousand new words in 5th Year.”
Share, D. L. (1995). Phonological recoding and self-teaching: Sine qua non of reading acquisition. Cognition, 55, 151-218.
“Word complexity changes dramatically beyond 3rd grade” (p.70).
Henry, M.K. (1991). Organising decoding instruction. MA: Orton Dyslexia Society.
“Without accurate decoding skills, these youngsters’ performance will deteriorate rapidly in the middle elementary grades, when greatly increased demands are made on comprehension and on the ability to recognise a large number of unfamiliar words (Chall, 1983; Mason, 1992)”.
Spear-Swerling, L., & Sternberg, R.J. (1994). The road not taken. An integrative theoretical model of reading disability. Journal of Learning Disabilities, 27(2), 91-103.
“Learning to read is not just one of the goals of schooling. It is essential if students are to succeed in any grade, in any subject. According to the National Reading Panel, only about 5% of children learn to read effortlessly. About 60% find early reading difficult, and of that number, 20-30% really struggle. By fourth grade, the seriousness of the problem for these children becomes obvious” (p.34).
Lewis, L. & Paik, S. (2001). Add it up: Using research to improve education for low-income and minority students. Washington: Poverty & Race Research Action Council. Retrieved from http://www.prrac.org/additup.pdf
“A study of 32 children from low-income homes found that the academic problems of low-achieving students increased as they moved through the school grades. This became especially evident by Year 4, when the more complex curriculum language led to a decline in reading achievement.”
Chall, J.S., Jacobs, V.A., & Baldwin, L.E. (1990). The reading crisis: Why poor children fall behind. Cambridge, MA: Harvard University Press.
“In a study of 3000 Australian students, 30% of 9 year olds still hadn’t mastered letter sounds, arguably the most basic phonic skill. A similar proportion of children entering high school continue to display confusion between names and sounds. Over 72% of children entering high school were unable to read phonetically regular 3 and 4 syllabic words. Contrast with official figures: In 2001 the Australian public was assured that ‘only’ about 19% of grade 3 (age 9) children failed to meet the national standards.”
Harrison, B. (2002, April). Do we have a literacy crisis? Reading Reform Foundation Newsletter, 48. Retrieved from http://www.rrf.org.uk/do%20we%20have%20a%20literacy%20crisis.htm
“In another longitudinal study, Lipka, Lesaux, and Siegel (2006) examined reading and reading-related abilities of children with poor word reading skills. From a sample of 1,100 children who had been followed from kindergarten through fourth grade, 22 children were identified with word-reading deficits in fourth grade. Seven of the poor readers had persistent problems across grades, eight had late-emerging deficits (after third grade), and seven had borderline deficits at other grades. Additional results indicated that those with late-emerging word-reading problems had phonological processing deficits, especially after second grade. Such deficits were evident on tests of phonological awareness, phonological decoding, and spelling. Lipka et al. suggested that these children may have been able to compensate for their phonological deficits in the early grades, but as words became more complex, they showed reading and spelling difficulties” (p.167).
Lipka, O., Lesaux, N., & Siegel, L. (2006). Retrospective analyses of the reading development of Grade 4 students with reading disabilities: Risk status and profiles over 5 years. Journal of Learning Disabilities, 39, 364–378.
When students don’t read much, their vocabulary doesn’t develop sufficiently to comprehend what they read (even if they could decode adequately)
“Stanovich (1986) uses the label Matthew Effects (after the Gospel according to St. Matthew) to describe how, in reading, the rich get richer and the poor get poorer. Children with a good understanding of how words are composed of sounds (phonemic awareness) are well placed to make sense of our alphabetic system. Their rapid development of spelling-to-sound correspondences allows the development of independent reading, high levels of practice, and the subsequent fluency which is critical for comprehension and enjoyment of reading. There is evidence (Stanovich, 1988) that vocabulary development from about Year 3 is largely a function of volume of reading. Nagy and Anderson (1984) estimate that, in school, struggling readers may read around 100,000 words per year while for keen mid-primary students the figure may be closer to 10,000,000, that is, a 100 fold difference. For out of school reading, Fielding, Wilson and Anderson (1986) suggested a similar ratio in indicating that children at the 10th percentile of reading ability in their Year 5 sample read about 50,000 words per year out of school, while those at the 90th percentile read about 4,500,000 words per year”.
Hempenstall, K. (1996). The Matthew Effects. Retrieved http://www.educationnews.org/articles/the-matthew-effects-.html%E2%80%8E
“Hirsh (2001) has written about "the reading gap" between middle-class and low-income students, a gap that widens as students move from 1st grade thru 3rd and 4th grade. Hirsh believes that this gap is really a "vocabulary deficit" and comes about because the lower-class children do not have the broad range of knowledge and language that middle-class students have.”
Hirsch, E.D. (2001). The latest dismal NAEP scores. Education Week, May 2, 2001. http://www.edweek.org/ew/ew_printstory.cfm?slug=33hirsch.h20
"When they were in primary grades, students could employ the Predict-the-Next-Word-by-Looking- at-the-Picture-and-Guessing technique currently in vogue. Even though they had not been explicitly taught to decode and thus had never reached the point of rapid, accurate, fluent decoding, they could sometimes wing it by predicting words that they were familiar with. For example, given "John had a little red _____", most children predict 'wagon'. The word 'wagon' is in their listening vocabulary. Off they go to middle school, relying on guessing words they cant decode. But during middle school, kids reach a "break point" in reading, a point at which contextual guessing is no longer effective. Three factors contribute to this phenomenon:
a) New content-area vocabulary words do not pre-exist in their listening vocabularies. They can guess 'wagon'. But they can't guess' circumnavigation' or 'chlorophyll' based on context (semantics, syntax, or schema); these words are not in their listening vocabularies.
b) When all of the words readers never learned to decode in grades one to four are added to all the textbook vocabulary words that don't pre-exist in readers' listening vocabularies, the percentage of unknown words teeters over the brink; the text now contains so many unknown words that there's no way to get the sense of the sentence.
c) Text becomes more syntactically embedded, and comprehension disintegrates. Simple English sentences can be stuffed full of prepositional phrases, dependent clauses, and compoundings. Eventually, there's so much language woven into a sentence that readers lose meaning. When syntactically embedded sentences crop up in science and social studies texts, many can't comprehend."
Greene, J.F. (1998, spring summer). Another chance: Help for older students with limited literacy. American Educator, 1-6. Retrieved from http://www.aft.org/newspubs/periodicals/ae/springsummer1998/index.cfm
“Students with reading difficulties may lag behind their average performing grade level peers in vocabulary acquisition by as much as two years by the end of 2nd grade (Biemiller, 2005). Therefore, students with low reading comprehension may benefit from interventions provided later in their educational career, specifically during upper elementary school that focus on both vocabulary and reading. The grade-level demands of text become increasingly difficult in 4th grade as indicated by the “fourth grade slump” with many 3rd grade students who were reading on grade level experience a drop in normative reading scores in 4th grade (Chall & Jacobs, 1983). More recently Compton et al. (2008) identified students with typical reading performance in 3rd grade who begin to demonstrate reading problems in 4th grade with late emerging reading disability. Since 4th grade is typically the year that students are introduced to more complex text, it is an appropriate year to consider vocabulary and reading interventions for students who have low reading comprehension.” (p.103)
Solís, M., Scammacca, N., Barth, A.E., &. Roberts, G.J. (2017). Text-based vocabulary intervention training study: Supporting fourth graders with low reading comprehension and learning disabilities. Learning Disabilities: A Contemporary Journal, 15(1), 103–115.
“Students in the bottom 25% of the reading continuum have a trajectory of progress that diverges early from their peers who have learned to read successfully”.
National Assessment of Educational Progress (1999). Report card for the nation and the states. National Center for Education Statistics. Washington, D. C.: U. S.
“Good and poor readers differed in their listening comprehension by only one month at school beginning, but by 30 months by Year 4”.
Juel, C. (1988). Learning to read & write: A longitudinal study of 54 children from first through fourth grades. Journal of Educational Psychology, 80, 437-447.
“Poor reading comprehension is not always accompanied with word recognition (i.e., decoding) difficulties. One such case is students with specific poor comprehension (SPC) whose reading difficulties are specific to the area of reading comprehension. Although the estimation of the population proportion in the previous studies has varied, studies substantiate that poor reading comprehenders, with good word recognition abilities, exist (Nation, Clarke, Marshall, & Durand, 2004; Yuill & Oakhill, 1988, 1991). Students with SPC could become more prevalent in the upper elementary grades with a well-known phenomenon of the ‘‘fourth-grade slump’’ in a literacy development (Chall & Jacobs, 2003). … their poor reading comprehension could be due to weak meaning-related spoken language abilities, poor metacognitive knowledge, and/or a lack of text structure knowledge.” (p. )
Lee, S. H., & Tsai, S. (2017). Experimental intervention research on students with specific poor comprehension: A systematic review of treatment outcomes. Reading and Writing: An Interdisciplinary Journal, 30, 917-943.
Instruction stops before some students are adept
“Snow and Moje (2010) described the widespread and misguided assumption that we should finish reading instruction by the end of third grade. They used the term “inoculation fallacy” to illustrate the notion that an early vaccination of reading instruction, especially in grades K–3, does not protect permanently against reading failure. Educators must continue to provide reading instruction beyond third grade. In sum, academic literacy goes beyond being able to read—a successful reader should be able to navigate advanced narrative and content-area text with ease and understanding” (p.162).
Marchand-Martella, N.E., Martella, R.C., Modderman, S.L., Petersen, H.M., & Pan, S. (2013). Key areas of effective adolescent literacy programs. Education and Treatment of Children, 36(1), 161-184.
“ … we find little instruction in the upper grades, yet word complexity changes dramatically beyond third grade” p. 70
Henry, M. K. (1991). Organizing decoding instruction. In The Orton Dyslexia Society (Eds.), All language and the creation of literacy (pp.57-62).
“Assigning to early primary teachers the responsibility of teaching children to read resulted in a lack of reading instruction from grade 4 on … Teachers after 3rd grade viewed the statement as permission to abandon reading instruction in all content areas, believing that it was the primary teachers' job to teach reading. This outlook greatly diminishes the reading potential of our nation's children” (p.1).
Houck, B.D., & Ross, K. (2012). Dismantling the myth of learning to read and reading to learn. Reading: The Core Skill, 7(11),
“Torgesen (2006) therefore suggested that it may be the lack of reading practice over such an extended amount of time that is the obstacle to closing the fluency gap for samples of older children with reading disabilities (e.g., 9-12 years and beyond)”.
Torgesen, J. K. (2006). Recent discoveries from research on remedial interventions for children with dyslexia. In M. Snowling & C. Hulme (Eds.), The science of reading: A handbook. Oxford: Blackwell Pulishers.
Reading interventions for struggling older students
Is it hard? You bet! Better to get it right the first time.
“The results presented in this report support the efficacy of reading interventions for struggling readers in Grades 4 to 12, though the magnitude of the effects obtained may be less than originally thought based on the results of Scammacca et al. (2007). More recent research on these interventions have included more rigorous measures of results that capture the extent to which the skills gained through the interventions generalize beyond the immediate context of the intervention. In addition, studies are providing more hours of intervention and increasingly these interventions are compared to an alternative intervention instead of a true no-intervention control group. As a result, smaller effects are observed. Despite these smaller effects, the more recently published interventions likely are more representative of the kind of intervention that struggling readers need. Reading difficulties that have perseverated past the primary school years likely do require many hours of intervention to remediate. Progress is likely to be slow but steady. Teachers are better positioned than researchers to provide longer term interventions” (p.387).
Scammacca, N., Roberts, G., Vaughn, S., & Stuebing, K. (2015). A meta-analysis of interventions for struggling readers in grades 4-12: 1980-2011. Journal of Learning Disabilities, 48(4), 369–390.
"Significant gains in key literacy skill development have been hard-earned for struggling adolescent and particularly adult learners (Alamprese, MacArthur, Price, & Knight, 2011; Calhoon, 2005; Calhoon, Sandow, & Hunter, 2010; Greenberg et al., 2011; Sabatini, Shore, Holtzman, & Scarborough, 2011; Vaughn et al., 2010, 2011, 2012). This difficulty, to find robust responses to intervention, may not be surprising in view of the atypical educational histories of older learners and the heterogeneity of their backgrounds and skill deficits. It may be fruitful to supplement such analyses of group differences with analyses of outcomes for individual learners to enable a teasing apart of learner-by-treatment effect” (p.490)
Calhoon, M.B., Scarborough, H.S., & Miller, B. (2013). Interventions for struggling adolescent and adult readers: instructional, learner, and situational differences. Reading and Writing, 26, 489–494.
“Neuroplasticity - the one thing we know about plasticity, which is the capacity to adjust and adapt, is it's greatest when the brain is immature, and it is less as the brain becomes more mature. It's never completely gone. There is plasticity in the brains of adults.So we do know that there are some functions that emerge, in terms of brain development, in critical periods. And the well described ones are in the sensory area, vision and hearing, to some extent. But there has never been demonstrated in humans a critical period for anything related to cognition or emotional development or social development.
In a sensitive period, there isn't a time when the window closes and it's too late. But what it means is that when you pass the sensitive period, it's harder for these things to develop in an adaptive way, or they may develop in a way that is not as efficient as it might be, and that you have to try to overcome later. Unlike a critical period where it's too late, missing a sensitive period means that it just gets harder as you get older, it's harder to get it right.
So the messages that come out of that basic principle of brain development is that getting things right the first time is better than trying to fix them later, trying to adapt to something that was not developed in the best way at the time that it was supposed to be developed. So the sobering message here is that if children don't have the right experiences during these sensitive periods for the development of a variety of skills, including many cognitive and language capacities, that's a burden that those kids are going to carry; the sensitive period is over, and it's going to be harder for them. Their architecture is not as well developed in their brain as it would have been if they had had the right experiences during the sensitive period. That's the sobering message. But there's also a hopeful message there, which is unlike a critical period where it's too late.
The sensitive period says: It's not too late to kind of try to remediate that later. And you can develop good, healthy, normal competencies in many areas, even if your earlier wiring was somewhat faulty. But it's harder. It costs more in energy costs to the brain. The brain has to work at adapting to earlier circuits that were not laid down the way they should have been. And from a society's point of view, it costs more in terms of more expensive programming, more specialized help”.
Shonkoff, J.P. (2007). The neuroscience of nurturing neurons. Children of the Code. Retrieved from http://www.childrenofthecode.org/interviews/shonkoff.htm
"A few studies have evaluated whether poor reading performance negatively impacts “distal” feelings and behaviors that are not specific to reading activities. In these studies, poor readers have been reported to be more likely to act out or be aggressive (e.g., Morgan, Farkas, & Wu, 2009; Trzesniewski, Moffitt, Caspi, Taylor, & Maughan, 2006), distractible and inattentive (Goldston et al., 2007; Morgan, Farkas, Tufis, & Sperling, 2008), and anxious and depressed (Arnold et al., 2005; Carroll, Maughan, Goodman, & Meltzer, 2005). Older poor readers have been reported to be more likely to consider or attempt suicide (Daniel et al., 2006)
The increasingly generalized Matthew effects are more likely to occur as children age (Stanovich, 1986) if they begin to avoid reading activities both at home and in school, thereby further constraining growth in their basic reading skills, comprehension, and, eventually, cognitive functioning (Cunningham & Stanovich, 1991; Echols, West, Stanovich, & Zehr, 1996; Griffiths & Snowling, 2002; Guthrie, Schafer, & Huang, 2001; Senechal, LeFevre, Hudson, & Lawson, 1996). The children’s resulting inability to meet their classroom’s academic demands can lead to increasingly frequent feelings of frustration, agitation, withdrawal, and social isolation (e.g., Fleming, Harachi, Cortes, Abbott, & Catalano, 2004; Kellam, Mayer, Rebok, & Hawkins, 1998; Lane, Beebe-Frankenberger, Lambros, & Pierson, 2001; Wehby, Falk, Barton-Arwood, Lane, & Cooley, 2003). These feelings and behaviors may in turn further interfere with children’s learning (p.361).
We investigated whether and to what extent being a poor reader increases a child’s likelihood of reporting feeling angry, distractible, sad, lonely, anxious, and unpopular. Poor reading performance has repeatedly been hypothesized to contribute to children’s socioemotional maladjustment (e.g., Stanovich, 1986). Although there is some evidence indicating that poor reading performance results in “proximal” negative Matthew effects (e.g., poorer attitude toward reading, less persistence during reading tasks, less independent reading practice), less is known about the “distal” or more generalized effects on socioemotional maladjustment (e.g., frequently feeling angry, sad, or unpopular). To better estimate these predicted relationships, we statistically controlled for a range of child-, family-, school-, and community-level confounds including the autoregressor. Multilevel logistic regression analyses indicated that poor readers are at substantially greater risk of socioemotional maladjustment. This was the case across multiple self-report measures as well as after extensive statistical control of possible confounding factors." (p.373)
Morgan, P.L., Farkas, G., & Qiong, W. (2012). Do poor readers feel angry, sad, and unpopular? Scientific Studies of Reading, 16(4), 360-381.
“A study by Schiffman provides support for monitoring programs for reading disabilities in the first and second grades. In a large scale study of reading disabilities (n = 10,000),
82% of those diagnosed in Grades 1 or 2 were brought up to grade level.
46% in Grade 3 were brought up to grade level.
42% in Grade 4 were brought up to grade level.
10-15% in Grades 5-7 were brought up to grade level.”
Berninger, V.W, Thalberg, S.P., DeBruyn, I., & Smith, R. (1987). Preventing reading disabilities by assessing and remediating phonemic skills. School Psychology Review, 16, 554-565.
“If children have not grasped the basics of reading and writing, listening and speaking by Year Three, they will probably be disadvantaged for the rest of their lives.”
Australian Government House of Representatives Enquiry. (1993).The Literacy Challenge. Canberra: Australian Printing Office
“Unless these children receive the appropriate instruction, over 70 percent of the children entering first grade who are at risk for reading failure will continue to have reading problems into adulthood”.
Lyon, G.R. (2001). Measuring success: Using assessments and accountability to raise student achievement. Subcommittee on Education Reform Committee on Education and the Workforce U.S. House of Representatives Washington, D.C. [On Line]. Available: http://www.nrrf.org/lyon_statement3-01.htm
"These findings extend into adolescence data previously reported on the persistence of reading disability [18] that is, that children who were initially poor readers in the early school years remain poor readers relative to other children in the sample. This finding suggests that shortly after school entry, the reading achievement of children changes very little relative to their peers. These special services, however, consisted of eclectic approaches to teaching reading that were provided in an inconsistent fashion and for relatively brief periods."
Shaywitz, S.E., Fletcher, J.M., Holahan, J.M., Shneider, A.E., Marchione, K.E., Stuebing, K.K., Francis, D.J., Pugh, K.R., & Shaywitz, B.A. (1999). Persistence of dyslexia: The Connecticut longitudinal study at adolescence. Pediatrics, 104, 1351-1339.
"Juel (1988) found that the probability of a poor reader in first grade remaining a poor reader at the end of fourth grade was .88. Satz, Fletcher, Clark, and Morris (1981) found that 93.9% of severely poor readers in second grade continued to be poor readers in fifth grade. Scarborough (1998b) found similar results for students from second grade to eighth grade."
Juel, C. (1988). Learning to read & write: A longitudinal study of 54 children from first through fourth grades. Journal of Educational Psychology, 80, 437-447.
Satz, P., Fletcher, J. M., Clark, W., & Morris, R. (1981). Lag, deficit, rate and delay constructs in specific learning disabilities: A re-examination. In A. Ansara, N. Geschwind, A. Galaburda, M. Albert, & N. Gartrell (Eds.), Sex differences in dyslexia (pp. 129-150). Towson, MD: The Orton Dyslexia Society.
Scarborough, H. S. (1998b). Predicting the future achievement of second graders with reading disabilities: Contributions of phonemic awareness, verbal memory, rapid naming, and IQ. Annals of Dyslexia, 48, 114-136.
"...a longitudinal study of students with poor word identification skills in the third grade (Felton & Wood, 1992) indicated that most of these students failed to significantly improve their skills by the end of eighth grade."
Felton, R. H., & Pepper, P. P. (1995). Early identification and intervention of phonological deficit in kindergarten and early elementary children at risk for reading disability. School Psychology Review, 24, 405-414.
"Students in the bottom 25% of the reading continuum have a trajectory of progress that diverges early from their peers who have learned to read successfully."
National Assessment of Educational Progress (1999). Report card for the nation and the states. National Center for Education Statistics. Washington, D. C.: U. S.
"National longitudinal studies show that approximately 75% of those with reading problems in third grade still experience reading difficulties in the ninth grade (Francis, Shaywitz, Stuebing, Shaywitz, & Fletcher 1996; Shaywitz, Holahan, & Shaywitz, 1992). Students who experience reading difficulties in the early grades often suffer what has been called the "Matthew Effects" (Stanovich, 1986), a gap between good and poor readers that widens through the grades. Mikulecky (1990), for example, found that a group of secondary students two or more years behind their peers in reading ability were differentially affected by their tendency to avoid reading. These students read very little during or outside of school. Over the two-year period of the study, their reading comprehension performance actually declined."
Mikulecky, L. J. (1990). Stopping summer learning loss among at-risk youth. Journal of Reading, 33(7), 516-521.
“Students who are behind do not learn more in the same amount of time as students who are ahead. Catch-up growth is driven by proportional increases in direct instructional time. Catch-up growth is so difficult to achieve that it can be the product only of quality instruction in great quantity.” (p. 62)
Fielding, L., Kerr, N., Rosier, P. (2007). Annual growth for all students, Catch-up growth for those who are behind. Kennewick, WA: The New Foundation Press, Inc.
“We have learned that for 90% to 95% of poor readers, prevention and early intervention programs that combine instruction in phoneme awareness, phonics, fluency development, and reading comprehension strategies, provided by well trained teachers, can increase reading skills to average reading levels. However, we have also learned that if we delay intervention until nine-years-of-age, (the time that most children with reading difficulties receive services), approximately 75% of the children will continue to have difficulties learning to read throughout high school. To be clear, while older children and adults can be taught to read, the time and expense of doing so is enormous”.
Lyon, G.R. (1998).Overview of reading and literacy initiatives. Committee on Labor and Human Resources. Retrieved from http://www.readbygrade3.com/readbygrade3co/lyon.htm
Rebecca Felton (1993) argued that: (i) at-risk children should be identified in their first school year. (ii) phonemic awareness training should be available for those students, and taught using a direct instruction approach. (iii) structured code emphasis teaching should follow, using controlled vocabulary. Explicitly taught strategies such as blending (rather than guessing strategies) should be promoted. (iv) a significant portion of the school day should be assigned to direct instructional activities. (v) teaching the onset-rime distinction will hasten students progress from letter-by-letter decoding to skilled reading. (vi) reading, writing and spelling instruction should be integrated, with correct spelling emphasised. (vii) it should be recognised that at-risk students may need three years of direct instruction in basic reading skills. (viii) it must be recognised that teaching to mastery is insufficient, and provision should be made for adequate opportunities for the practice necessary to achieve automaticity. Felton's conclusions represent a confluence of the research on phonemic awareness, and that on effective teaching.
Ref: Felton, R.H. (1993). Effects of instruction on the decoding skills of children with phonological processing problems. Journal of Learning Disabilities, 26, 583-589.
“The lowest 10% of readers will make no discernible progress in reading between Year Four and Year Ten”.
Hill, P. (1995). School effectiveness and improvement: Present realities and future possibilities. Deans Research Seminar Series. Inaugural Professorial Lecture 24 May, 1995, Faculty of Education, The University of Melbourne. Retrieved from http://www.edfac.unimelb.edu.au/Seminars/dean_lec/list.html
“When students with borderline intellectual functioning begin school, academic motivation is rarely a problem (Hihi & Harachiewicz, 2000). However, as they get older, a consistent pattern emerges. Strong effort in the early grades is met with academic frustration and possible failure. Despite the students' efforts, this cycle of academic frustration and failure is repeated. After some years, the cumulative effect of frustration and failure is that students simply stop trying (Guay & Vallerand, 1996-1997). In the classroom, such students are referred to as unmotivated and, sometimes, as lazy. Academic motivation may have some temperamental and early environmental factors that place a child at risk for motivation deficits (Levine, 2003). However, academic motivation and effort are often extinguished by repeated failure and frustration (Levine, 2003)”.
Shaw, S. R. (2008). An educational programming framework for a subset of students with diverse learning needs: Borderline intellectual functioning. Intervention in School and Clinic, 43, 291-299.
“Of students identified as reading disabled in Year Three, 75% will remain so at Year Nine”.
Francis, D.J., Shaywitz, S.E., Stuebing, K.K., Shaywitz, B.A., & Fletcher, J.M. (1996). Developmental lag versus deficit models of reading disability: A longitudinal, individual growth curves analysis. Journal of Educational Psychology, 88, 3-17.
Shaywitz, S. E., Escobar, M. D., Shaywitz, B. A., Fletcher, J.M., & Makuch, R. (1992). Distribution and temporal stability of dyslexia in an epidemiological sample of 414 children followed longitudinally. New England Journal of Medicine, 326, 145-150.
“The pathway to unemployment is very clear a long way back. ... They go downhill from an early age, and we do not seem to have many mechanisms to reverse that.”
McDonald, P. (1994). Failure patterns set at an early age. The Age, 30/9, p.6.
“If pressed to provide an estimate of the likely progress of older low-progress readers (Years 5 to 8) who are at least two years behind in terms of reading skill, and who are offered either no or only limited non-intensive remedial support, then we would conclude that progress of about half normal rate is probably typical”.
Wheldall, K., & Beaman, R. (2000). An evaluation of MULTILIT: ‘Making Up Lost Time In Literacy’. Canberra: Department of Education, Training and Youth Affairs.
“But that is the least efficient use of their time; a study showed that it takes 30 minutes a day to bring a struggling kindergartner up to par in reading, whereas a third-grader would need an hour and a half”.
Nguyen, T. (1998, October 4). Education: Lack of standardized approach stymies state's goal of boosting the bottom rung of literacy. Los Angeles Times.
“Children who do well in reading from the beginning rarely stumble later on. Those who have difficulty in the primary grades tend to remain behind their classmates as the years go by--even though they receive remediation. This fact, reconfirmed again and again, is a painful testimony to the importance of addressing reading difficulties as early as possible in a child's life. As important as it is to hold out hope for every struggling reader in our middle and high schools, there is no substitute for an all-out effort to ensure that all of our children start out right, so that they never have to experience the consequences of failure and frustration that are so prevalent in our schools.”
Burns, M.S., Griffin, P., & Snow, C.E. (Eds.) (1999). Starting out right: A guide to promoting children’s reading success. Committee on the Prevention of Reading Difficulties in Young Children; National Research Council. Retrieved from: http://www.nap.edu/catalog.php?record_id=6014
How far behind might a student be?
“The struggling readers in this study were multiple grade levels (3–7 years) behind their typically developing peers in reading ability. Results of both group and individual analyses indicate these older struggling readers can be remediated and for some, gains of two, three, four, or more years can be accomplished with only 1 year of instruction.” (p.588)
Calhoon, M. B., & Prescher, Y. (2013). Individual and group sensitivity to remedial reading program design: Examining reading gains across three middle school reading projects. Reading and Writing: An Interdisciplinary Journal, 26, 565-592.
“Students from the 10th and 90th percentiles differ by grade equivalents equal to their grade (i.e., 6 grade difference at the end of 6th grade)”. (Biemiller, personal communication, August 1, 2002) Professor Andrew Biemiller, Institute of Child Study, University of Toronto.
What does it take for older students? Intensity is one key element.
“Fourth grade is a critical time in the development of reading proficiency. Chall and Jacobs (1983) first identified the “fourth grade slump”, a phenomenon in which many 3rd grade students who were previously reading on grade level experienced a drop in normative reading scores in 4th grade. They suggested that these students were not going “backwards” in reading, but were instead failing to meet grade-level expectations as text complexity increased and the reading task became more difficult. Compton, Fuchs, Fuchs, Elleman, and Gilbert (2008) described students whose reading performance was relatively typical prior to 4th grade but then demonstrated significant reading problems in grade 4 as students with late-emerging reading disability. Thus, the upper elementary grades (4th and 5th) may be particularly good targets for reading interventions because these grades are the last in which formal reading instruction traditionally occurs and also are grades in which reading to learn and understand complex texts is initially most apparent. … From this study, we can form hypotheses about more effective ways to remediate upper elementary students with reading difficulties. It may be necessary to provide even more intensive intervention for some students (e.g., longer time, smaller groups, intervention even more specifically focused to meet students’ needs). For example, it is possible to consider ways to integrate text reading throughout the school day and thus relying less on the reading intervention time to compensate for the vocabulary, background knowledge, and text-based challenges these students demonstrate. We also think it may be important to consider ways to provide individualized treatments to better align with the individual learning needs of students. This might be accomplished with even smaller groups (e.g., one-on-one, pairs), but these approaches may be less feasible for schools to implement. We are also examining ways to increase impact by considering mechanisms for improving attention and self-regulation within the reading intervention treatment.” (p.23, 40)
Vaughn, S., Solis, M., Miciak, J., Taylor, W. P. & Fletcher, J. M. (2016). Effects from a randomized control trial comparing researcher and school-implemented treatments with fourth graders with significant reading difficulties. Journal of Research on Educational Effectiveness, 9(1), 23–44.
“Together, these findings provide evidence that dyslexic adults are not, as may have been assumed, unable to profit from remedial practice," wrote the researchers. "In fact, the same strategies that are effective in teaching children phonological awareness skills are helpful in adults. Further, they are accompanied by neural changes known to underlie reading remediation of developmental dyslexia in childhood combined with those previously observed during the rehabilitation of adults with acquired dyslexia [due to brain damage].”
Eden, G. F., Jones, K.M., Cappell, K., Gareau, L., Wood, F.B., Zeffiro, T.A., Dietz, N.A.E., Agnew, J.A. & Flowers, D.L. (2004). Neurophysiological recovery and compensation after remediation in adult developmental dyslexia, Neuron, 44, 411–422.
“Although it is unlikely that these students will make accelerated progress without intensive interventions, there is evidence that secondary students may experience improved reading outcomes when provided explicit reading intervention with adequate time and intensity for reading instruction (Archer, Gleason, & Vachon, 2003; Torgesen et al., 2001)” (p.932).
Vaughn, S., Wanzek, J., Wexler, J., Barth, A.E., Cirino, P.T., Fletcher, J.M., Romain, M.A., Denton, C.A., Roberts, G., & Francis, D.J. (2010). The relative effects of group size on reading progress of older students with reading difficulties. Reading and Writing: An Interdisciplinary Journal, 23(8), 931-956.
“Impressive and unexpected were the large gains made in comprehension by students in the Additive modality, insofar as they receive relatively few hours of explicit comprehension instruction (12–13 h.) in comparison to the other modalities (24–39 h). The theoretical underpinnings of the Additive modality are that reading is hierarchical and that automaticity of lower level skills (decoding, spelling) allows cognitive efforts to then be allocated to attaining higher level skills (fluency, comprehension; LaBerge & Samuels, 1974; Reynolds, 2000, Samuels & Kamil, 1984). Clearly, the changes brought about by other aspects of instruction (front loading of phonics instruction, followed by the addition of spelling instruction, followed by the addition of fluency instruction) laid the groundwork for comprehension gains, without having to supply a great deal of explicit comprehension instruction. These older struggling readers were able to master decoding, spelling, and fluency, before comprehension was even introduced into instruction, enabling them to more fully understand strategy instruction and achieve comprehension gains with very little explicit comprehension strategy instruction. These results strongly suggest that it may not be how many hours of instruction for each component that is important, but instead when those hours are incorporated into organization of instruction, that matters most” (p.587).
Calhoon, M. B., & Prescher, Y. (2013). Individual and group sensitivity to remedial reading program design: Examining reading gains across three middle school reading projects. Reading and Writing: An Interdisciplinary Journal, 26, 565-592.
“This paper has reported a systematic review to investigate the effectiveness of interventions designed to support the reading skills of secondary school students. It has focused only on studies which have used RCTs. The paucity of research is striking; only eight studies met our inclusion criteria. Five of the studies were conducted in the US, two in the UK and one in Australia. This is consistent with the findings of a review conducted by Slavin, Cheung, Groff, and Lake (2008) which argued that more large scale methodologically rigorous studies are needed in this area. With regards multiple component training, three of the four studies reviewed were evaluations of individualised CAI and in each case the findings did not support the use of this approach. Slavin, Lake, Davis, and Madden (2009) in reviewing approaches for struggling readers ingrades K-5, concluded that CIA generally had few effects on reading. Furthermore, Khan and Gorard (2012) reported that previous studies have failed to demonstrate the effectiveness of computer based instruction as a means of improving reading skills.Taken together the evidence suggests that CAI should not be relied on to produce gains in reading ability in secondary school aged students and that in some circumstances using these programmes may have a negative impact on student’s progress (Gorard &Taylor, 2004). An avenue for future intervention design could be to combine computer administered tasks with face-to-face instruction. It is of note that these studies, which all included some focus on word recognition and decoding skills, were delivered to individual students via computer programmes. There is therefore no evidence from RCTs to show the possible effectiveness of tutor led one-to-one instruction in this essential component of reading at secondary school level. ... There is therefore a significant gap in the evidence base from RCT’s concerning the efficacy of language comprehension intervention. Clarke, Snowling, Truelove, and Hulme (2010) demonstrated using an RCT the effectiveness of an oral language intervention (comprising strategy use, vocabulary, figurative language and spoken narrative) in improving the reading comprehension skills of primary school students. To date such an approach has not been evaluated using an RCT in secondary schools.” (p.124-125)
Paul, S-A.S., & Clarke, P.J. (2016). A systematic review of reading interventions for secondary school students. International Journal of Educational Research, 79, 116–127.
“Questions have sometimes been raised about the extent to which reading skills of struggling adolescents can be remediated and whether the money spent on such interventions is justified in light of the degree of benefit attained (Vaughn et al., 2010, 2011, 2012). Adolescents who have already gone through years of reading instruction and still lag behind their same age peers are a very heterogeneous group in their reading abilities. Through the use of both group and individual differences analysis we were able to gain a more complete and finely-tuned picture of how these struggling readers respond to treatment. The struggling readers in this study were multiple grade levels (3–7 years) behind their typically developing peers in reading ability. Results of both group and individual analyses indicate these older struggling readers can be remediated and for some, gains of two, three, four, or more years can be accomplished with only 1 year of instruction. While two to three years of gain for students who are four to six years behind by no means closes the achievement gap, these findings are encouraging in providing information on which modality of instruction closes the achievement gap best. ... Most compelling from the current analyses are results directly investigating the differences between three modalities (Alternating, Integrated, Additive) of instruction. Outcomes showed clearly that modality of instruction can matter considerably for these older struggling readers. The differences in gains clearly demonstrate that the Additive modality, with its sequential addition of each component (isolated phonological decoding instruction, followed by addition of spelling instruction, followed by addition of fluency instruction, and finally the addition of comprehension instruction [see Table 1]) is potentially the best modality for remediating reading skills (decoding, spelling, fluency, comprehension) in older struggling readers, of the three approaches that were compared in this research. These students show that they are highly sensitivity to the scheduling of the components and the amounts of instructional time per component; this is an important finding for the development and refinement of reading programs for struggling adolescent readers. While more research still needs to be conducted in this area, this study lends credence to the different requirements this unique population of students may need in order to close the achievement gap in acquiring adequate reading skills” (p.588-9).
Calhoon, M. B., & Prescher, Y. (2013). Individual and group sensitivity to remedial reading program design: Examining reading gains across three middle school reading projects. Reading and Writing: An Interdisciplinary Journal, 26, 565-592.
‘‘Direct instructional time is proportional to their [children’s] deficiency. The greater the need, the more time they get.’’ Further, they caution that ‘‘catch up growth’’ requires more time and better quality instruction. Ikeda and colleagues cautioned that in most schools within the Iowa Heartland district, ‘‘interventions were not sufficiently rigorous to impact reading performance’’ (p.20).
Al-Otaiba, S., Calhoon, M. B., & Wanzek. J. (2010). Response to intervention: Treatment validity and implementation challenges in the primary grades and beyond to middle school. In Thomas E. Scruggs, Margo A. Mastropieri (Eds.). Advances in learning and behavioral disabilities, Volume 23; Learning and Literacy. Emerald Publishing.
“From grades 2 to 6 there is no evidence of a developmental window beyond which phonological deficits cannot be effectively remediated with intensive phonological training”.
Lovett, M.W., & Steinbach, K.A. (1997). The effectiveness of remedial programs for reading disabled children of different ages: Does the benefit decrease for older children? Learning Disability Quarterly, 20, 189-209.
“Students who are behind do not learn more in the same amount of time as students who are ahead. Catch-up growth is driven by proportional increases in direct instructional time. Catch-up growth is so difficult to achieve that it can be the product only of quality instruction in great quantity.” (p. 62)
Fielding, L., Kerr, N., Rosier, P. (2007). Annual growth for all students, Catch-up growth for those who are behind. Kennewick, WA: The New Foundation Press, Inc.
”We would also wish to emphasise that there is perhaps unnecessary pessimism concerning the prognosis for older low-progress readers. We support the desirability of a preventative early intervention approach to reading difficulties but some low-progress readers will still slip through the net and early intervention is not always effective. Similarly, the often heard view that remedial instruction for students beyond Year 2 is ineffective may have been true, but this is a criticism of the ineffectiveness of past programs, not a necessary truth. We can rehabilitate older low-progress readers, as we have shown, with effective programs based on contemporary, empirically validated best practice, if we have the will and the resources to do so.”
Wheldall, K., & Beaman, R. (2000). An evaluation of MULTILIT: ‘Making Up Lost Time In Literacy’. Canberra: Department of Education, Training and Youth Affairs. Retrieved from http://www.dest.gov.au/schools/literacy&numeracy/publications/multilit/summary.htm
“This study reports findings on the relative effects from a year long secondary intervention contrasting large-group, small-group, and school-provided interventions emphasizing word study, vocabulary development, fluency, and comprehension with seventh- and eighth-graders with reading difficulties. Findings indicate that few statistically significant results or clinically significant gains were associated with group size or intervention. Findings also indicate that a significant acceleration of reading outcomes for seventh- and eighth-graders from high-poverty schools is unlikely to result from a 50 min daily class. Instead, the findings indicate, achieving this outcome will require more comprehensive models including more extensive intervention (e.g., more time, even smaller groups), interventions that are longer in duration (multiple years), and interventions that vary in emphasis based on specific students’ needs (e.g., increased focus on comprehension or word study)” (p.931).
Vaughn, S., Wanzek, J., Wexler, J., Barth, A.E., Cirino, P.T., Fletcher, J.M., Romain, M.A., Denton, C.A., Roberts, G., & Francis, D.J. (2010). The relative effects of group size on reading progress of older students with reading difficulties. Reading and Writing: An Interdisciplinary Journal, 23(8), 931-956.
“Intervention components such as tutoring (ES = 0.36), feedback and progress monitoring (ES = 0.32), and cooperative learning (ES = 0.22) have averageES that are educationally important, statistically significant, and robust.This systematic review has examined interventions that aim to improve the educational achievement for low SES students in elementary and middle school. … The meta-analysis included 101 studies with a treatment-control design, 76% of which were RCTs, and where outcomes were measured by standardized tests in reading and mathematics. The analysis showed that there are interventions that improve the educational achievement of low SES students, and also that there is substantial variation in effect sizes.” (p.243, 272)
Dietrichson, J., Bøg, M., Filges, T., & Jørgensen, A-M.K. (2017). Academic interventions for elementary and middle school students with low socioeconomic status: A systematic review and meta-analysis. Review of Educational Research, 87(2), 243–282.
“Several recent syntheses provide useful summaries of intervention studies and effective practices for secondary readers with reading difficulties (Edmonds et al., 2009; Scammacca et al., 2007). In a meta-analysis of reading intervention studies for older students (grades 6–12) with reading difficulties, Edmonds et al. (2009) reported a mean weighted average effect size for reading comprehension of 0.89 favoring treatment over comparison students. Interventions that focused primarily on decoding were associated with moderate effect size gains in reading comprehension (ES = 0.49). Scammacca et al. (2007) reported a mean effect size of 0.95 for reading comprehension. Moderator analyses revealed that researcher-developed instruments were associated with larger effect sizes than standardized, norm-referenced measures (ES = 0.42). In addition, word study interventions were associated with moderate effects, with researcher-implemented interventions associated with higher effects than teacher-implemented interventions. Higher overall outcomes were associated with students in the middle grades rather than students in high school” (p.932).
Vaughn, S., Wanzek, J., Wexler, J., Barth, A.E., Cirino, P.T., Fletcher, J.M., Romain, M.A., Denton, C.A., Roberts, G., & Francis, D.J. (2010). The relative effects of group size on reading progress of older students with reading difficulties. Reading and Writing: An Interdisciplinary Journal, 23(8), 931-956.
“Increasingly intense interventions. Perhaps the most important distinction among tiers is the intensity of instruction associated with each tier. Instructional intensity, while a term that is commonly understood by educators, merits some discussion in the RTI context. Often educators view increased intensity as something that can be accomplished “primarily by increasing instructional time or reducing size of the instructional group, or doing both” (Torgesen, 2005, p. 3). Mellard (2009) suggests that schools evaluate 10 distinct variables that may be adjusted to increase instructional intensity. These variables include three dosage-related elements (minutes of instruction, frequency, and duration), as well as instructional group size, immediacy of corrective feedback, the mastery requirements of the content, the number of response opportunities, the number of transitions among contents or classes, the specificity and focus of curricular goals, and instructor specialty and skills.
Dosage: Instructional Minutes, Frequency, and Duration Dosage of intervention is a very fundamental construct of instruction that directly relates to opportunities for learning. Students need sufficient learning opportunities to acquire and practice curricular knowledge, skills, and abilities. To increase instructional intensity by varying dosage, teachers may change three key time-related variables: (1) the instructional minutes given to each student (i.e., minutes per lesson); (2) the frequency of the instruction (i.e., tutoring sessions per week); and/or (3) duration of the instruction (i.e., number of weeks). In an RTI framework, we expect to see increases in some or all of these time elements as an indication of increasingintensity from a lower tier to a higher, more intense tier” (p. 219).
Mellard, D., McKnight, M., & Jordan, J. (2010). RTI tier structures and instructional intensity. Learning Disabilities Research & Practice,25(4), 217–225.
“Instructional intensity can be increased by adding additional time to total classroom instruction (e.g., a "double dose" of reading instruction within the regular classroom routine) or can be provided through individualized or small-group instruction (Foorman & Torgesen, 2001; No Child Left Behind Act of 2001; Wasik & Slavin, 1993). The use of instructional intensity through increased academic engaged time is solidly substantiated in the research (e.g., Foorman & Torgesen, 2001; Torgesen et al., 1999). In fact, recent findings from a meta-analysis on grouping practices and reading outcomes for children with reading difficulties reported positive findings for individualized and small-group instruction (Elbaum, Vaughn, Hughes, & Moody, 1999). Torgesen et al. (2001) also demonstrated the powerful impact of individualized instruction when compared to large-group instruction on word reading outcomes” (p.422).
Jitendra, A.K., Edwards, L.L., Starosta, K., Sacks, G., Jacobson, L.A., & Choutka, C.M. (2004). Early reading instruction for children with reading difficulties: Meeting the needs of diverse learners. Journal of Learning Disabilities 37(5), 421-39.
“Recently, a practice guide (Kamil et al., 2008) provided a summary of effective practices for adolescent literacy broadly, not specifically for students with LD or reading difficulties. They identified three practices that had strong research evidence: (a) providing explicit vocabulary instruction, (b) providing direct and explicit comprehension instruction, and (c) providing intensive and individualized interventions by trained specialists. The recommendation for providing intensive interventions was derived from approximately 12 small-scale studies, many of which were not focused specifically on students with LD. ... Remediation of reading difficulties in older students may require considerable intensity and differentiation of instruction. A significant problem is that intensive, small-group instruction provided by highly skilled teachers is an expensive and infrequently applied instructional practice within most educational settings (Vaughn, Levy, Coleman, & Bos, 2002; Vaughn, Moody, & Schumm, 1998). Therefore, it is perhaps not surprising that the few available studies of students who receive special education services show fair levels of growth and little evidence that interventions through special education actually close the achievement gap (Bentum & Aaron, 2003; Foorman et al., 1997; Hanushek, Kain, & Rivkin, 1998; Torgesen et al., 2001) (p.74)”.
“Our results suggest that a supplemental intervention for students with LD may increase the intensity of intervention for students, allowing them to make additional gains in reading. However, the gains seen were in automaticity of basic reading skills (i.e., word reading fluency); the small effect in comprehension suggests that even more intensity is needed for students to accelerate their reading achievement gains. One possible way to further increase the intensity may be to provide supplemental intervention in smaller groups. The students in this study received the supplemental intervention in class sizes of 10 to 15 students. Using small-group instruction for the supplemental intervention may assist students with LD in making additional gains” (p. 85).
Wanzek, J., Vaughn, S., Roberts, G., & Fletcher, J.M. (2011). Efficacy of a reading intervention for middle school students with learning disabilities. Exceptional Children, 78(1), 73-87.
“Once children fall behind in the growth of critical word reading skills, it may require very intensive interventions to bring them back up to adequate levels of reading accuracy (Allington & McGill-Franzen, 1994; Vaughn & Schumm, 1996), and reading fluency may be even more difficult to restore because of the large amount of reading practice that is lost by children each month and year that they remain poor readers (Rashotte,Torgesen, &Wagner, 1997). ... Greater intensity and duration of instruction is required because the increased explicitness of instruction for children who are at risk for reading failure requires that more things be taught directly by the teacher. Intensity of instruction is increased primarily by reducing teacher/student ratios. Unless beginning reading instruction for children with phonological weaknesses is more intensive (or lasts significantly longer) than normal instruction, these children will necessarily lag significantly behind their peers in reading growth. An effective preventive program may involve several levels of instructional intensity ranging from small-group to one-on-one instruction, depending upon the severity of the risk factors for each child”.
Torgesen, J.K. (1998, Spring/Summer). Catch them before they fall: Identification and assessment to prevent reading failure in young children. American Educator. http://www.ldonline.org/ld_indepth/reading/torgeson_catchthem.html
“Many children are already behind in vocabulary and print knowledge when they enter school. To “close the gap” by third grade, poor children must learn vocabulary words at a faster rate than their middle class peers in grades K-3 The most direct way to increase learning rate is by increasing the number of positive, or successful, instructional interactions per school day. There are a variety of ways to increase the number of positive instructional interactions per school day”.
Torgesen, J. (2003). Using science, energy, patience, consistency, and leadership to reduce the number of children left behind in reading. Barksdale Reading Institute, Florida. Retrieved 3/5/2004 from http://www.fcrr.org/staffpresentations/Joe/NA/mississippi_03.ppt
“Elbaum, Vaughn, Hughes, and Moody’s (2000) meta-analysis provides support for the view that benefits of small group instruction are equal to one-to-one tutoring in elementary reading instruction. Several studies showed that when highly qualified teachers rigorously implement a well-designed intervention, the academic benefit to students is the same, whether students are taught individually or in a group of 2 to 6 students. Conversely, a recent synthesis of evidence-based studies with struggling readers indicates that, although small group tutorials can be effective, they are not as effective as one-to-one instruction by teachers or paraprofessionals (Slavin, Lake, Davis, & Madden, 2009). Further, Ritter, Barnett, Denny, and Albin’s (2009) meta-analysis found elementary and middle school volunteer tutoring programs compared to no tutoring had significant positive effects on reading letters and words, oral reading fluency, writing, and reading in general” (p.220).
Mellard, D., McKnight, M., & Jordan, J. (2010). RTI tier structures and instructional intensity. Learning Disabilities Research & Practice,25(4), 217–225.
“11. The provision for secondary age children with persistent reading difficulties calls for greater attention. Despite differences in school organisation, the same principles embodied in ‘Simple View of Reading’ and the three Waves of Provision for children with literacy difficulties should apply in secondary schools, as they do in primary schools. However, it is well known that the nature of the problems for secondary aged children who have experienced repeated failure with reading often include negative attitudes and disengagement that are much more entrenched than in primary schools. Additional support for those children starting secondary school without secure reading skills is essential if they are to make progress and not fall further behind their peers” (p.13).
Rose, J. (2009). Identifying and teaching children and young people with dyslexia and literacy difficulties.An independent report from Sir Jim Rose to the Secretary of State for Children, Schools and Families.
Retrieved from www.dera.ioe.ac.uk/14790/1/00659-2009DOM-EN.pdf
“Remediation of reading difficulties in older students may require considerable intensity and differentiation of instruction. A significant problem is that intensive, small-group instruction provided by highly skilled teachers is an expensive and infrequently applied instructional practice within most educational settings (Vaughn, Levy, Coleman, & Bos, 2002; Vaughn, Moody, & Schumm, 1998). Therefore, it is perhaps not surprising that the few available studies of students who receive special education services show flat levels of growth and little evidence that interventions through special education actually close the achievement gap (Bentum & Aaron, 2003; Foorman et al., 1997; Hanushek, Kain, & Rivkin, 1998; Torgesen et al., 2001) (p.74)”.
Wanzek, J., Vaughn, S., Roberts, G., & Jack, M. F. (2011). Efficacy of a reading intervention for middle school students with learning disabilities. Exceptional Children, 78(1), 73-87.
“For older students with LD who continue to struggle in reading, the challenge is providing instruction that is powerful enough to narrow or close the gap with grade-level standards in reading. This means that students who previously have struggled to even keep pace with expectations for average yearly growth in reading must now make considerably more than expected yearly growth each year if they are to catch up. While adolescence is not too late to intervene, intervention must be commensurate with the amount and breadth of improvement students must make to eventually participate in grade-level reading tasks. Because most intervention studies provide only a limited amount of instruction over a relatively short period of time, we do not yet have a clear understanding of all the conditions that must be in place to close the gap for older students with serious reading disabilities. However, it does seem likely that the intensity and amounts of instruction necessary to close the gap for many older students with LD will be considerably beyond what is currently being provided in most middle and high schools”.
Roberts, G., Torgesen, J.K., Boardman, A., & Scammacca, N. (2008) Evidence-based strategies for reading instruction of older students with learning disabilities. Learning Disabilities Research & Practice 23(2), 63–69.
“Many older struggling readers are victims of poor early reading instruction. They were not taught or were insufficiently taught the basic skills necessary for fluent reading and deep processing of text. Some of these students are able to catch up in critical reading skills if provided with additional, sustained instruction in small, focused instructional groups (Torgesen, 2005). Of course, the older and further behind the student, the more ground he or she will have to cover, impacting the intensity and duration of necessary intervention. However, for many students in this situation, reading at grade level with good comprehension is a reasonable goal”.
Roberts, G., Torgesen, J.K., Boardman, A., & Scammacca, N. (2008). Evidence-based strategies for reading instruction of older students with learning disabilities. Learning Disabilities Research & Practice, 23, 63.
“Effective programs make highly effective use of instructional time and provide multiple reading opportunities”.
Schacter J. (1999). Reading programs that work: A review of programs for Pre-Kindergarten to 4th Grade. Retrieved from https://www.google.com.au/#q=Reading+programs+that+work%3A+A+review+of+programs+for+Pre-Kindergarten+to+4th+Grade.
“Best results are generally achieved by providing instruction every day, rather than lengthy periods with days between sessions”.
Horowitz, J. (2000). Teaching older nonreaders how to read The Reading Teacher, 54, 24-26.
“The National Literacy Strategy (1998) involves a daily "literacy hour” to attempt to address the problem of reading failure”.
Department for Education and Employment. (1998). The National Literacy Strategy: Framework for Teaching. London: Crown.
“If reading assistance fails to exert a significant impact on the reading performance of low-achieving older readers one reason is that the instruction provided is not sufficiently intense”.
National Institute of Child Health and Human Development (2000). National Reading Panel: Teaching children to read. Retrieved from http://www.nationalreadingpanel.org.
“We found that extended practice was particularly important toward increasing the magnitude of treatment outcomes”.
Swanson, H.L. (2001). Research on interventions for adolescents with learning disabilities: A meta-analysis of outcomes related to higher-order processing. The Elementary School Journal, 101, 331-348.
What about group size? Must intervention be 1:1?
“Although instructional group size is often noted as an important intervention variable for early elementary students (Elbaum et al., 2000; Vaughn et al., 2003; Wanzek & Vaughn, 2007), this synthesis did not find support for instructional group size as a significant moderator of effects for students in Grades 4 through 12, despite the variety of group sizes noted in the corpus of studies. This finding aligns with an experimental study that directly compared large- (10–15 students) and small-group (3–5 students) extensive intervention at the sixth-grade level, reporting no differences in student outcomes based on group size (Vaughn, Wanzek, et al., 2010). There are several possible interpretations of this finding. First, perhaps group size needs to be reduced further to yield effects. Second, teachers in these studies may not have adequately differentiated instruction, so that adjustments in group size are associated with differential outcomes. Third, for students struggling with reading after Grade 3, receiving the same instruction in a smaller group size may not be sufficient for improving student outcomes. … For secondary students with significant reading difficulties, very intense and sustained interventions may be required to maintain reading growth each year of school (p.187-8).
Wanzek, J., Vaughn, S., Scammacca, N.K., Metz, K., Murray, C.S., Roberts, G., & Danielson, L. (2013). Extensive reading interventions for students with reading difficulties after Grade 3. Review of Educational Research, 83(2), 163-195.
“There was no reliable indication that one-to-one interventions were associated with greater effect sizes, perhaps calling into question whether Tier III students (similar to the reading disabled category used here) are always best treated by one-to-one interventions (see also Scholin & Burns, 2012). Based on the current study, it would appear more important that students in need receive the appropriate services, with it being less important if these are offered in individual or small-group settings” (p.90).
Suggate, S.P. (2016). A meta-analysis of the long-term effects of phonemic awareness, phonics, fluency, and reading comprehension interventions. Journal of Learning Disabilities, 49(1) 77–96.
“Elbaum, Vaughn, Hughes, and Moody’s (2000) meta-analysis provides support for the view that benefits of small group instruction are equal to one-to-one tutoring in elementary reading instruction. Several studies showed that when highly qualified teachers rigorously implement a well-designed intervention, the academic benefit to students is the same, whether students are taught individually or in a group of 2 to 6 students. Conversely, a recent synthesis of evidence-based studies with struggling readers indicates that, although small group tutorials can be effective, they are not as effective as one-to-one instruction by teachers or paraprofessionals (Slavin, Lake, Davis, &Madden, 2009). Further, Ritter, Barnett, Denny, and Albin’s (2009) meta-analysis found elementary and middle school volunteer tutoring programs compared to no tutoring had significant positive effects on reading letters and words, oral reading fluency, writing, and reading in general” (p.220).
Mellard, D., McKnight, M., & Jordan, J. (2010). RTI tier structures and instructional intensity. Learning Disabilities Research & Practice, 25(4), 217–225.
“This longitudinal randomized-control trial investigated the effectiveness of scientifically based reading instruction for students with IQs ranging from 40 to 80, including students with intellectual disability (ID). Students were randomly assigned into treatment (n - 76) and contrast (n = 65) groups. Students in the treatment group received intervention instruction daily in small groups of 1 to 4 for approximately 40 to 50 min for 1 to 4 academic years. On average, students in the treatment group made significantly greater progress than students in the contrast condition on nearly all language and literacy measures. Results demonstrate the ability of students with low IQs, including students with mild to moderate ID, to learn basic reading skills when provided appropriate, comprehensive reading instruction for an extended period of time” (p.287).
Allor, J. H., Mathes, P. G., Roberts, J. K., Cheatham, J. P., & Al Otaiba, S. (2014). Is scientifically based reading instruction effective for students with below-average IQs? Exceptional Children, 80(3), 287-306.
"Group size. Evidence from reviews of the literature has shown small group delivery (typically three to four pupils per adult) can be as effective as individual tutoring (1:1) when effect sizes are compared across studies employing one of these two methods of delivery (Elbaum, Vaughn, Hughes & Moody, 2000; Vaughn, Linan-Thompson, Kouzekanani et al., 2003; see also Hatcher et al., 2006). To our knowledge, only one study has manipulated group size using the same intervention programme (Helf, Cooke & Flowers, 2009), reporting equivalent levels of programme efficacy for small group instruction (1:3) relative to individual tutoring (1:1). Even with older pupils with severe and persisting dyslexic reading difficulties, there is evidence from well-controlled studies that small group intervention, when intensive (e.g. 100 hours), can be as effective as individually delivered intervention, bringing below average pre-intervention reading levels into the average range (Lovett et al., 2000; Rashotte et al., 2001; Torgesen, Rashotte, Alexander, Alexander & MacPhee, 2003). Such findings have clear implications for costs, when making decisions about effective Wave 2 reading intervention programmes” (p.101-102).
Griffiths, Y., & Stuart, M. (2013). Reviewing evidence-based practice for pupils with dyslexia and literacy difficulties. Journal of Research in Reading, 36(1), 96-116.
"Most of the studies I've read haven't carefully examined group size, but typically simply note that smaller group sizes usually produce larger effects than bigger groups. See attached.
Remember, however, that many of the recommendations of around three students per group relate to analyses of different studies - of varying levels of design and implementation care. All other things being equal, small is better. However, when a well-trained practitoner uses carefully designed and implemented programs, this rule of thumb doesn't necessarily apply, as you well know.
I haven't come across a magic rule of 6."
Vaughn, S., & Linan-Thompson, S. (2003). Group size and time allotted to intervention: Effects for students with reading difficulties. In B. Foornun (Ed.), Preventing and remediating reading difficulties: Bringing science to scale (pp. 275-298). Parkton, MD: York Press.
Iversen, S., Tunmer, W. E., & Chapman, J. W. (2005). The effects of varying group size on the Reading Recovery approach to preventive early intervention. Journal of Learning Disabilities, 38, 456–472.
Torgesen, J.K., Rashotte, C.A., Mathes, P.G., Menchetti, J.C., Grek, M.L., Robinson, C.S., et al. (2003). Effects of teacher training and group size on reading outcomes for first-grade children at-risk for reading difficulties. Unpublished manuscript. Florida State University, Tallahassee.
"Students with the most intensive instructional needs require more of their instruction delivered in small groups (Bos & Vaughn, 2002). Small group instruction focused on prioritized skills increases the instructional support in meaningful ways by allowing instruction to be efficient by targeting the specific skill needs for students. In addition, small group instruction increases students’ opportunities to practice skills and receive corrective feedback from teachers to enhance learning (Howell & Nolet, 2000). Although there is no agreed on number of how many students makes a “small group,” group size can vary significantly from 1-to-1 to as many as 1-to-10. Although not conclusive for making individual student-level decisions, there is compelling research indicating that instruction provided to groups of 3 to 5 students is as effective as 1-to-1 instruction, even for the most at-risk students (Elbaum, Vaughn, Hughes, & Moody, 2000). Some students may require 1-to-1 instruction; however, prior to allocating this intensive level of resources, school personnel could try a less intensive grouping format, 1-to-3 to 5, first and intensify if student performance warrants it (Fuchs, Fuchs, Mathes, & Simmons, 1997). The use of small group formats to provide supplemental instruction may require a change in resource allocation within schools by increasing the number of personnel and space needed and the complexity of scheduling. Schools with limited personnel may have to provide “small group instruction” to struggling readers in arrangements beyond what would be instructionally effective (i.e., 8 or more). Larger instructional groups minimize the likelihood that interventions will be targeted to student instructional needs and provide the active engagement necessary to support and maximize student learning."
Wanzek, J., & Vaughn, S. (2008). Response to varying amounts of time in reading intervention for students with low response to intervention. Journal of Learning Disabilities, 41(2), 126-142.
"Group size The studies summarized in Table 4 also vary in the size of the tuition group, from one to four students per teacher (average 2.3 students). Because of confounding variables such as degree of impairment, overall amount of instruction and qualifications and experience of instructors, analysis of the effects of group size is problematic. Nevertheless, inspection of the results of the various studies in Table 4 does not suggest that, on the whole, literacy gains were any better in studies where tuition was individual than in those in which groups of pupils were taught together. Indeed, the studies by Torgesen et al. (2003), Wise et al. (1999) and Rashotte et al. (2001), which all employed groups of four pupils, all obtained good results. The study by Rashotte et al. (2001) is particularly noteworthy in that not only was instruction delivered in small groups of four pupils rather than 1:1, it was also over a relatively short period of 8 weeks, and by instructors who were trained (6-day training programme) but who were not specialists in remedial education (only one of the four instructors was a qualified teacher). Comparable results in favour of using instructors who are not necessarily teachers were obtained in the successful secondary intervention study by Vadasy et al. (2002) reported above. In this case, intervention was provided by parents and other adults from within the community who had been trained to deliver the programmes. Unsurprisingly, intervention programmes delivered to small groups of children by instructors who are not necessarily teachers is a great deal more cost-effective than programmes requiring 1:1 tuition delivered over much longer periods by specialist teachers whose training is lengthy and expensive. For example, supposing an instructor taught four groups of three children each day, and was paid £15 per hour plus an allowance of two additional paid hours per day for planning and preparation, the overall cost per child for the 35 hours of instruction would be £275. However, it should be noted that the students in the study by Rashotte et al. (2001) were not, in general, severely disabled in reading: at the outset of the study the average reading accuracy was standard score 89."
Singleton, S. (2009). Intervention for dyslexia: A review of published evidence on the impact of specialist dyslexia teaching. University of Hull, May 2009
“In Language! individual tutoring was not possible. Group sizes ranged from ten to 28 students per class (recommended class size should not exceed fifteen).
Moats, L.C. (2004). Efficacy of a structured, systematic language curriculum for adolescent poor readers. Reading & Writing Quarterly, 20, 145-159.
Could computers do the trick with older students?
“This paper has reported a systematic review to investigate the effectiveness of interventions designed to support the reading skills of secondary school students. It has focused only on studies which have used RCTs. The paucity of research is striking; only eight studies met our inclusion criteria. Five of the studies were conducted in the US, two in the UK and one in Australia. This is consistent with the findings of a review conducted by Slavin, Cheung, Groff, and Lake (2008) which argued that more large scale methodologically rigorous studies are needed in this area. With regards multiple component training, three of the four studies reviewed were evaluations of individualised CAI and in each case the findings did not support the use of this approach. Slavin, Lake, Davis, and Madden (2009) in reviewing approaches for struggling readers in grades K-5, concluded that CIA generally had few effects on reading. Furthermore, Khan and Gorard (2012) reported that previous studies have failed to demonstrate the effectiveness of computer based instruction as a means of improving reading skills. Taken together the evidence suggests that CAI should not be relied on to produce gains in reading ability in secondary school aged students and that in some circumstances using these programmes may have a negative impact on student’s progress (Gorard &Taylor, 2004). An avenue for future intervention design could be to combine computer administered tasks with face-to-face instruction. It is of note that these studies, which all included some focus on word recognition and decoding skills, were delivered to individual students via computer programmes. There is therefore no evidence from RCTs to show the possible effectiveness of tutor led one-to-one instruction in this essential component of reading at secondary school level. ... There is therefore a significant gap in the evidence base from RCT’s concerning the efficacy of language comprehension intervention. Clarke, Snowling, Truelove, and Hulme (2010) demonstrated using an RCT the effectiveness of an oral language intervention (comprising strategy use, vocabulary, figurative language and spoken narrative) in improving the reading comprehension skills of primary school students. To date such an approach has not been evaluated using an RCT in secondary schools.” (p.125)
Paul, S-A.S., & Clarke, P.J. (2016). A systematic review of reading interventions for secondary school students. International Journal of Educational Research, 79, 116–127.
The difficulty involved in catchup.
Kerry Hempenstall
Below is a graphical representation of what is known as learning trajectory. Graphing allows visualising what happens when students make progress over time in a given domain, for example, reading development. The average student is seen to make about a year’s progress for each year of instruction. The slope of the line is an indicator of the rate of this progress. In reality, of course, this is an over-simplification. However, it is a useful device when considering the complexities of catchup.
In the first graph, we note that at the end of grade 1 the student’s achievement in reading (grade 1 level) corresponds to that expected after the year of instruction. By the end of grade 7 the progress has been more or less constant, and the attainment corresponds to grade 7. The angle of the line (the slope) in this graph represents an average rate of progress, and the trajectory is 1:1. It takes 7 years to make 7 years progress.
A student who makes more rapid progress would have a steeper slope, with a trajectory greater than 1:1. For example, a student reading at a grade 10 level in grade 7 would have a trajectory of 10:7.
In the second graph, we see that from the start a student is making consistently slower progress. After 7 years the student’s attainment is about a grade 4 standard. The slope in this graph represents a below average rate of progress, and the trajectory is 4:7. It takes 7 years to make 4 years progress. In many cases, progress of the struggling student is better represented by a curve rather than a straight line.
In the third graph, we see the two graphs superimposed and the schooling extended to grade 10. Here we see represented the challenge in starting an intervention after the early years. If the student is to catch up in reading by grade 10, the slope of progress must be dramatically increased. In the next 3 years, the student must make 6 years progress. This would represent two years progress for each year of instruction. The trajectory from grade 7 to grade 10 would be 2:1. This is a steeper slope than even the rapidly progressing students have accomplished. Remember too that the rapidly progressing student continues to do so (assuming motivation is maintained) because the skills and domain knowledge preceding the current learning challenge have been entrenched and automatized over the years through practice. These develop as a consequence of reading volume (both instructional and for pleasure), and include vocabulary growth, a higher developed sense of syntax, and conceptual/world knowledge. The difference in reading volume between even the average reader (50th percentile) and the struggling reader (10th percentile) is of the order of 15 times (Anderson, Wilson, & Fielding, 1988).
These lower order components of learning are now automatic, allowing the rapidly progressing student to focus only on the new challenge. By contrast, the student with a history of learning struggles hasn’t fully internalised those earlier skills and knowledge systems. So, many of the new challenges faced by the student require regularly reviewing those earlier components that are taken for granted even by his average progress peers.
There is also the impact of long-term failure on the student’s sense of self-efficacy and motivation. See these pages http://www.nifdi.org/resources/news/hempenstall-blog/404-literacy-and-mental-health; http://www.nifdi.org/resources/news/hempenstall-blog/405-literacy-and-behaviour for an extended coverage of the behavioural and mental health issues that may impede catchup.
This may partly explain the finding from some of the research into the progress made by students whose early struggles have not been addressed until late in elementary or early high school. In the literacy domain, the research has pointed to the possibility of reasonable success in addressing a student’s decoding skills, although the instructional intensity of interventions needed (even when evidence-based approaches are employed) is all too rarely achieved. Even so, catch-up in reading fluency and reading comprehension has proved more elusive (Spencer & Manis, 2010). It is likely that long-term, intensive, multi-component evidence-based interventions are required by these students, and the time and resources needed to achieve this is a serious challenge to the school system (Vaughn et al., 2011; Wanzek, Wexler, Vaughn, & Ciullo, 2010).
These graphs are simplifications of what occurs. In fact, most readers’ progress is not linear, but rather has rises and plateaux. For many struggling readers a slowing rate is evident, for reasons described above.
More on problems with Catch-up
“These results thus support previous studies conducted mainly with older children showing that anxiety, social problems, and conduct problems were closely associated with literacy difficulties (Casey et al., 1992; Conners, 1997; Willcutt & Pennington, 2000). These finding are also consistent with research showing that kindergarten academic variables have been shown to predict problem behavior at the end of elementary school (McIntosh, Chard, Boland, & Horner, 2006), with an increasing relationship over years of schooling.” (p. 199-200)
Pierce, M.E., Wechsler-Zimring, A., Noam, G., Wolf, M., & Tami Katzir, T. (2013). Behavioral problems and reading difficulties among language minority and monolingual urban elementary school students. Reading Psychology, 34(2), 182-205.
“This survey showed the difficulty of closing students’ gaps in the middle years (from 4th to 8th grade). Fewer than 10%” of far off track students (more than one standard deviation below benchmark in 4th grade) caught up in the four years to 8th grade. Between 8th grade and 12th grade only 6% of those far off track students in 8th grade reached benchmark by 12th grade”.
Dougherty, C. (2014). Catching up to college and career readiness: The challenge is greater for at-risk students. ACT Research & Policy, May 2014. 1-12. Retrieved from http://www.act.org/research/policymakers/pdf/CatchingUp-Part3.pdf
“As early as first grade, a pattern is established whereby children with strong early reading skills engage in reading more than their less skilled peers. Through reading, they strengthen not only their reading skills but also reading-related and cognitive skills such as spelling, vocabulary, listening comprehension, and declarative knowledge. The roots for this productive habit can be seen in early exposure to print through caregiver shared reading experiences and effective early reading instruction in which strong decoding skills are established. Some researchers have conceptualized this relationship between strong reading skills, engagement in reading, and development of reading-related and cognitive abilities as a ‘‘virtuous circle’’ (Snowling & Hulme, 2011). Other researchers have described the process by which children who fail to establish early reading skills find reading to be difficult and unrewarding, avoid reading and reading-related activities, and fail to develop reading-related and cognitive abilities as a ‘‘vicious circle’’ that is disastrous for their cognitive development and school achievement (Pulido & Hambrick, 2008). An early start in learning to read is crucial for establishing a successful path that encourages a ‘‘lifetime habit of reading’’ (Cunningham & Stanovich, 1997, p. 94) and for avoiding the decline in motivation for reading that can have devastating effects on reading growth and cognitive development over time” (p.209-210).
Sparks, R. L., Patton, J., & Murdoch, A. (2014). Early reading success and its relationship to reading achievement and reading volume: Replication of ‘10 years later’. Reading and Writing, 27(1), 189-211.
“A major goal of Tier 2 or secondary intervention is to allow the majority of students with learning (e.g., reading) difficulties to attain grade-level expectations. If students with below-grade- level performance are to catch up with normally developing students, their rate of growth must be accelerated; simply learning at an average rate will only maintain the deficit. Thus, Tier 2 interventions must be intensive enough to not only improve students’ performance, but to actually enable students with learning difficulties to progress at rates that are faster than the learning rates of average students. At the same time, these interventions must be feasible for teachers to implement and sustain” (p.433).
Vaughn, S., Denton, C. A., & Fletcher, J. M. (2010). Why intensive interventions are necessary for students with severe reading difficulties. Psychology in the Schools, 47(5), 432–444.
‘‘Direct instructional time is proportional to their [children’s] deficiency. The greater the need, the more time they get.’’ Further, they caution that ‘‘catch up growth’’ requires more time and better quality instruction. Ikeda and colleagues cautioned that in most schools within the Iowa Heartland district, ‘‘interventions were not sufficiently rigorous to impact reading performance’’ (p.20).
Al-Otaiba, S., Calhoon, M. B., & Wanzek. J. (2010). Response to intervention: Treatment validity and implementation challenges in the primary grades and beyond to middle school. In Thomas E. Scruggs, Margo A. Mastropieri (Eds.). Advances in learning and behavioral disabilities, Volume 23; Learning and Literacy. Emerald Publishing.
“A child with a reading disability who is not identified early may require as many as 150 – 300 hours of intensive instruction (at least 90 minutes a day for most school days over a 1 – 3 year period) if he is going to close the reading gap… between himself and his peers. And, of coursethe longer identification and effective reading instruction is delayed, the longer the child will require to catch up” (p.259)
Shaywitz, S. (2003). Overcoming dyslexia: A new and complete science-based program for reading problems at any level. New York: Alfred A. Knopf.
What focus is optimal?
Along with vocabulary and comprehension, getting the words off the page remains an important focus for most older students
“The findings of the present study have educational implications for the remediation of reading difficulties. The main intervention required for children struggling with decoding is instruction in decoding skills. Meta-analyses of reading intervention studies have found that systematic phonics instruction has the greatest effect on decoding, sight word reading, and reading comprehension compared with other forms of instruction such as whole language and whole word approaches (Ehri et al., 2001; Hattie, 2008). Most poor readers can successfully be taught phonological decoding skills and make gains in reading through participation in evidence-based intervention programmes (Lovett & Steinbach, 1997; Moats & Foorman, 1997; Torgesen et al., 1977). Evidenced-based instructional components include explicit, systematic instruction in phonological decoding, alphabetic coding skills and word-level strategies, with opportunities to practise skills in isolation and while reading connected text (Slavin, Lake, Chambers, Cheung, & Davis, 2009; Vellutino et al., 2004). Focusing intervention on word recognition skills does not discount the importance of developing language skills. They helped with syntactic similarity miscues in this study. Language skills are also important for developing vocabulary size and general knowledge which are known to support the ability to decode unknown words and provide the reader with access to word meanings, conceptual and grammatical structures (Connor et al., 2004; Scarborough, 1991; Snowling et al., 2003).” (p.90)
Blick, M., Nicholson, T., Chapman, J., & Berman, J. (2017) Does linguistic comprehension support the decoding skills of struggling readers? Australian Journal of Learning Difficulties, 22(2), 75-94.
"The important role of phonological awareness in learning to read has become widely accepted. The purpose of this paper is to examine the role of phonological processing skills when attempting to assist older low-progress readers to develop literacy skills. While researchers generally agree that the key variables in reading acquisition (letter sound knowledge and phonological awareness) influence a student’s ability to understand and implement the alphabetic principle, it is still unresolved, however, whether or not the absence of these variables is sufficient to categorise a student as dyslexic. Furthermore it is unclear as to whether or not the research on beginning reading practices should inform the development of instructional programs for older low-progress readers. Despite a move away from traditional discrepancy criteria towards a more phonologically based definition of reading disability, the current researchers conclude that the adoption of a non-categorical approach to reading disability is the most helpful for remediation practices for older low-progress readers. This contrasts with the current pursuit of some researchers to diagnose a set of underlying causes through the development of phonological testing batteries. It is concluded that reading disability is best understood by a continuum model, where the difficulties in reading are influenced by two major causal factors; phonological ability and quality of literacy learning environment."
Pogorzelski, S., & Wheldall, K. (2005). The importance of phonological processing skills for older low progress readers. Educational Psychology in Practice, 21(1), 1-22.
“Lack of ability to accurately recognize many words that occur in grade-level text (limited “sight word” vocabulary) also limits these children’s reading fluency. In fact, recent research has demonstrated that the primary factor that limits struggling readers’ fluency is the high proportion of words in grade-level text that they cannot recognize at a single glance (Jenkins, Fuchs, van den Broek, Espin, and Deno, 2003; Torgesen and Hudson, 2006; Torgesen, Rashotte, and Alexander, 2001). Problems with reading fluency are emerging as one of the most common and difficult to remediate traits of older struggling readers (Torgesen & Hudson, 2006). For example, a recent study of the factors associated with unsatisfactory performance on one state’s third-grade reading accountability measure—a measure of comprehension of complex text—found that students reading at the lowest of five levels on the test had reading fluency scores at the 6th percentile (Schatschneider et al. 2004)”.
Torgesen, J., Myers, M., Schirm, A., Stuart, E., Vartivarian, S., Mansfield, W., Stancavage, F., Durno, D., Javorsky, R., & Haan, C. (2006). National Assessment of Title I Interim Report to Congress: Volume II: Closing the Reading Gap, First Year Findings from a Randomized Trial of Four Reading Interventions for Striving Readers. Washington, DC: U.S. Department of Education, Institute of Education Sciences.
“Fluent decoding appears to be an important predictor of reading comprehension across elementary, middle, and high school” (p.463).
Kershaw, S., & Schatschneider, C. (2012). A latent variable approach to the simple view of reading. Reading & Writing: An Interdisciplinary Journal, 25(2), 433–464.
“The vast majority of school-age struggling readers experience word-level reading difficulties (Fletcher et al., 2002; Torgesen, 2002). This “bottleneck” at the word level is thought to be particularly disruptive because it not only impacts word identification but also other aspects of reading, including fluency and comprehension (LaBerge & Samuels, 1974). According to Torgesen (2002), one of the most important discoveries about reading difficulties over the past 20 years is the relationship found between phonological processing and word-level reading. Most students with reading problems, both those who are diagnosed with dyslexia and those who are characterized as “garden variety” poor readers, have phonological processing difficulties that underlie their word reading problems (Stanovich, 1988)” (p.179).
Nelson, J.M., Lindstrom, J.H., Lindstrom, W., & Denis, D. (2012): The structure of phonological processing and its relationship to basic reading. Exceptionality: A Special Education Journal, 20(3), 179-196.
“Older students with reading difficulties benefit from interventions focused both at the word level and at the text level.Identifying need and intervening accordingly in the appropriate areas (e.g., vocabulary, word reading, comprehension strategies, and so on) is associated with improved outcomes for older students with reading difficulties”.
Scammacca, N., Roberts, G., Vaughn. S., Edmonds, M., Wexler, J., Reutebuch, C. K., & Torgesen, J. K. (2007). Interventions for adolescent struggling readers: A meta-analysis with implications for practice. Portsmouth, NH: RMC Research Corporation, Center on Instruction.
http://flare.ucf.edu/Research/Interventions%20for%20Struggling%20Readers.pdf
“Phonological decoding made a significant unique contribution to reading comprehension for the eighth/ninth-grade group, to spelling for the fourth/fifth- and eighth/ninth-grade groups, and to the decoding rate and accuracy measures for all three groups, with only three exceptions”.
Nagy, W., Berninger, V.W., & Abbott, R.D. (2006). Contributions of morphology beyond phonology to literacy outcomes of upper elementary and middle-school students. Journal of Educational Psychology, 98(1), 134-147.
“More research is also needed to explore the impact of other co-occurring difficulties, such as inattention, on responsiveness. Empirical investigation of ways to increase the motivation of older struggling readers could also impact on the effectiveness of teaching these hard-to-teach pupils” (p.112).
Griffiths, Y., & Stuart, M. (2013). Reviewing evidence-based practice for pupils with dyslexia and literacy difficulties. Journal of Research in Reading, 36(1), 96-116.
“A recent meta-analysis of 85 studies with struggling readers in preschool through 7th grades suggests that the optimal type or modality of reading intervention may vary with grade level (Suggate, 2010). Phonics interventions produced greater effect sizes for kindergarten and 1st grade students, while mixed (phonics with comprehension) interventions and pure comprehension interventions yielded larger effects for older students. However, given the wide range of results from adolescent intervention studies (Fuchs, Fuchs, & Kazdan, 1999; Hasselbring & Goin, 2004; Lovett, Borden, DeLuca, Lacerenza, Benson, & Brackstone, 1994; Lovett, Lacerenza, Borden, Frijters, Steinbach, & De Palma, 2000; Lovett & Steinbach, 1997; Lovett, Steinbach, & Frijters, 2000; Mastropieri et al. 2001; Vaughn et al., 2010, 2011,2012), additional research on this issue is needed to provide a more complete picture that can inform the design and delivery of instruction for older struggling readers (Suggate, 2010)” (p.566).
Calhoon, M. B., & Prescher, Y. (2013). Individual and group sensitivity to remedial reading program design: Examining reading gains across three middle school reading projects. Reading and Writing: An Interdisciplinary Journal, 26, 565-592.
“A key and unique finding from this meta-analysis is the greater retention of intervention effect to follow-up for at-risk, low, and disabled readers in comparison to normal readers. This finding is certainly encouraging for interventionists targeting struggling readers, suggesting that promising long-term effects are attainable. There was no reliable indication that one-to-one interventions were associated with greater effect sizes, perhaps calling into question whether Tier III students (similar to the reading disabled category used here) are always best treated by one-to-one interventions (see also Scholin & Burns, 2012). Based on the current study, it would appear more important that students in need receive the appropriate services, with it being less important if these are offered in individual or small-group settings” (p.90).
Suggate, S.P. (2016). A meta-analysis of the long-term effects of phonemic awareness, phonics, fluency, and reading comprehension interventions. Journal of Learning Disabilities, 49(1) 77–96.
“Based on the research reviews and meta-analyses on adolescent literacy instruction, recommendations can be organized into five general areas: word study, fluency, vocabulary, comprehension, and motivation (Boardman et al., 2008; Kamil et al., 2008; Roberts et al., 2008; Scammacca et al., 2007; Torgesen et al., 2007)” (p.167).
Marchand-Martella, N.E., Martella, R.C., Modderman, S.L., Petersen, H.M., & Pan, S. (2013). Key areas of effective adolescent literacy programs. Education and Treatment of Children, 36(1), 161-184.
“Post hoc analyses indicated that students in the treatment group who entered the study with higher decoding skills (TOWRE > 93) achieved larger gains in comprehension than those with lower decoding skills. The extent to which deficits in word reading interfere with reading comprehension for adolescent readers has been documented in a previous study conducted by Hock et al. (2009). These findings support the notion that secondary students with reading comprehension difficulties and lower decoding skills (TOWRE < 92) will continue to need instruction with word-study as component of intensive interventions” (p.111).
Solis, M., Vaughn, S., & Scammacca, N. (2015). The effects of an intensive reading intervention for ninth graders with very low reading comprehension. Learning Disabilities Research & Practice, 30(3), 104–113.
“Older students demonstrate a broad and complex range of difficulties related to reading. These include problems in recognizing words, understanding word meanings, and understanding and connecting with text; students often lack background knowledge required for reading comprehension (Biancarosa & Snow, 2004). We examined several syntheses on interventions for secondary students with reading difficulties to identify effective interventions to meet this range of reading difficulties. Edmonds et al. (2009) conducted a meta-analysis examining the effects of adolescent reading interventions (Grades 6 through 12) that included instruction in decoding, fluency, vocabulary, or comprehension on reading comprehension outcomes. Analyses revealed a mean weighted effect size in the moderate range in favor of treatment students over comparison students. Promising approaches were those that provided targeted reading intervention in comprehension, multiple reading components, or word-recognition strategies” (p.392).
Vaughn, S., Wexler, J., Roberts, G., Barth, A.A., Cirino, P.T., Romain, M.A., Francis, D., Fletcher, J., & Denton, C.A. (2011). Effects of individualized and standardized interventions on middle school students with reading disabilities. Exceptional Children, 77(4), 391-407.
“Roberts et al. (2008) suggest the five areas recognised by the NRP as key ingredients for early reading intervention should be adapted for older readers to include: (i) word study, (ii) fluency, (iii) vocabulary, (iv) comprehension and (v) motivation. Low levels of motivation are a common barrier to learning (Guthrie & Davis, 2003) and a predictor of response to intervention (RTI: Duff, 2008), particularly in older pupils. Reduced reading experience following a long-lasting reading difficulty may also impact on a pupil’s spoken and written vocabulary, reading fluency and effective comprehension strategies. Hence, careful assessment and diagnosis of older pupils is essential to ensure the appropriate programme of intervention is provided.” (p.100).
Griffiths, Y., & Stuart, M. (2013). Reviewing evidence-based practice for pupils with dyslexia and literacy difficulties. Journal of Research in Reading, 36(1), 96-116.
“Reading comprehension instruction was included in nearly all of the multicomponent interventions. The research base continues to show that teaching reading comprehension strategies to struggling readers in Grades 4 to 12 is beneficial. In addition, the most current research affirms that teachers can provide effective reading interventions. The mean effect sizes for teacher- and researcher-provided interventions in the 2005-2011 group of studies were nearly identical, both on all measures and on standardized measures. A greater proportion of the studies in the 2005-2011 group used teachers to implement the intervention (26 of 50, 52.0%, compared to 12 of 32, 37.5%, in the 1980-2004 group of studies). The largest and most rigorous studies relied on teachers to implement the intervention (e.g., Lang et al., 2009, with N = 1,197; Somers et al., 2010, with N = 5,595). Therefore, it appears that teachers increasingly are being trained as interventionists and are proving to be as effective as researchers at providing interventions. Finally, the most current data show that reading interventions are effective both for struggling readers with LD and those not identified as having LD. No differences based on LD status were found when looking only at the 2005-2011 group of studies. Notably, most of the studies in the 2005-2011 group included both students with and without LD. Therefore, the most recent research suggests that all struggling readers benefit from intervention regardless of their diagnosed LD status.” (p. 386-7)
Scammacca, N., Roberts, G. Vaughn, S., & Stuebing, K. K. A. (2015). A metaanalysis of interventions for struggling readers in Grades 4–12:1980–2011. Journal of Learning Disabilities, 48(4) 369–390.
“In a recent meta-analysis examining the effect of print exposure from infancy through young adulthood, Mol and Bus (2011) found that the role of print exposure becomes stronger (additive) as children get older. Their findings showed that print exposure explained increasing amounts of variance in the oral language skills of preschoolers and kindergarteners (12 %) and students in primary school (13 %), middle school (19 %), and high school (30 %). At the postsecondary level, print exposure explained 34 % of the variance in the oral language skills of undergraduate and graduate students. Although the aforementioned evidence suggests that reading should start early to take advantage of the positive effects of print exposure, Stanovich et al. (1996) have indicated that exposure to print is helpful regardless of children’s cognitive ability or their level of reading comprehension. Therefore, it is crucial to ensure that young children are taught the word recognition skills needed for successful reading early in school so that they have the opportunity to become active and engaged readers. Likewise, it is equally important to provide broad and frequent reading experiences for older children, particularly those with low verbal abilities, because reading itself improves the language skills they need to become strong readers (Cunningham & Stanovich, 2001)” (p.190-191).
Sparks, R. L., Patton, J., & Murdoch, A. (2014). Early reading success and its relationship to reading achievement and reading volume: Replication of ‘10 years later’. Reading and Writing, 27(1), 189-211.
“Many (but not all) older children with severe reading disabilities (grades 3 through 5) can significantly improve their reading skills with intensive intervention approaches that emphasise direct remediation of phonological processing and the systematic integration of these phonological skills into phonics instruction, textual reading, and reading comprehension strategies” (p. 579).
Lyon, G. R., & Moats, L.C. (1997). Critical conceptual and methodological considerations in reading intervention research. Journal of Learning Disabilities, 30, 578-588.
“The need for progress monitoring tools is also essential for students with learning disabilities, whose reading progress can be slow and incremental without implementation of highly effective interventions and whose reading progress may not be captured by end of year final status assessments (Deno, Fuchs, Marston, & Shin, 2001; Fuchs, Fuchs, Hamlett, Walz, & Germann, 1993; Ticha et al., 2009). This study attempted to fill this gap in the literature by demonstrating that ORF probes for the first minute of reading or for the full passage are reliable and valid measures of indexing reading ability among struggling middle-grade readers and may be used as part of a secondary-school teachers’ intervention design to measure reading abilities and inform instructional decisions.” (p. 62)
Barth, A. E., Stuebing, K. K., Fletcher, J. M., Denton, C. A., Vaughn, S., & Francis, D. (2014). The effect of reading duration on the reliability and validity of middle school students' ORF performance. Assessment for Effective Intervention, 40(1), 53–64.
“We evaluated the technical adequacy of oral reading fluency (ORF) probes in which 1,472 middle school students with and without reading difficulties read fluency probes for 60 s versus reading the full passage. Results suggested that the reliability of 60-s probes (rs ≥ .75) was not substantively different than full passage probes (rs ≥ .77) among struggling readers and typically developing readers in Grades 6 to 8. “ (p. 53).
Barth, A.E., Stuebing, K.K., Fletcher, J.M., Denton, C.A., Vaughn, S., & Francis, D. (2014). The effect of reading duration on the reliability and validity of middle school students' ORF performance. Assessment for Effective Intervention, 40(1), 53–64.
“My prediction is that the new ORF norms won't change much from our 2006 norms (or our 1992 norms). My prediction is based on the fact that ORF is, outside of expected measurement error (which Christ & Coolong-Chaffin, 2007 suggest is in the range of 5 wcpm for grades 1 and 2 and 9 wcpm in grades 3-8+), fairly stable. You can see evidence of this on our 2006 norms when looking at the spring 50th %iles for grades 6 (150), grade 7 (150), and grade 8 (151). When you think that these three scores represent approximately 30,000 students reading a variety of grade level passages that pretty darn stable. Other studies of older readers (high school; college) also find that 150 wcpm is a common "average.”
Hasbrouck, J. (2016). Are oral reading norms accurate with complex text? Retrieved from http://www.shanahanonliteracy.com/search/label/text%20complexity
Program fidelity is very important in evidence-based practice
How do we find out whether psychotherapy works? Efficacy and effectiveness studies.
An efficacy study contrasts some kind of therapy to a comparison group under well-controlled conditions. Patients with additional problems are not usually selected as participants. Patients are randomly assigned. Controls are rigorous. Placebos control for such influences as rapport, expectation of gain, and sympathetic attention. Treatments are manualized. Fidelity to the manual is assessed using videotaped sessions. Patients are seen for a fixed number of sessions. The target outcomes are well operationalized. Raters and diagnosticians are blind to which group the patient belongs.
Effectiveness studies of treatment use a different method.
A survey of large numbers of people who have gone through a single form of treatment. It samples treatment as it is actually delivered in the field. Patients may have additional problems. The program manual may be ignored entirely or some elements altered by practitioners. Does a program that has "proof of concept", and that has worked with one class of patients still work under these conditions? How much "tolerance" does the program have?
There are some obvious parallels with educational programs. In education programs, however, there is rarely provided the level of detail that is routinely specified in a psychotherapy treatment manual.
Education program designers usually assume that teachers know how to structure a lesson effectively, if they are provided with some worthwhile content. This assumption is far from universally justified. The content may be research-based, but its presentation may be competent, slipshod, or cursory; corrective feedback may or may not occur systematically; mastery by students may or may not be expected; and practice opportunities may or may not be adequate. Regular data-based monitoring may or may not occur. Teacher creativity may abound. This loose coupling between content and delivery would horrify an empirically-trained psychologist, as it would a surgeon. It also highlights why the crucial element in evaluation is not simply that a program is consistent with scientific findings, but also that it has been demonstrably successful with the target population.
It is for this reason that some programs provide a high degree of specificity as to how the program is to be implemented. In some cases (e.g., Direct Instruction) this involves scripted lessons – a means of increasing the likelihood that the program that has been shown to be effective in empirical trials is replicated in the classroom. This is rather similar to protocols carefully followed by surgeons, pilots, and disaster management professionals. It is often described as program fidelity or treatment integrity.
So what is program fidelity?
“High fidelity implementation means that you get a program with an internal design and follow that design. That would include using the materials in a particular sequence, adhering to the amount of time and practice called for by the program and following the recommendations for grouping or re-teaching students. It would mean using of all the essential components as they are designed, including differentiated instructional time and program assessments”.
Diamond, L. (2004). High fidelity—It’s all about instructional materials. An interview with Linda Diamond of CORE. Retrieved from http://www.corelearn.com/backup/PEV-SITE/about/printable-documents.html
“In the medical sphere there are well-established protocols that need to be adhered to prior to the introduction of any new drug or treatment. No such protocols apply in education, an area in which lives are also at stake (Dinham, 2014b)” (p.14).
Dinham, S. (2015). The worst of both worlds: How the U.S. and U.K. are influencing education in Australia. Education Policy Analysis Archives, 23(49). Retrieved from http://dx.doi.org/10.14507/epaa.v23.1865
“Fidelity of implementation is traditionally defined as the extent to which the intervention is implemented as designed during an experimental study (e.g., Hord, Rutherford, Huling- Austin, & Hall, 1987; National Research Council, 2004). Dane and Schneider (1998) reported that there are five criteria for measuring fidelity of implementation. These criteria include (a) adherence, whether the components of the program are being delivered as designed; (b) exposure, the number, length, or frequency of sessions being implemented; (c) quality of delivery, the manner in which the implementer delivers the program using the prescribed methods and techniques; (d) responsiveness, the extent to which the participants are engaged by and involved in the activities and content of the program; and (e) program differentiation, whether critical features that distinguish the program from the comparison condition are present or absent during implementation (O’Donnell, 2008). More specifically, fidelity of implementation can be differentiated into two primary categories: (a) fidelity of structure (i.e., adherence and exposure) and (b) fidelity of process (i.e., program differentiation, quality of delivery, and responsiveness; Dane & Schneider, 1998; Mowbray, Holter, Teague, & Bybee, 2003)” (p.79-80).
“Our analyses revealed that overall fidelity of implementation accounted for 22% of the variance in the gains in basic reading skills and 18% of the passage comprehension gains of middle school students with reading difficulties” (p.85). … The findings of the present study are consistent with previous research that has demonstrated that fidelity of implementation has statistically and educationally significant effects on student outcomes (Allinder et al., 2000; Hall & Loucks, 1977; Penuel & Means, 2004; Songer & Gotwals, 2005; Ysseldyke et al., 2003)”. … Our findings have implications for the challenge of moving effective approaches to practice, particularly those designed to close the reading achievement gap. Cook, Landrum, Tankersley, and Kauffman (2003) highlight that approaches may be rendered ineffective or counterproductive if not used with adequate dosage (amount of treatment) or when implemented without adequate fidelity. Placing this concern in the context of the present investigation, teachers implementing Corrective Reading Decoding (an evidence-based remedial reading intervention) with low fidelity did not experience large reading improvements commensurate with their colleagues implementing with high fidelity. Thus, these teachers may feel justified in concluding that Corrective Reading Decoding is not effective in building the reading skills of striving middle school readers. Limited or no consideration to fidelity of intervention is a large threat to internal validity; without consideration of level of fidelity, it is difficult to ascertain whether the intervention was responsible for enhanced or constrained treatment outcomes. We draw on and concur with the more than 30-year-old findings of Hall and Loucks (1977), who found that those implementing an innovation vary on adherence to the structure and quality of implementation of the innovation. Our findings underscore that reading outcomes appear to significantly vary according to how well the intervention was delivered and the degree to which the structure of lessons was followed”. (p.86).
Benner, G.J., Nelson, J.R., Stage, S.A., & Ralston, N.C. (2011). The influence of fidelity of implementation on the reading outcomes of middle school students experiencing reading difficulties. Remedial and Special Education, 32, 79–88.
“In intervention research, treatment fidelity is defined as the strategies that monitor and enhance the accuracy and consistency of an intervention to ensure it is implemented as planned and that each component is delivered in a comparable manner to all study participants over time. Reviews of the literature in special education and other disciplines reveal that reports of treatment fidelity are limited” (p.121).
Smith, S.W., Daunic, A.P., & Taylor, G.G. (2007). Treatment fidelity in applied educational research: Expanding the adoption and application of measures to ensure evidence-based practice. Education & Treatment of Children 30(4), 121-134.
“In duplicating a research study, administrators need to read carefully the details of the program they are hoping to implement. Dynarski (2010) states that "knowledge drawn from science doesn't come with instructions on how to put it into practice" (p. 61). The administrator as researcher will need to ask many questions: How was the intervention implemented? Who administered it? Under what conditions? For what duration of time? Principals sometimes "tweak" programs to suit their particular school setting, but a lack of implementation fidelity or a failure to adhere to the study's implementation protocol may affect the outcomes. Significantly higher outcomes are achieved when programs are implemented as intended by the developer (O'Donnell, 2008)” (p. 124).
Bair, M. A., & Enomoto, E. K. (2013). Demystifying research: What's necessary and why administrators need to understand it. National Association of Secondary School Principals. NASSP Bulletin, 97(2), 124-138.
“Increasing the likelihood of teachers implementing research-based strategies in authentic school settings is a major goal of education leaders. Likewise, decreasing the variability of instruction practices and increasing fidelity of implementation to models of instruction and intervention is particularly difficult (Gersten, Chard, & Baker, 2000; Gresham, MacMillan, Beebe-Frankenberger, & Bocian, 2000). To address these issues in the context of ECRI, we developed highly specified lesson plans and teaching routines to support standard implementation of instruction and intervention materials. Our goal was to increase the level of specificity to ensure that teachers provided students with explicit and, when appropriate, intensive instructional supports (i.e., in the context of both Tier 1 and Tier 2). These routines provided clear expectations to teachers for what content to cover during instruction and intervention lessons and highly specified guidance for explicit and engaging teacher- student interactions. Akin to the Checklist Manifesto (Gawande, 2009), the goal of the specified routines was to increase the degree to which practitioners implement evidence- based practices with fidelity and integrity. The approach of using highly specified instruction and intervention routines can also be used as a tool for coaches and school leaders to define and measure implementation fidelity and to provide subsequent implementation goals for teachers. It is important to note that school based personnel (rather than researchers) delivered both the Tier 1 portion and the Tier 2 portions of the model. Having school personnel as implementers, notably a unique feature of this study, increases the external validity of the study’s results. The study findings also have potential implications for publishers and developers of core reading programs and tier 2 interventions. First, in our opinion, the degree of specificity and guidance provided to teachers for delivering explicit instruction in current reading programs is lacking. Many programs do not provide enough explicit, scaffolded instruction or practice opportunities for learners at risk of reading difficulty (Gersten, 1999). Second, core program and intervention developers and publishers should strive to align instruction and intervention materials to ensure struggling students are delivered a robust and coherent tiered support plan.” (p.617)
Fien, H., Smith, J. L. M., Smolkowski, K., Baker, S. K., Nelson, N. J., & Chaparro, E. A. (2015). An examination of the efficacy of a multitiered intervention on early reading outcomes for first grade students at risk for reading difficulties. Journal of Learning Disabilities, 48(6), 602–621.
“Skills-based instruction here means instruction reflecting an intent to strengthen academic skills (e.g., lettersound correspondence and math problem solving) and to enhance knowledge in areas such as social studies and science. We also use the term to signify an approach inspired by Direct Instruction (DI; e.g., Becker, Englemann, Carnine, & Rhine, 1981). According to Gersten, Woodward, and Darch (1986), the key to DI is that "materials and teacher presentation of [these] materials must be clear and unambiguous" (p. 18), "much more detailed and precisely crafted" (p. 19) than the norm, for successful use with students with academic challenges. Moreover, wrote Gersten et al. (1986), this instruction "must contain clearly articulated [learning] strategies" (p. 19): a step-bystep process involving teaching to mastery, a procedure for error correction, a deliberate progression from teacher-directed to student-directed work, systematic practice, and cumulative review (cf. Gersten et al., 1986). A belief in the efficacy of skills-based instruction seems well founded. When implemented with fidelity, carefully scripted programs in reading, writing, and math - often involving learning strategies similar to DI - have been shown to benefit numerous at-risk students (e.g., Graham ÒC Perin, 2007; Kroesbergen & Van Luit, 2003; Stuebing, Barth, Cirino, Francis, & Fletcher, 2008)” (p.263).
Kearns, D. M., & Fuchs, D. (2013). Does cognitively focused instruction improve the academic performance of low-achieving students? Exceptional Children, 79(3), 263-290.
“The evidence base reviewed above, of the characteristics of effective Wave 2 intervention programmes for early and more persisting word reading difficulties, suggests more research is needed to better understand the role of: (a) instructional intensity (length of intervention, hours of instruction, optimal ratios of teachers to students, reading time, etc.); (b) programme integrity/fidelity; (c) teacher ability/experience; (d) programme focus/explicitness/multidimensionality; and (e) individual student prior instructional experiences/exposure and reading abilities. The ways in which these factors, individually and together, affect treatment outcomes are just beginning to be addressed, particularly for treatment resisters (Shaywitz et al., 2008)” (p.106).
Griffiths, Y., & Stuart, M. (2013). Reviewing evidence-based practice for pupils with dyslexia and literacy difficulties. Journal of Research in Reading, 36(1), 96-116.
“Our findings from the present study have several important implications for serving students with low IQs in general and special education settings. First and foremost, students with low IQs, including those with ID and those with IQs in the borderline range (i.e., 70-80), should be provided with evidence-based reading instruction. Although it might seem unsurprising to some that these students made meaningful progress, our study provides strong empirical evidence of reading progress across several academic years with a relatively large sample of students with low IQs who participated in a randomized control trial in which the treatment was delivered by highly trained interventionists. Specifically, our data indicate what is possible for students with low IQs if they are given access to evidence-based reading instruction. The curriculum is very explicit and systematic and was delivered with fidelity, providing very consistent, explicit, and repetitive routines, focusing on key skills, and delivering clear and explicit modeling. Thus, students with low IQs do benefit from comprehensive reading programs that were designed for struggling readers and readers with LD, but progress is slower” (p. 302-3).
Allor, J. H., Mathes, P. G., Roberts, J. K., Cheatham, J. P., & Al Otaiba, S. (2014). Is scientifically based reading instruction effective for students with below-average IQs? Exceptional Children, 80(3), 287-306.
“A recent meta-review of five intervention studies reported in the United States identified seven cognitive–linguistic variables related to variation in RTI, listed from strongest to weakest predictor (see Duff, 2008 for further details): slow rapid naming (RAN), problem behaviour, poor PA, limited understanding of the alphabetic principle, weak verbal memory, IQ and demographics. Environmental factors influencing RTI potentially include quality of Wave 1 teaching, point of intervention (early or late, where ‘late’ is defined as after KS 1 in England or G2 in the United States) and programme fidelity. The careful training, implementation, supervision and monitoring which characterises research studies may not always be observed in other circumstances with detrimental effects on the outcome of the intervention (Byrne & Fielding-Barnsley, 1995; Byrne et al., 2010; see Carter & Wheldall, 2008 for further discussion of this issue). Programme content may also influence outcome when the evidence base for inclusion of that content is weak or the content and/or implementation is inappropriate for the individual’s profile of needs, due to insufficient assessment and monitoring” (p.105).
Griffiths, Y., & Stuart, M. (2013). Reviewing evidence-based practice for pupils with dyslexia and literacy difficulties. Journal of Research in Reading, 36(1), 96-116.
“In summary, drill and practice through high-quality CAI, implemented with fidelity, can be considered a useful tool in developing students’ automaticity, or fast, accurate, and effortless performance on computation, freeing working memory so that attention can be directed to the more complicated aspects of complex tasks” (6/142).
Gersten, R., Ferrini-Mundy, J., Benbow, C., Clements, D., Loveless, T., Williams, V., Arispe, I., & Banfield, M. (2008). Report of the task group on instructional practices (National Mathematics Advisory Panel). Retrieved from the U.S. Department of Education Web site:http://www.ed.gov/about/bdscomm/list/mathpanel/report/instructional-practices.pdf
The What Works Clearinghouse (Institute of Education Sciences, 2003) identified several features of intervention research designs that improve confidence in findings from research. Three of the most significant criteria identified include (a) the use of random assignment, (b) evidence of the use of a fidelity of treatment check, and (c) the use of standardized measurements. … A fidelity of treatment check, often referred to as treatment integrity, can improve our confidence in the accuracy and consistency of an intervention’s implementation (Gresham, MacMillan, Beebe-Frankenberger, & Bocian, 2000). Data on intervention fidelity are necessary to determine whether the intervention was implemented as intended and, therefore, whether the intended intervention is responsible for the outcomes reported.
Institute of Education Sciences. (2003). What Works Clearinghouse study review standards. Retrieved from http://www.whatworks.ed.gov/.
“Some students may also need treatment with additional treatment components (e.g., fluency training, behavioral training, or vocabulary instruction). The results also highlight the importance of conducting treatment with fidelity” (p.343).
Al Otaiba, S. (2001). Children who do not respond to early literacy instruction: A longitudinal study across kindergarten and first grade Reading Research Quarterly, 36, 344-346.
"To address our concerns about variable implementation, research staff observed each tutor at least once a week. During these 15- to 30-minute observations, project staff (Vadasy or Pool) looked for the following actions: starting lessons on time, making error corrections, following lesson formats, managing student behavior, using positive encouragement strategies, and providing a full 30 minutes of instruction. A total percentage of these six behaviors was obtained for each tutor, averaging across behaviors (reported under Results). Both observers at times observed each tutor, and they frequently compared their notes. In conjunction with the observations, tutors were often given brief written or oral feedback (e.g., suggestions for another way to teach a child having difficulty, or praise for a tutor's instructional skills). At other times, project staff modeled a strategy or adjusted a student's placement in the program (e.g., directing the tutor to go back to review previous lessons or lesson components until skills were solidly mastered, or to skip lessons when students had clearly mastered a skill and needed more challenging material).
Finally, students were tested every 10 lessons on mastery of lesson content. Project staff administered these curriculum-based tests with items drawn directly from a recently completed lesson. The mastery tests were a check on the tutor's lesson pacing and the student's acquisition of skills.
Regarding fidelity of implementation, we found that providing more training in lesson components before tutors began working with children, along with increased supervision, resulted in more accurate implementation, relative to levels observed in prior field tests. Whereas in the previous field test only 30% of tutors were observed to implement the majority of the lesson activities consistent with program protocols (Vadasy et al., 1997b), in this field test 71 % of tutors were observed to be high implementors. Moreover, anecdotal evidence (e.g., tutors who increasingly followed program elements and implemented them with greater skill) suggests that the frequent supervision and technical assistance contributed to improved implementation. Obtaining more accurate program implementation was important because a previous finding had indicated a relation between fidelity of implementation and reading outcomes (Vadasy et al., 1997b)”.
Vadasy, P.F., Jenkins, J.R., & Pool, K. (2000). Effects of tutoring in phonological and early reading skills on students at risk for reading disabilities. Journal of Learning Disabilities, 33, 579-590.
An example of a fidelity checklist
(from Benner, G.J., Nelson, J.R., Stage, S.A., & Ralston, N.C. (2011). The influence of fidelity of implementation on the reading outcomes of middle school students experiencing reading difficulties. Remedial and Special Education, 32, 79–88.
Why wouldn’t a teacher follow protocols when they are written into a program?
One reason may be that the teacher doesn’t believe that instruction is a major driver of educational attainment. This teacher may view learning as a natural outcome of a student who engages with the curriculum, and hence may focus solely upon motivational strategies rather than instructional strategies: “If I can engage the student, then good things will follow”. Other teachers may believe that they have the skill to modify, adapt, and combine elements from different programs thus enhancing their effect on student progress. Others may believe that all children learn differently, and hence a single program cannot possibly meet the unique needs of each student. They may respond that protocols represent a one-size-fits-all approach. Some may find such a level of prescription as being demeaning to their professional standing or as a means of teacher disempowerment through control-oriented instructional policies.
One alternative is known as the eclectic approach
The eclectic approach implies a teacher's use of techniques and activities from a range of teaching curricula and methodologies. The teacher decides which to use depending on the aims of the lesson and the learners in the group.
Heward (2003) comments on eclecticism:
“Eclecticism Is Good
Eclecticism—using a combination of principles and methods from a variety of theories or models—is based on the realization that no single theory or model of teaching and learning is complete and error-free. It is thought that incorporating components from a number of different models will cover the gaps or deficiencies found in any single model. The logic is reasonable and, superficially, much appears to be gained by eclecticism. The problems likely to arise from unbridled eclecticism, however, outweigh its logical appeal. First, not all theories and models are equally trustworthy and valuable. The more models represented in the eclectic mix, the more likely it is that ineffective and possibly even harmful components will be included (Maurice, 1993, 2000). Second, teachers might not choose the most important and effective parts of each model, and might select weaker, perhaps ineffective components instead. Third, some strategies or components of a given model may not be effective when implemented in isolation, without other elements of the model.
Fourth, elements from different models may be incompatible with one another. For example, children in a phonics-based program should practice reading with decodable text composed of previously learned letter-sound relationships and a limited number of sight words that have been systematically taught (Grossen, 2000). Using the less decodable and often predictable text typical of some language models limits the beginning reader’s opportunity to integrate phonological skills with actual reading and encourages the use of prediction and context to comprehend a passage. Although prediction is a useful skill, children who must rely on the predictability of text will not become successful readers (Chard & Kame’enui, 2000).
Fifth, an eclectic mix might prevent any of the included models from being implemented continuously or intensely enough to obtain significant effects. A little bit of everything and a lot of nothing often reduces eclecticism to a recipe for failure (Kauffman, 1997). Sixth, teachers who use elements of multiple models may not learn to implement any of the models with the fidelity and precision necessary for best results. The eclectic practitioner is likely to be an apprentice of many models but master of none” (p. 196).
Heward, W. L. (2003). Ten faulty notions about teaching and learning that hinder the effectiveness of special education. The Journal of Special Education, 36, 186–205. Retrieved from http://www.updc.org/assets/files/professional_development/uspin/RS-2012-Heward-Artlce.pdf.
Teacher resistance
"The resistance of some teachers to prescribed evidence-based curricula has been characterised in various ways. For example,
“Research has typically reduced teacher resistance to a psychological deficit in the "resistor," who is characterized as being unwilling to change (Gitlin & Margonis, 1995; Moore, Goodson, & Hargreaves, in press) and resisting policies and programs that attempt to improve education by controlling their instructional practices (Berman & McLaughlin, 1977; Cohen, 1991; Cuban, 1993; Huberman, 1973)” (p. 130).
Defenders of this resistance consider it a principled stand:
“Professional principles are conceptions about teaching and professionalism in which teachers view themselves as professionals with specialized expertise, who have discretion to employ repertoires of instructional strategies to meet the individual needs of diverse students, hold high expectations for themselves and students, foster learning communities among students, and participate in self-critical communities of practice” (p.32).
(A newly graduated teacher in a school using Open Court as their literacy curriculum) was concerned about having "fidelity to the program, which means you follow it exactly and don't add in your creativity” (p.37). Another wanted "major ownership in teaching the lessons" (p.40). “"I don't know if this is just a power issue . . . but I don't enjoy being told what to do every day. That is kind of how I felt when I was teaching Open Court. . . . [Prescriptive programs] just don't hold true with my philosophy" (p.43).
Achinstein, B., & Ogawa, R. T. (2006). (In)fidelity: What the resistance of new teachers reveals about professional principles and prescriptive educational policies. Harvard Educational Review, 76(1), 30-63,130.
“There is a long-standing acknowledgment in behavioral research with human implementers that procedural infidelity is possible and perhaps likely (e.g., Baer, Wolf, & Risley, 1968; Billingsley, White, & Munson, 1980; LeLaurin & Wolery, 1992). In early intervention research, infidelity may be even more likely, with two “levels” of fidelity measurement: (a) whether researchers implement training procedures correctly and (b) whether indigenous implementers (e.g., early childhood special education teachers, parents) can (and do) implement interventions successfully after training” (p.173-4).
Ledford, J.R., & Wolery, M. (2013). Procedural fidelity: An analysis of measurement and reporting practices. Journal of Early Intervention, 35(2), 173-193.
"Multi-tiered system of supports represents one of the most significant advancements in improving the outcomes of students for whom typical instruction is not effective. While many practices need to be in place to make multi-tiered systems of support effective, accurate implementation of evidence-based practices by individuals at all tiers is critical to obtain student outcomes. Effective strategies to achieve program fidelity are available; however, maintaining program fidelity at the individual level remains elusive. Lessons drawn from medicine indicate strategies to maintain program fidelity should address the implementer. Medical practitioners have used self-monitoring checklists to maintain fidelity with striking results. Research evaluating strategies to maintain program fidelity at the individual level represents an important next step in the field of education. Recommendations for a systematic research agenda focused on self-monitoring checklists are presented” (p.14).
Nelson, J.R., Oliver, R.M., Hebert,M.A, & Bohaty, J. (2015). Use of self-monitoring to maintain program fidelity of multi-tiered interventions. Remedial and Special Education, 361), 14-19.
“Young children with and at risk for emotional/behavioral disorders (EBD) present challenges for early childhood teachers. Evidence-based programs designed to address these young children’s behavior problems exist, but there are a number of barriers to implementing these programs in early childhood settings. Advancing the science of treatment integrity measurement can assist researchers and consumers interested in implementing evidence-based programs in early childhood classrooms. To provide guidance for researchers interested in assessing the integrity of implementation efforts, we describe a conceptual model of implementation of evidence-based programs designed to prevent EBD when applied in early childhood settings. Next, we describe steps that can be used to develop treatment integrity measures. Last, we discuss factors to consider when developing treatment integrity measures with specific emphasis on psychometrically strong measures that have maximum utility for implementation research in early childhood classrooms” (p. 181).
Sutherland, K.S., McLeod, B.D., Conroy, M.A., & Cox, J.R. (2013). Measuring implementation of evidence-based programs targeting young children at risk for emotional/behavioral disorders: Conceptual issues and recommendations. Journal of Early Intervention, 35(2), 129-149.
“Abstract: This study examined whether direct, interval-by-interval measures of treatment integrity would make it possible to distinguish whether equivocal intervention results could be attributed to the intervention itself, or to poor implementation. Josh, an eight-year-old 3rd grader, performed at or slightly above his peers' academically, yet engaged in problem behaviors (yelling, throwing objects, slamming his desk into a peer's desk) on a daily basis. A functional behavioral assessment (FBA) identified these behaviors were maintained by gaining attention (positive reinforcement) and escaping from certain assignments (negative reinforcement). A function based intervention was then developed, tested, and implemented during ongoing activities in the classroom. On-task behavior occurred throughout more than 91% of the intervals when the intervention was implemented correctly, compared to only 9% when it was implemented incorrectly. Positive treatment acceptability ratings were obtained from both Josh and his teacher, even though she continued to implement inconsistently throughout the study. Implications for both research and practice are presented (p.105). … The data reported here are consistent with reviews that have emphasized the importance of reporting the degree to which interventions are implemented as intended (Gresham, Gansle, &Noell, 1993; Gresham, Gansle, Noell, Cohen, &Rosenblum, 1993; Perepletchikova & Kazdin, 2005; Peterson et al., 1982; Moncher &Prinz, 1991). Treatment integrity data make it possible to attribute observed effects to a particular intervention, rather than to extraneous variables. Without these data, one cannot distinguish weak interventions that are implemented perfectly from strong interventions that are implemented poorly. Unfortunately, most intervention researchers have a poor record of assessing and reporting implementation data. … Past research has suggested that levels of treatment integrity can be greatly improved by providing teachers with performance feedback on the accuracy of treatment implementation (Noell, Witt, Gilbertson, Ranier, &Freeland, 1997; Noell, Witt, LaFleur, Mortenson, Ranier, &LeVeIIe, 2000). Moreover, programmed consequences for teachers, including both performance feedback and negative reinforcement (escape from a meeting with a behavior analyst) have produced higher levels of treatment integrity than a single programmed consequence or no programmed consequence conditions (DiGennaro, Martens, &Kleinmann, 2006)”. (p.114).
Wood, B.K., Umbreit, J., Liaupsin, C.J, & Gresham, F.M (2007). A treatment integrity analysis of function-based intervention. Education & Treatment of Children 30(4), 105-120.
“Gresham and Kendall (1987) reviewed consultation studies published prior to 1987 and found that no study reviewed included TI data. Gresham (1989) reflected on his previous finding and concluded that most studies relied on the "consult and hope" (p. 48) approach, described as the act of consultation services occurring, but with no follow-up to ensure that teachers performed the treatment as prescribed. Gresham (1989) suggested the following reasons as to why treatments are not carried out as intended: (a) the complexity of the plan, (b) the number of treatment agents, (c) the time required to implement, (d) the resources required, and (e) perceived effectiveness or motivation of treatment agent. Many PF studies, including Duhon, Mesmer, Gregerson, and Witt (2009), Noell, Witt, Gil bertson, Ranier, and Freeland (1997), Witt, Noell, LaFleur, and Mortenson (1997), and Noell, Duhon, Gatti, and Connell (2002), have documented notable negative slopes in integrity during the baseline phase that would have gone unnoticed had follow-up not occurred. To combat threats to integrity, Gresham suggested that researchers and practitioners use direct observation coupled with the provision of behavior-specific feedback to the consultee. He also suggested self-monitoring strategies. Performance feedback is a tool to manipulate levels of TI so practitioners can conclusively state whether students received the intended intervention or support. (p. 160). … Scheeler, Ruhl, and McAfee (2004) conducted a literature review of ten studies that described using PF for in-service and preservice teachers. They concluded immediacy of PF was the only clear moderator of effect--PF should occur as soon after observation as possible. This observation falls in line with prior experimentation that has shown a negative correlation between delay of a contingency and its effect on altering behavior (Lattal, 1993; Renner, 1964). … Included studies occurred in preschools, elementary schools, and middle/high schools, as well as general education and special education classrooms, with a strong bias toward the elementary level and general education. Results demonstrated that PF was effective in preschool through high school and that grade level alone did not significantly moderate the effectiveness of the PF. This is a promising finding, demonstrating that teachers in any setting are responsive to PF” (p.166).
Solomon, B.G, Klein, S.A, & Politylo, B.C. (June 2012). The effect of performance feedback on teachers' treatment integrity: A meta-analysis of the single-case literature. School Psychology Review, 41(2), 160-176).
“The intent of all intervention efforts is to demonstrate that changes in dependent variables (e.g., student performance) is the direct result of systematic manipulation of a given independent variable (e.g., a particular intervention or treatment; Lane, Beebe-Prankenberger, Lambros, & Pierson, 2001; Wolf, 1978). In other words, is the intervention, rather than other factors, responsible for the observed changes in student performance? When designing and implementing schoolbased interventions, careful attention usually is given to the following: (a) designing the intervention, (b) training the appropriate personnel, (c) identifying the appropriate target audience, (d) selecting outcome variables, and (e) monitoring the accuracy with which the outcome data is collected. Yet, often a pivotal intervention component is forgotten, namely treatment integrity (Gresham, 1989, 1998; Yeaton & Sechrest, 1981)” (p.36) … There are a number of ways school personnel can assess treatment integrity. Methods include: (a) direct observation, (b) feedback from consultants, (c) self-monitoring, self-reporting, and behavioral interview techniques, (d) permanent products, and (e) manualized treatments and intervention scripts (Elliott & Busse, 1993; Lane & Beebe-Frankenberger, in press)” (p.37).
“What Factors Influence Treatment Integrity? There are several factors that impact treatment integrity (Gresham, 1989, 1998) including: (a) intervention complexity, (b) implementation time required, (c) materials required, (d) number of personnel involved, (e) perceived and actual effectiveness, and (f) motivation of the treatment agents (teachers). In general, as the intervention increases in terms of complexity and time requirements, the level of treatment integrity decreases. Similarly, the more materials and resources required to implement the intervention, the lower the treatment integrity, particularly when the intervention requires materials that are not typically found in the classroom setting. Furthermore, interventions that require assistance from more than one person (e.g., teacher, paraprofessional, or support staff) are less likely to be implemented with integrity, relative to those interventions requiring support from only one individual. Perceptions of potential effectiveness also influence treatment integrity. Specifically, if the person implementing the intervention (e.g., teacher) views the intervention to be potentially effective, or socially valid, he or she may be more likely to implement the intervention as originally designed than if he or she perceives the intervention to be ineffective (see Lane et al., 2001 for a discussion of the relationship between social validity and treatment integrity). Finally, teacher motivation may also influence the extent to which treatments are implemented with integrity. If the teacher is attempting to find ways of better serving a child in the general education classroom, treatment integrity ratings will probably be higher than if the teacher's goal is to have the child assessed and moved to another setting for support services (Witt & Martens, 1988; Ysseldyke, Christenson, Pianta, & Algozzine, 1983)” (p.41).
Lane, K.L., Bocian, K.M., MacMillan, D.L., & Gresham, F.M. (2004). Treatment integrity: An essential--but often forgotten--component of school-based interventions. Preventing School Failure, 48(3), 36-43.
Educational Resources, Inc. produced an “app”, entitled “Lesson Fidelity Checklists”, for Apple’s iPad, iPod, and iPhone mobile devices. The app is available from the “App Store” section of Apple’s iTunes, and comes preloaded with a Lesson Fidelity Checklist, which is based on the extensive teacher effectiveness research literature. As such, it is applicable to any program, subject area, or grade level.
“The teachers in this study who adhered more closely to the PD materials had a greater impact on student achievement than those who did not. The PD focused on evidence-based practices for vocabulary and comprehension instruction, as well as general effective instructional practices. Fidelity in our study was not exceptionally high, with an average of 4.7 on a scale of 1 to 10. The role of fidelity in the interpretation of findings and the importance of the teacher in maintaining fidelity to the treatment are critical in research (Hulleman & Cordray, 2009)” (p.254).
Hairrell, A., Rupley, W.H., Edmonds, M., Larsen, R., Simmons, D., Willson, V., Byrns, G., & Vaughn, S. (2011). Examining the impact of teacher quality on fourth-grade students' comprehension and content-area achievement. Reading & Writing Quarterly: Overcoming Learning Difficulties, 27(3), 239-260.
Hulleman, C., & Cordray, D. S. (2009). Moving from the lab to the field: The role of fidelity and achieved relative intervention strength. Journal of Research on Educational Effectiveness, 2(1), 88–110.
“Implementation refers to the process by which an intervention is put into practice. Research studies across multiple disciplines, including education, have consistently demonstrated that interventions are rarely implemented as designed and, crucially, that variability in implementation is related to variability in the achievement of expected outcomes.” (p. 635)
Lendrum, A., & Humphrey, N. (2012). The importance of studying the implementation of interventions in school settings. Oxford Review of Education, 38(5), 635-652.
“ … although existing research suggests that the average effect of CSR [comprehensive school reform] programs on student achievement is small, variability in effectiveness from CSR program to CSR program is substantial. (p. 300)… If innovative programs produce only very few differences in instruction (in comparison to normative practice), we should not expect them to produce large effects on student achievement. For these reasons, we urge researchers interested in studying innovative instructional programs to venture inside the black box not only by explicitly measuring rates of faithful program implementation but also by looking closely at the nature of instruction being implemented. Both factors are needed if we are to explain why some programs have more effects on student achievement than others.” (p. 332) … we conclude that well-defined and well-specified instructional improvement programs that are strongly supported by on-site facilitators and local leaders who demand fidelity to program designs can produce large changes in teachers' instructional practices.” (p.298)
Correnti, R., & Rowan, B. (2007). Opening up the black box: Literacy instruction in schools participating in three comprehensive school reform programs. American Educational Research Journal 44(2) 298–338.
"The teachers in this study who adhered more closely to the PD materials had a greater impact on student achievement than those who did not. The PD focused on evidence-based practices for vocabulary and comprehension instruction, as well as general effective instructional practices. Fidelity in our study was not exceptionally high, with an average of 4.7 on a scale of 1 to 10. The role of fidelity in the interpretation of findings and the importance of the teacher in maintaining fidelity to the treatment are critical in research (Hulleman & Cordray, 2009)” (p.254).
Hairrell, A., Rupley, W.H., Edmonds, M., Larsen, R., Simmons, D., Willson, V., Byrns, G., & Vaughn, S. (2011). Examining the impact of teacher quality on fourth-grade students' comprehension and content-area achievement. Reading & Writing Quarterly: Overcoming Learning Difficulties, 27(3), 239-260.
Some further fidelity references:
Blakely, M. R. (2001). A survey of levels of supervisory support and maintenance of effects reported by educators involved in Direct Instruction implementations. Journal of Direct Instruction, 1(2), 73-83.
Coulter, G. & Grossen, B. (1997). The effectiveness of in-class instructive feedback versus after-class instructive feedback for teachers learning Direct Instruction teaching behaviors. Effective School Practices, 16(4), 21-35.
Gersten, R. M., & Carnine, D. W. (1982). Measuring implementation of a structured educational model in an urban school district: An observational approach. Education Evaluation and Policy Analysis, 4(1), 67-79.
Hummel, J., Wiley, L., Huitt, W., Roesch, M. & Richardson, J. (2002). Implementing Corrective Reading: Coaching issues. Georgia Educational Research Association.
Pyle, N. (2012). The influence of fidelity of implementation on the reading outcomes of middle school students experiencing reading difficulties. Evidence-Based Communication Assessment and Intervention, 6(2), 108-112. doi: 10.1080/17489539.2012.735812
Stockard, J. (2011). Direct Instruction and first grade reading achievement: The role of technical support and time of implementation. Journal of Direct Instruction, 11 (1), 31-50.
Harn, B., Parisi, D., & Stoolmiller, M. (2013). Balancing fidelity with flexibility and fit: What do we really know about fidelity of implementation in schools? Exceptional Children, 79, 181–193.
On the benefits of early intervention: Kerry Hempenstall
There is little doubt that the failure to establish reading skills early leads to a cascading skill deficit that pervades all curriculum areas eventually. Additionally, the deleterious effects on motivation can so severe for some students as to be largely intractable. Further, the years of employing inadequate reading strategies can produce a strong resistance to the modifications of style necessary for progress. The modifications themselves tend to slow the reading rate initially, and also require markedly increased attention to graphemic detail - both of these changes can irritate students sufficiently to preclude their serious cooperation. A group of poor readers will almost inevitably contain a higher than average proportion of students with "interesting" behaviours - making teaching just that little bit more challenging. The years of little exposure to print compared with their reading-facile peers can leave these students with a vocabulary insufficient to cope with the complexity of language in secondary school texts.
There are many compelling reasons for early intervention, neatly described by Stanovich (1986) as the Matthew Effects (The Rich get Richer, and the Poor get Poorer). Despite the potential impact of early intervention on the incidence of older poor readers, we have now and will probably always have a cohort of students whose progress is jeopardised by reading difficulty. Questions arise, such as, what is the optimum focus for intervention; and can intervention be successful?
It is the understanding of the alphabetic principle that allows students to decipher novel words. Using the alphabetic principle as a cipher represents what Perfetti (1991) calls a productive process in contrast to the very limited process of memorising words. Share (1995) sees this phonological recoding process as critical to the development of skilled reading, and describes it as being "... a self-teaching mechanism, enabling the learner to acquire the detailed orthographic representations necessary for rapid, autonomous, visual word recognition" (p. 152). This point is also critically important in designing effective programs for older students. Tempting as it may be to teach whole word recognition to older struggling readers because the phonic strategies seem so ‘babyish’, one cannot bypass the ‘sounding-out’ stage. It is a necessary step on the path to automatic whole word recognition. It is only by practising these steps that ‘word pictures’ arise. An interesting study by Shankweiler, Lundquist, Dreyer, and Dickinson (1996) provides evidence for the location of the fundamental problem areas and provides an intervention focus.
Basic skills in reading and spelling and supporting metalinguistic abilities were assessed in ninth and tenth grade students in two school settings. Students attending a private high school for the learning disabled comprised one group and the other comprised low to middle range students from a public high school. Both the LD students and the regular high school students displayed deficiencies in spelling and in decoding, a factor in reading difficulty that is commonly supposed to dwindle in importance after the elementary school years. Treating the overlapping groups as a single sample, multiple regression analysis was used to investigate the contribution of non-word decoding skill and phonological and morphological awareness to spelling ability. The analysis revealed that decoding was the major component, predicting about half of the variance in spelling. The effect of phonological awareness was largely hidden by its high correlation with decoding, but was a significant predictor of spelling in its own right. Morphological awareness predicted spelling skill when the words to be spelled were morphologically complex. An additional study showed that differences in decoding and spelling ability were associated with differences in comprehension after controlling for reading experience and vocabulary. Even among experienced readers individual differences in comprehension of text reflect efficiency of phonological processing at the word level. (Shankweiler, Lundquist, Dreyer, & Dickinson, 1996. p.267).
Can intervention be successful, given the circumstances militating against such an outcome? There is not a great deal of published empirical evidence at this level. In the RMIT Psychology Clinic we have used the McGraw Hill series called Corrective Reading very successfully. For a description of the evidence supporting the use of DI programs, see Reviews supporting Direct Instruction program effectiveness at http://www.nifdi.org/resources/news/hempenstall-blog/403-reviews-supporting-direct-instruction-program-effectiveness and Jean Stockard’s DI Research Database at http://www.nifdi.org/?Itemid=294
In the Clinic, we have trained teachers, aides and parents to implement the programs, which have the advantage of being self-contained - thus there is no requirement that the person presenting the program be a reading teacher. Program fidelity is very important - in the Clinic we provide initial training, monitor the presenters during the program, employ additional daily progress checks, and ensure all mastery tests are completed. Given these caveats, the Corrective Reading program is measurably and noticeably effective in most circumstances, whether presented by teachers in groups (up to 15) or by parents or aides individually. There is no quick fix however - gains, in my experience are of the order of 18 months in the 3 months or so it optimally takes to complete 65 lessons. An 18 month gain in a Year 7 student formerly reading at Grade 3 is impressive, but insufficient to presume the student can subsequently progress unaided. The programs are sequential, so given the commitment, continues progress will occur as more advanced levels are introduced. The effects do not appear to be transient or related to novelty.
In the numerous evaluations I have completed over many years, I have noted that gains are generally maintained and progress continues while programs are in operation. In my doctoral thesis which involved providing one level of the CRP to 134 mid to upper primary school students, and comparing the outcome with 72 waitlist students, a very large effect size of 1.34 on Word Attack (Woodcock) was noted for the experimental group and an effect size of only 0.15 for the non-intervention group. A few students who continued with a subsequent level of the program achieved a similarly large effect size of 1.63 from the end of the first to the conclusion of the second level.
Refs:
Perfetti, C. A. (1991). Representations and awareness in the acquisition of reading competence. In L. Rieben, & C. A. Perfetti (Eds.), Learning to read: Basic research and its implications, pp. 33-44. NJ: Lawrence Erlbaum. Share, D. L. (1995). Phonological recoding and self-teaching: Sine qua non of reading acquisition. Cognition, 55, 151-218.
Shankweiler, D., Lundquist, E., Dreyer, L. G., & Dickinson, C. C. (1996). Reading and spelling difficulties in high school students: Causes and consequences. Reading and Writing: An Interdisciplinary Journal, 8, 267-294.
Stanovich, K. E. (1986). Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly, 21, 360-406.
From Kerry Hempenstall: Using the Corrective Reading program in the RMIT University Clinic
Can intervention be successful, given the circumstances militating against effectiveness when reading issues are addresses at a late stage? There is not a great deal of published empirical evidence at this level. In the RMIT Psychology Clinic we have used the McGraw Hill series called Corrective Reading very successfully to help develop decoding and fluency.
In the RMIT University Clinic, we have trained teachers, aides and parents to implement the programs, which have the advantage of being self-contained - thus there is no requirement that the person presenting the program be a reading teacher. Program fidelity is very important - in the Clinic we provide some initial training, monitor the presenters during the program, and ensure all mastery tests are completed. Given these caveats, the Corrective Reading program is measurably and noticeably effective in most circumstances, whether presented by teachers in groups (up to 15) or by parents or aides individually. There is no quick fix however - gains, in my experience are of the order of 18 months in the 3 months or so it optimally takes to complete 65 lessons. An 18 month gain in a Year 7 student formerly reading at Grade 3 is impressive, but insufficient to presume the student can subsequently progress unaided. The programs are sequential, so given the commitment, continues progress will occur as more advanced levels are introduced. The effects do not appear to be transient nor related to novelty.
In the numerous evaluations I have completed over many years, I have noted that gains are generally maintained and progress continues while programs are in operation. In my doctoral thesis which involved providing one level of the CRP to 134 mid to upper primary school students, and comparing the outcome with 72 waitlist students, a very large effect size of 1.34 on Word Attack (Woodcock) was noted for the experimental group and an effect size of only 0.15 for the non-intervention group. A few students who continued with a subsequent level of the program achieved a similarly large effect size of 1.63 from the end of the first to the conclusion of the second level.
Hempenstall, K. (1997). The effects on the phonological processing skills of disabled readers of participating in Direct Instruction reading programs. Australian Digital Theses Program, RMIT University Library. Retrieved from http://adt.lib.rmit.edu.au/adt/uploads/approved/adt-VIT20050628.114735/public/02whole.pdf
“Even successful schools will have some students who do not do well in reading and writing. On average, 20% of students entering high school will be reading below what we would expect students to be able to read at that age. About half of these students (10%) will be three years below average for their age in reading (Flockton & Crooks, 1997)”.
Flockton, L., & Crooks, T. (1997). Reading and speaking assessment results 1996. Wellington: Ministry of Education.
“ … the reality for many years has been that high school subject teachers have resisted teaching reading” (Vacca, 1998).
Vacca, R. T. (1998). Let’s not marginalize adolescent literacy. Journal of Adolescent and Adult Literacy, 4, 604-609.
What does research say about specific narrow focus teaching practices for poor readers in high school?
Widdowson, Moore and Dixon (1998) found that when the teacher modelled sustained silent reading in front of students, by sitting at his or her desk reading a book, student on-task reading behaviour increased, though only for average and below-average readers. This study was unable to show whether the extra on-task behaviour led to gains in reading skill, but it does suggest a possible reason to explain why there are often no improvements in reading comprehension.
Widdowson, D.A., Moore, D.W., & Dixon, R. (1999). Engaging in recreational reading. In G.B. Thompson & T. Nicholson (Eds.), Learning to read: Beyond phonics and whole language. New York: Teachers College Press.
After-school reading. Year 9 students attended an after-school reading tuition programme for two days each week, 3:30 p.m. to 5:00 p.m. Results showed that reading improved, and failure rates decreased compared to a control group that did not receive the programme.
Leslie, A.V. (1998). The effects of an after-school tutorial program on the reading and mathematics achievement, failure rate, and discipline referral rate of students in a rural middle school. Unpublished doctoral dissertation, University of Georgia.
In another study (Newton, 1990), 48 remedial readers in Year 9 were given either a pull-out programme (17 students) of basal reading instruction, or were given in-class instruction (31 students) which combined instruction in geography with instruction about reading strategies. The results showed that the pull-out programme was more effective for reading comprehension (though not vocabulary) than the in-class programme, especially for boys.
Newton, C.R. (1990). The impact of compensatory education models upon reading achievement of seventh-grade students in a Pennsylvania school district. Unpublished doctoral thesis, LeHigh University.
In another study of the effects of cross-age tutoring, high school remedial readers acted as tutors for primary school children who needed help with reading. Results showed that only the primary school students’ reading benefited.
Charnofsky, N.M. (1983). The effects of a cross-age tutoring program for teaching remedial reading to high school students. Unpublished doctoral dissertation, Brigham Young University.
In another study, sustained silent reading was used with Year 9 remedial readers. Results showed it did not make any extra difference when compared with their normal remedial instruction (Lund, 1983). On a positive note, it was no worse than the regular tuition they received. This result is consistent with a number of studies on sustained silent reading.
Lund, J.M. (1983). Sustained silent reading with junior high school remedial readers. Unpublished doctoral study, Yeshiva University.
Both Year 9 and 10 LD students and regular high school students displayed deficiencies in spelling and in decoding, a factor in reading difficulty that is commonly supposed to dwindle in importance after the elementary school years. Data analysis revealed that decoding was the major component. Differences in decoding and spelling ability were associated with differences in comprehension after controlling for reading experience and vocabulary. Even among experienced readers individual differences in comprehension of text reflect efficiency of phonological processing at the word level."
Shankweiler, D., Lundquist, E., Dreyer, L. G., & Dickinson, C. C. (1996). Reading and spelling difficulties in high school students: Causes and consequences. Reading and Writing: An Interdisciplinary Journal, 8, 267-294.
How to assess current performance and progress?
The assessment of reading attainment of older students(and adults) is governed largely by what areas have been identified as potentially problematic with this cohort. As noted above, the same areas that trouble younger students continue to be stumbling blocks for struggling older readers, along with others that follow a history of insufficiently wide reading. These include a greater attention to motivational factors, vocabulary, comprehension, and world knowledge.
“Using assessments that are similar to those used with children, literacy skills can be measured in adults. In addition, these measures are reliable and identify differences in reading proficiency in different segments of the adult literacy population”.
Fletcher, J. (2010). Construct validity of reading measures in adults with significant reading difficulties.Journal of Learning Disabilities, 43(2), 166-168.
“Among these struggling readers, most (85 %) also had weaknesses on nationally standardized measures, particularly in comprehension; however, most of these also had difficulties in decoding or fluency”.
Cirino, P.C., Romain, M., Barth, A.E., Tolar, T., Fletcher, J.M., & Vaughn, S. (2013). Reading skill components and impairments in middle school struggling readers. Reading and Writing, 26(1), 1059–1086.
Who might still be left behind?
With evidence-based instruction provided initially, the usual 20-30% of students who are the instructional casualties can be reduced to 3-10%.
“The reduction of severe decoding problems to the 3-10% range has occurred using specially trained teachers (Vellutino et al., 1996; Torgesen et al., 2001), paraprofessionals (parents with a few hours training, Jenkins, Vadasy, Firebaugh, and Prolifet, 2000), student peers (Fuchs and Fuchs, 2005), and a mixture of providers (King and Torgesen, 2000). In all cases, monitoring of the instructional fidelity of the providers (teachers, paraprofessionals, peers) was of course necessary for a study, but also critical to achieving results (for example, Jenkins et al.). In the usual school, teachers receive "training" which, when the classroom doors are closed, has little impact on actual instruction (Vaughn, Moody, and Schumm, 1998; Vaughn, Hughes, Moody, and Elbaum, 2001). Don McCabe (www.avko.org) has long said it well - he reminds us that much of what many students lack in knowledge of reading and spelling is not taught in the classroom”.
Fuchs, D., & Fuchs, L.S. (2005). Peer-assisted learning strategies: Promoting word recognition, fluency, and reading comprehension in young children." The Journal of Special Education 39(1), 34-44.
"3.35 However, as with the early intervention studies, even the most effective intervention programmes do not lead to significant reading gains for all of the participating children and depending on the reading skills measured, from 15 to 60% of older pupils with dyslexia may fail to respond [i.e., scores falling below the 30th percentile]" (p.70).
Rose, J. (2009). Identifying and teaching children and young people with dyslexia and literacy difficulties. An independent report from Sir Jim Rose to the Secretary of State for Children, Schools and Families, June 2009. Retrieved from dera.ioe.ac.uk/14790/1/00659-2009DOM-EN.pdf
“With regard to skill improvement, outcomes in three of the studies were associated with differences in the initial reading skill level of older learners (Calhoon and Petscher, 2013; Greenberg et al., 2012; Scarborough et al., 2012)” (p.492). That is the higher were the pretest scores the higher were the gains."
Calhoon, M.B., Scarborough, H.S., & Miller, B. (2013). Interventions for struggling adolescent and adult readers: instructional, learner, and situational differences. Reading and Writing, 26, 489–494.
“Characteristics of pupils who fail to respond to Wave 2 intervention Recent research has begun to investigate the characteristics of these ‘nonresponders’ to otherwise effective early reading intervention (at Wave 2), with a small number of longitudinal studies evaluating progress of pupils from Wave 1 through Wave 3 within the RTI framework. A number of the large-scale intervention studies reviewed in previous sections concur that predictors of poor response rates in their studies include weak pre-intervention levels of phonological/reading skills, problem levels of teacherrated behaviour and inattention and low SES (Hatcher et al., 2006; Torgesen et al., 1999; for reviews, see Al Otaiba & Fuchs, 2002, 2006; Duff, 2008; Nelson, Benner & Gonzalez, 2003). A small number of studies have identified co-occurring weak oral language skills as an additional predictor of response to phonological-based interventions (e.g. Vadasy, Sanders & Abbott, 2008; Whiteley et al., 2007; but see Hatcher & Hulme, 1999; Vellutino et al., 1996).
A recent meta-review of five intervention studies reported in the United States identified seven cognitive–linguistic variables related to variation in RTI, listed from strongest to weakest predictor (see Duff, 2008 for further details): slow rapid naming (RAN), problem behaviour, poor PA, limited understanding of the alphabetic principle, weak verbal memory, IQ and demographics. Environmental factors influencing RTI potentially include quality of Wave 1 teaching, point of intervention (early or late, where ‘late’ is defined as after KS 1 in England or G2 in the United States) and programme fidelity. The careful training, implementation, supervision and monitoring which characterises research studies may not always be observed in other circumstances with detrimental effects on the outcome of the intervention (Byrne & Fielding-Barnsley, 1995; Byrne et al., 2010; see Carter & Wheldall, 2008 for further discussion of this issue). Programme content may also influence outcome when the evidence base for inclusion of that content is weak or the content and/or implementation is inappropriate for the individual’s profile of needs, due to insufficient assessment and monitoring” (p.105).
Griffiths, Y., & Stuart, M. (2013). Reviewing evidence-based practice for pupils with dyslexia and literacy difficulties. Journal of Research in Reading, 36(1), 96-116.
“Calhoon and Petscher found compelling indications that the level of improvement by their adolescent sample and the percentages of students classified as gainers, were influenced by the way that elements of how a common curriculum were organized and sequenced during instruction” (p.492).
Calhoon, M.B., Scarborough, H.S., & Miller, B. (2013). Interventions for struggling adolescent and adult readers: instructional, learner, and situational differences. Reading and Writing, 26, 489–494.
“Remediation of reading difficulties in older students may require considerable intensity and differentiation of instruction. A significant problem is that intensive, small-group instruction provided by highly skilled teachers is an expensive and infrequently applied instructional practice within most educational settings (Vaughn, Levy, Coleman, & Bos, 2002; Vaughn, Moody, & Schumm, 1998). Therefore, it is perhaps not surprising that the few available studies of students who receive special education services show fair levels of growth and little evidence that interventions through special education actually close the achievement gap (Bentum & Aaron, 2003; Foorman et al., 1997; Hanushek, Kain, & Rivkin, 1998; Torgesen et al., 2001) (p.74)”.
Wanzek, J., Vaughn, S., Roberts, G., & Fletcher, J.M. (2011). Efficacy of a reading intervention for middle school students with learning disabilities. Exceptional Children, 78(1), 73-87.
“Despite advances in the science of teaching reading, there still exists a small percentage of students who fail to make the expected progress in reading-related skills, notwithstanding attempts at intervention. Even if these struggling readers learn to decode adequately, fluency remains a problem for many, and little is known about the effectiveness of fluency interventions for older students with severe reading deficits. This study used a randomized experimental design to test the efficacy of a fluency intervention program on the word-identification and reading-comprehension outcomes of 60 middle-school students with severe reading delays. Results showed that students in the experimental group made more progress on standardized tests of reading fluency than students in the control group. No gains were seen in reading comprehension” (p.76).
Spencer, S.A., & Manis, F.R. (2010). The effects of a fluency intervention program on the fluency and comprehension outcomes of middle-school students with severe reading deficits. Learning Disabilities Research & Practice, 25(2), 76–86.
“The rates of responsiveness are less positive than those following early intervention, ranging from 15% to 60% of pupils in any sample of dyslexics (depending on reading outcome measures) unable to make significant, longlasting gains, when assessed up to 2 years following the end of the intervention (Torgesen, 2000, 2005). The gains in word reading fluency from phonologically based interventions are typically weaker for older pupils than for younger pupils receiving early intervention” (p.100).
Griffiths, Y., & Stuart, M. (2013). Reviewing evidence-based practice for pupils with dyslexia and literacy difficulties. Journal of Research in Reading, 36(1), 96-116.
“In summary, this study suggests that regardless of the variations in measures of the same construct across studies, variations in how RTI was implemented in terms of curriculum, length of intervention session, and how responders and low responders are defined, a clear pattern emerged suggesting that low responders can be identified prior to intervention. The key measures that play an important role in predicting posttest outcomes are related to initial level of real word reading, word attack, passage comprehension and rapid naming speed” (p.293).
Tran, L., Sanchez, T., Arellano, B., & Swanson, H.L. (2011). A meta-analysis of the RTI literature for children at risk for reading disabilities. Journal of Learning Disabilities, 44(3), 283-295.
“Effect sizes for all outcomes except oral reading fluency met criteria for substantive importance; however, many of the students in the intervention continued to struggle. An evaluation of cognitive profiles of adequate and inadequate responders was consistent with a continuum of severity (as opposed to qualitative differences), showing greater language and reading impairment prior to the intervention in students who were inadequate responders (p.1). … the results are consistent with prior studies of the cognitive attributes of Tier 2 adequate and inadequate responders, suggesting a continuum of severity corresponding with the level of reading ability at baseline. These results show little evidence of qualitative differences that might suggest differences in the type of intervention or alternative approaches to intervention other than a more intense focus on oral language development” (p.12).
Denton, C. A., Tolar, T. D., Fletcher, J. M., Barth, A. E., Vaughn, S., & Francis, D. J. (2013). Effects of Tier 3 Intervention for students with persistent reading difficulties and characteristics of inadequate responders. Journal of Educational Psychology, 105(1), 1-16.
“Across studies, the generalized findings are that Matthew effects are present in LD and that disadvantaged students continue to be at a great disadvantage in the future. This finding was evident particularly with regard to the relationship between vocabulary and reading comprehension (Oakhill & Cain; Sideridis et al.) as well as with regard to other reading skills such as phonological awareness (McNamara et al.) or math abilities (e.g., Morgan et al.; Niemi et al.). When looking at the framework of responsiveness to instruction implemented in the United States and various parts of theworld, the message from the present studies is clear: Students with LD are likely to be classified as nonresponders as their trajectories of growth suggest. We need to switch our attention from assessing the difficulties of students with LD to how to intervene to solve their problems” (p.401).
Sideridis, G.D. (2011).Exploring the presence of Matthew Effects in learning disabilities. Journal of Learning Disabilities, 44(4),399-401.
Further thoughts on catch-up
In considering the provision of additional instruction to older struggling students, catch up is a laudable goal. However, the longer a student has been falling behind, the less likely a school can provide the intensity of instruction required to even make headway against his average achieving peers. So, the task of elevating a year 1 student to the average range in, say, reading is much less demanding than attempting to do so for a third grade student. Beyond that, the difficulty does not increase in a simple linear progression, but much more severely.
The process has some of the characteristics of compound debt, a situation that arises when a debt cannot be fully serviced, and so the capital upon which the interest rate is calculated is rising rather than falling. Thus, a student who makes less than average progress in his first year accumulates a debt. If, through intervention, he makes average progress in his second year, he will remain behind because he is moving from a lower base than his peers. However, it is more likely that his progress in second year (assuming no intervention) will also fall below the average rate, and he is now even further behind because he has been unable to fully service his debt. His skill-capital, upon which progress depends in his third year, has now fallen further behind. He is beginning to experience an inability to make sense of average classroom instruction because of this deficit. He will lose touch with the curriculum, no matter how hard he tries. In a double whammy, it is likely that his resilience has been sorely tested over the preceding years, and now his efforts are diminishing – and his progress slows even further. Meanwhile, his peers have built upon their skill-capital, classroom instruction is pitched at a level they can understand. Further, their sense of self-efficacy has been enhanced by success, and they provide the effort to achieve. During these three years they have been engaged in lots of reading, and their vocabulary and knowledge store have kept pace with grade level expectations. The struggler has been able to gain little from text, his vocabulary and knowledge store have not kept pace, and his capacity to follow the language of classroom instruction is impaired.
The longer this process occurs, the greater the compounded loss and the more difficult is effective intervention. So, by the times our student has reached high school, the Matthew Effect (see http://www.nifdi.org/resources/news/hempenstall-blog/399-what-are-these-matthew-effects) have made life very difficult for him.
As indicated earlier, few secondary schools have the will, the knowledge, or the resources needed to provide anything like catch up for these students. Research has yet to provide more than ballpark estimations as to what might be required to create even a measurable acceleration in literacy skills, such as reading (decoding fluency and comprehension), vocabulary, writing, spelling, mathematics – without even considering the resulting loss of time taken from participation in the normal secondary curriculum.
See below:
Late-stage intervention recipe
Below are some extracts from a litigation report on a ninth grade student whose parents were suing the state for failing to teach their child adequately. It is included to indicate the difficulty of attempting to redress a long-standing failure with late-stage intervention.
Recommendations from the Clinic report on the student:
- As suggested from XXX’s assessment results, it is likely that XXX would benefit from a highly structured literacy program tailored to his various areas of difficulty. Recognising that there are several literacy areas of need, the question arises as to how best to intervene. The research into students with a variety of problems in making progress suggests that all students are capable of learning if the learning environment is sufficiently supportive. It appears from the cognitive assessment that XXX does not have a strong innate capacity to manage his own learning; thus, he requires a highly structured environment in which every component sub-skill of a valued curriculum outcome is presented systematically, and practised assiduously to mastery and beyond, in order to obtain retention.
- Research also suggests that for children with delays, piecemeal approaches have insufficient intensity to make a significant difference. These are strategies that are either too shallow, or of insufficient frequency, to alter XXX’s current learning trajectory. Therefore, it is recommended that XXX’s educational program be focussed acutely on the skills involved in literacy. The approach needs to be multi-pronged without being shallow. Given that XXX is considerably behind in his academic development, in order to facilitate progress; it is recommended that XXX be engaged in intensive direct instruction programs for approximately 3 hours per week day.
- It is suggested that the deficits in phonological processing are addressed first as these skills are central to many educational tasks. It is recommended that XXX participate in a beginning reading program, such as, ‘Teach Your Child to Read in 100 Easy Lessons’. Five lessons per week are considered essential in order to consolidate phonological skills. Following the completion of a beginning reading program, it is recommended that XXX continue with an intensive direct instruction reading program such as the Corrective Reading Programs: Decoding and Comprehension.
- XXX’s writing and spelling abilities will also need to be addressed. Programs used to support the development of these skills can simultaneously support his reading progress. Programs for this area, such as Reasoning and Writing, and Spelling Mastery are available in the RMIT Clinic.
- Ideally, this exceptionally intensive intervention would be achieved through negotiation between home and school in order to spread the load. In some cases, parents have provided program(s) at home, and in other situations they have liaised with school staff to implement their program(s) in the school setting through daily withdrawal from class. Sometimes, teacher aides or school volunteers have accepted the challenge of addressing a child’s literacy requirements. Training can be provided to parents, volunteers, and teachers to implement these programs - which can be delivered in either an individual or group format.
Timetable for the recommended intervention?
Given that these are secondary students - older than most who undertake them, it may be that lessons take longer than the manual suggests. Issues such as student disheartenment, inattention, vocabulary deficits, misbehaviour, lesson transition time, and also teacher/tutor program competence can each add to lesson length. This recommended timetable would take up most of a morning’s classes, or about 3 hours.
Following Teach Your Child to Read in 100 Easy Lessons, Corrective Reading: Decoding A, B, and C should be implemented. The complete Corrective Reading: Decoding program series has a total of 315 lessons. This should require about 18-24 months of instruction. The lessons take about 45 minutes and occur at least daily.
The Corrective Reading: Comprehension program (Levels A and B, 185 lessons) is designed to be presented five times per week, taking approximately 30 minutes per session daily. This should require about 12 months of instruction.
At the same time, the Spelling Mastery Levels A, B, and C (total 300 lessons) program would address XXX’s spelling difficulties, and simultaneously supporting his reading progress. This should require about 18-24 months of daily 30 minute instruction. The training, presentation and follow-up for this program are similar to that for the Corrective Reading programs.
Additionally, Reasoning and Writing (Levels A-F) program (510 lessons) assists students to construct grammatically correct sentences that are expressive of the ideas that they develop. Eventually, the focus is on higher-order thinking skills. Lessons are about 45 min daily, total about 2.5 years.
Questions asked in court:
Why have you recommended the Direct Instruction programs in your report?
- There are two areas of justification for an educational/remedial program.
- The first is to consider whether the program is theoretically relevant: to investigate whether it is consistent with the best available knowledge about the development of literacy and literacy instruction.
- The second area is to examine whether such a program has evidence of effectiveness in the real world. The best programs are trialled and field tested before publication providing evidence of their likelihood of success. They are often tested on thousands of children, feedback gained, and modifications made based on this feedback. Programs should then have been independently evaluated in various settings and with various types of students. The direct instruction programs that we recommend meet these two criteria.
Why do these programs need to be undertaken for approximately three hours every day?
- Self learners are able to pick up new concepts very quickly, and continue self directed learning without requiring much assistance. Young people who struggle due to disabilities or other difficulties do not pick things up readily. Simply explaining something to them will not result in their understanding. You may teach one concept on a given day, and repeat it perhaps 5, 10 or 50 times, you may at the end of the day have successfully taught that concept to a child. However, two days later, it can appear as though this concept was never taught. This is why the programs are required to be taught every day.
- Through sufficiently intense and focussed teaching every day, the learning becomes entrenched and incorporated in long term memory, which is not easy for children who have learning difficulties.
- Most importantly, these programs are often recommended for children who are behind their peers. The objective of these programs is to provide for students an opportunity to make headway towards, or to even catch up to, the average level of their peers in the given skill. To do this, the children must learn faster than their peers, as previously they had learned more slowly.
- In a classroom, the child who is behind has to learn much more quickly, as his/her peers continue to learn every day. Not only does the child who is behind have to try and access the current curriculum, but also have to learn information and skills that they have missed out on in previous years. For this reason they must be taught more intensively and cover more content if they are ever to catch up.
- The programs need to be presented each day, and another reason for the three hour recommendation concerns the number of problem areas experienced by XXX, who is many years behind his peers. There is no single omnibus program that resolves all educational issues. XXX has significant delays in phonological skills, decoding, reading fluency, reading comprehension, and written expression/spelling. Each of these is best addressed with its own dedicated program.
What are the advantages and disadvantages to withdrawing a student for three hours per day from the normal curriculum to undertake these programs?
- When a child needs to undertake remedial programs, the reality is that they will have to miss out on some important components of the curriculum. A child will not participate some other subjects which, while less crucial to them in their situation, do have benefit. A withdrawal from the regular curriculum may also be seen as disruptive either by teacher or child.
- However for young people who are years behind their peers in literacy, even though they may be attending a full curriculum, they are disengaged from it because literacy is the access key to the curriculum. Such students are not able to make sense of the content of their everyday classes. When students are in late primary or in secondary school, there is a limited amount of time available for them to gain an acceptable education skill level sufficient for them to participate meaningfully in society. The issue is what does it take for them to make headway against their peers? What then are the most important uses of the time available in their subsequent education?
- As literacy is the entree to all academic classes, this is where considerable time should be spent.
There are positives and negatives for a child spending such time on remedial programs, such as the issue of being withdrawn away from classes with his/her peers. However, my discussions with teachers support the view that such students are viewed as an “embarrassment” in any event – because there is too big a gap between what they know or can do, and what the curriculum in their classroom demands of them. Teachers often simply want to give the students something to do that is consistent with their ability, such as easy but trivial worksheets, and hope to “contain” the students in order that they not disrupt the rest of the class. There is little learning occurring in regular classes for a young person who is significantly behind. When that is realised, the decision about regularly withdrawing them from the normal curriculum becomes more straightforward. Of course, if XXX’s difficulties had been recognised earlier in his career, and if intensive assistance had been provided, these later large scale interventions may have been avoided. When programs commence at the level of the student’s current attainment, and proceed from there in small, carefully graded steps, then high rates of success usually occur. This success tends to produce a changed attitude to one of “I can do it!” The programs recommended display these qualities, and each has a placement test to ensure that the student is presented with the correct level of the program.
Why do you recommend these programs for XXX?
- XXX is significantly behind his peers, and probably will require intensive assistance for the rest of secondary education. It is unlikely that he will be able to catch up on the learning he has missed in the time left ahead of him at school. I am advised he is at the end of Year 9 this year. If one assumes he will not have the skills and knowledge to complete VCE (12th year), it may be that he will leave school as soon as he is able, due to his long exposure to failure and his resultant disengagement from school life. If he stayed another three years, he would need to progress by two years of attainment for each one year of attendance at school to end up at the literacy level of his peers. This would involve a learning rate of twice that of his peers. I would not anticipate he could achieve such remarkable gains in the time remaining in his school education.
- At paragraph 9, Mr. ZZZ states that his school already provided the Corrective Reading Program to XXX with a group of students about two or three afternoons per week. Please provide us with your comments in relation to how effective this would have been for XXX.
- The effectiveness of the Corrective Reading Program is heavily dependent upon the completion of a lesson every day. This is particularly important for children who are struggling, to prevent them from forgetting what was learned in a given lesson, and to provide sufficient intensity of assistance for accelerated progress to occur. The provision of the Corrective Reading Program upon anything other than a daily basis is contrary to the protocols of the program and jeopardises, and in many cases precludes, success. This direct instruction program has had demonstrable success in many settings, with the proviso that the protocols are faithfully adhered to.
- It is not appropriate for individual teachers to simply use the program as they see fit. Professionals in other areas recognise that a manualised treatment has superior outcomes to "in the moment" decision making.This is because the evaluation of evidence-based programs incorporates careful adherence to the program’s routines, in order to ascertain which elements, when combined, re likely to produce strong beneficial effects. When these elements are altered or omitted, the same strong outcomes should not be expected.
- Finally, it appears that a placement test was not provided prior to the student being placed in the CRP. The Clinic assessment indicated that Level A Decoding is currently above XXX’s reading attainment level, and a beginning reading program is the appropriate initial intervention.
