Project Follow-Through: A lesson in cultivating orchids, not dandelions.

  

Image: ChatGPT

Regardless of what people have to say about the methodological imperfections of Project Follow-Through (PFT; see also here and here), it was, and remains, the largest education experiment ever conducted. Yes, it had its resource wings clipped by a redirection of funding as a result of America’s involvement in the Vietnam War, and no, it was not a randomised controlled trial (RCT). While RCTs are the most rigorous research design for establishing cause and effect (see here and here) they are not always feasible for a range of reasons. PFT is probably best described then, as a large-scale, longitudinal quasi-experimental evaluation of multiple educational models using non-equivalent comparison groups. That, by the way, makes it considerably higher-up the evidence hierarchy than many of the education research studies conducted since. Stockard et al. (2018, p. 482) observed of the PFT analysis that:

"External evaluators gathered and analyzed outcome data using a variety of comparison groups and analysis techniques. The final results indicated that DI was the only intervention that had significantly positive impacts on all of the outcome measures (Adams, 1996; Barbash, 2012; Bereiter & Kurland, 1996; Engelmann 2007; Engelmann, Becker, Carnine, & Gersten, 1988; Kennedy, 1978)". 

Notably, however, the (re-)analysis and dissemination process after PFT was heavily contaminated by partisan naysayers, such as Dr Gene Glass, who is heard in Episode 3 of SaS, Season 2 saying:

I grant them nothing. Nothing. I find no evidence in the entire Follow Through evaluation that they have anything special or worthy of wide distribution. It's a terrible way to organize teaching and learning. The best education starts with the interests, the motivations, the curiosity of the children. Sit down with the child, find out what the child is interested in. Let the child bring his own book, the book he loves. You know, see if he can read, see if he understands.

There’s (at least) two obvious problems with Glass’ position: its emphatic and complete rejection of Direct Instruction is glaringly at odds with the findings of volumes of research in the intervening decades, and it declares the kind of cognitive bias that undergraduate students in research methods 101 classes are taught to identify at 20 paces.

Consider, for example, the findings reported by Stockard and her colleagues in their 2018 meta-analysis of 50 years of research on Direct Instruction:

"The findings of this meta-analysis reinforce the conclusions of earlier meta analyses and reviews of the literature regarding DI. Yet, despite the very large body of research supporting its effectiveness, DI has not been widely embraced or implemented. In part this avoidance of DI may be fueled by the current popularity of constructivism and misconceptions of the theory that underlies DI. As explained in the first part of this article, DI shares with constructivism the important basic understanding that students interpret and make sense of information with which they are presented. The difference lies in the nature of the information given to students, with DI theorists stressing the importance of very carefully choosing and structuring examples so they are as clear and unambiguous as possible. Without such clarity students will waste valuable time and, even worse, potentially reach faulty conclusions that harm future progress and learning" (p. 502).

Something that has always intrigued (but not overly surprised) me, is the fact that so few practising teachers are aware of PFT and virtually none of them learnt about it in their pre-service teacher education. Of course, the naysayers can chime in here with any and all versions of “pre-service education can’t cover everything”; “it wasn’t a properly conducted experiment”; “it happened decades ago, so is irrelevant now” and countless other thinly-disguised versions of “move on, nothing to see here”.  Notwithstanding its significant budget cuts and methodological limitations, PFT was for its time, and remains today, a significant educational research study, from which interested policy makers, education academics, school leaders, and classroom teachers can learn a great deal. This learning applies not only to students’ academic achievement, but also to their wellbeing: outcomes we are still unfortunately prone to disconnect from each other. As noted by Watkins three decades ago (1997, p. 58), however,

"The Follow Through evaluation provides clear evidence that no such trade off is necessary. The Direct Instruction and Behavior Analysis models not only resulted in superior performance on achievement measures, they were ranked first and second, respectively, on measures of affective development. It appears that the way to improve the self concept of children is to provide them with successful experiences by programming instruction in such a way that the child can see evidence that he or she is learning".

I was especially pleased, therefore, when I learned that Season 2 of Emily Hanford’s Sold a Story (SaS) podcast does a deep-dive into PFT, building on her highly impactful and influential investigation into reading instruction, in Season 1. This is no small undertaking, given the six decades that have elapsed since PFT commenced in the mid-1960s and the mix of methodological and analytical obfuscation, goal-post shifting, and zeitgeist overshadowing that has occurred in recent decades.

I imagine that most readers of this blog will be following Season 2 of SaS and will have had a similar visceral reaction to mine, on listening to Episode 4, What can you do for my black children? I am not American, but the achievement gradient that is aligned to race in that country is familiar to me and also bears out, albeit in different ways, in my own country, Australia.

Episode 4 tells the simultaneously uplifting and demoralising story of what happened when a man named Thaddeus Lott became the principal of Wesley Elementary, in a poor, black neighbourhood of Austin Texas. Essentially, Thaddeus led a natural experiment. In the words of Emily Hanford, “…..he defied school district officials. He got rid of whole language. And replaced it with Direct Instruction”.

Put simply, this means Thaddeus abandoned romantic and ill-founded ideas that students can “absorb” reading and writing by being in text-rich environments. By adopting Direct Instruction, he showed that not only could poor black children succeed academically, but that they could outperform white counterparts in more affluent neighbourhoods. As those of you who have listened to the episode will know, this prompted a mix of disbelief and outrage among local officials, such that Thaddeus and his staff were accused of facilitating “cheating” by their students. The underlying assumption of course was that poor black children, with their poor black brains, should be expected, as a reflection of world-order, to under-perform relative to white peers.

Thaddeus was ahead of his time in so many ways, including his pushback against what President George W. Bush Jnr later referred to as “the soft bigotry of low expectations”. Now that we’ve heard the story of Wesley Elementary, it’s perhaps time for us to call this mindset what it really is: the hard bigotry of low expectations.

The educational attainment of children in disadvantaged communities was of course, one of the driving forces behind PFT. It is also where we need to navigate the often-murky discourse of the socio-political waters in which education policy is framed. The fact that “poor children perform poorly at school” is simultaneously accepted as an article of faith by many in education and met with a remarkable lack of so-called twenty-first century skills on the part of education leaders. You know the ones I’m referring to, I’m sure – critical thinking, creativity, collaboration, and communication. Even if we only got as far as (genuine) critical thinking, we would hopefully have a conversation about the fact that the single most important influence on children’s academic achievement should be the type of instruction they are exposed to at school. Assuming adequate attendance (which granted, can be challenging in some contexts), instructional experiences should be conceptualised and designed to counter the known effects of the home language and literacy environment. Indeed, this is what Thaddeus Lott demonstrated in the 1970s at Wesley Elementary. In an ironic twist, a few years after his departure, the cessation of his instructional model and reversal of student achievement was an inadvertent A-B-A natural experiment.

For reasons I find unfathomable, it remains unpalatable, however, to many in education, that instruction is the first rock we should be turning over when students’ academic progress falls short of expected levels. Many in education will talk endlessly about poverty and disadvantage, expressing grand but only vaguely-articulated ideas about governments needing to “fix" these as a prerequisite to improving the academic performance of poor students. (They do so, of course, from their own position of enormous privilege, having completed secondary school, a university degree, and a postgraduate qualification such as a PhD). This is as good as admitting the natural order of instruction not being the primary driver of student achievement. It is true that instruction does play second fiddle to parental income and education level, but I find it perplexing that so few education academics and influencers are actively campaigning to reverse this.

What would it mean for education academics to accept the evidence that low-variance direct instruction / Direct Instruction / explicit instruction / explicit direct instruction is more effective at scale than more eclectic, variable, project / inquiry-based models of instruction?

It would mean that education would divest itself of its self-proclaimed but misappropriated “special standing” as a human service and instead, take its place alongside other human service fields, such as medicine, nursing, psychology, criminology, and many more, that work with multiple, complex and unpredictable variables, in multiple, complex and adaptable ways, and are accountable for the outcomes of their protocols and practices.

Developmental psychologists Thomas Boyce and Bruce Ellis proposed in 2005 that children’s sensitivity to context places them on a notional continuum from “orchids” to “dandelions”. Dandelions are hardy and survive in a range of environments and climates. Orchids, on the other hand, are relatively easy to keep alive, but require quite specific and tailored care in order to flower. Thaddeus Lott treated the children in his school like orchids. He didn’t think it was enough for them to survive. Under his considered and deliberate care, they thrived, but when the careful and considered care ceased, so too, did the blooming of the children at Wesley Elementary.

Thaddeus Lott’s natural experiment was not enough to disturb the complacent slumber of education leaders in his state and shift the instructional needle. Stockard et al.’s 2018 meta-analysis of 50 years’ research on Direct Instruction was also met with indifference and an averted gaze from education academics.

Let’s hope that the “outing” of Project Follow Through via Sold a Story Season 2 stimulates vigorous discussion among teachers and school leaders about ways in which they can shift their pedagogical models closer to the explicit / direct instruction end of the continuum when working with novices. Education academics need to be on notice, because teachers and school leaders are now “in the know” about the evidence on what works at scale.

Arguing the toss about the 1970s analysis of Project Follow Through is a minor distraction in 2026. There’s the more awkward matter, now, of an additional 50-plus years of affirming evidence for Direct Instruction's effectiveness to wrangle. 

© Pamela Snow (2026) 

What are “levels of evidence” and why do we need to talk about them in education?

  

Image created using ChatGPT

Around ten years ago, when my colleague, Dr Caroline Bowen AM and I were writing a book on evidenced-basedsupport for children and young people with additional needs I found myself going down all kinds of rabbit holes to examine what were sometimes extraordinary claims about equally extraordinary practices. Examples of such practices were cross-body physical exercises to improve reading and writing skills, using coloured lenses and overlays to improve reading skills, “Earducators” to provide auditory-integration training to children diagnosed with made-up disorders, psychoastrology for children with autism spectrum disorders, Chelation therapy, and neurolinguistic programming – to name a very small sample.

In early 2026, we released a “companion” text that is aimed more fairly and squarely at a parent readerhsip, to support them in their increasingly complex journey through the noisy, invariably expensive and often-alluring intervention marketplace.

No-one should be surprised that the “average parent” struggles to access, appraise and distil research evidence but it is disappointing that many teachers and allied health professionals find this daunting also – invariably because of a paucity of preparation for this in their pre-service education.

Regardless of their lack of preparation for the task, parents, teachers and clinicians* all have vested interests, however, in being able to answer questions such as:

• What are the hallmarks of effective interventions to support struggling early readers?
• Does the timing of speech-language pathology intervention matter for young children whose speech is dysfluent?
• Are children with pre-school language disorders at higher risk of later reading and writing difficulties?
• Do coloured lenses/overlays improve the reading of children who have a diagnosis of dyslexia?
• If I read to my toddler, will that make them a strong reader when they attend school?
• Should phonemic awareness be taught in isolation in early years classrooms, or should printed letters be present?
• Can children with an intellectual disability be taught to read? 
• Will a gluten-free diet lessen a child’s autism symptoms?

*Policy-makers also.

This is where the “fun” starts.

I’m reminded of an email exchange some years back with an education academic at a major Australian university who was promoting so-called “brain-based interventions” for children with special needs. The term “brain-based” struck me then (and still does) as odd, given that all learning (declarative and/or procedural) ultimately comes back to the brain and its complex inter-connected systems. It seems a little like what my late dad would have described as “selling ice-creams to Eskimos”.  When I wrote to the academic in question and asked for published evidence to support said “brain-based interventions” their reply rather glibly informed me that “you can find a journal article to support just about anything” – as if this was a good thing.

While it is true that peer-reviewed journals publish material of wide-ranging academic merit, this is a problem – for the scientific community and for parents, teachers and clinicians (not to mention policy-makers and taxpayers) who are the consumers of said research. It is not a loop-hole that should be exploited by those who want to argue some version of:

• If there’s a peer-reviewed journal article showing an association between two factors, those factors must be linked in some important way;
• If there’s a peer-reviewed journal article showing lack of an association between two factors, those factors must not be linked;
• If there’s a peer-reviewed journal article that reports positively on an intervention, the intervention is trustworthy;
• If there’s a peer-reviewed article saying parents/teachers/clinicians liked a particular assessment or therapy, it should be adopted;
• If there’s a peer-reviewed journal article saying parents/teachers/clinicians disliked a particular assessment or therapy, it should be discarded;
• All research methodologies are equally valid for answering all research questions;
• We should not appraise individual papers or bodies of evidence to assess the weight they can add to the decision-making scales.

One paradox we need to address right here is the fact that peer-review of new research is both (a) well-intentioned and preferable and (b) imperfect.

Peer review is well-intentioned and preferable because

• It means that manuscripts are critiqued, often in a “double-blind” way, by academic peers who can identify flaws and gaps in arguments and analyses and suggest ways in which the paper can be strengthened before it is published.
• It helps to uphold academic standards regarding coverage of prior literature, application of ethical principles to recruitment and data collection, selection and justification of research methodology, data analysis, and appropriate extrapolation without over-reach.
• It sets published research apart from other written sources, such as newspaper and magazine articles, blogposts and newsletters, and product websites.

Peer review is imperfect because

• Some journals are inundated with submissions and can be highly selective, based on rigorous criteria, about what even goes out for review, let alone what is ultimately accepted for publication. Others have editors who are sitting by the phone, waiting for copy, and necessarily need to lower the bar to meet publication schedules.
• Sometimes editors can locate reviewers (who do their work without payment, often in their own time) with the necessary content knowledge and research background, and sometimes they can’t. Since the COVID-19 pandemic, this seems to have become more challenging than ever.
• Two reviewers may arrive at markedly different assessments of a paper, and the editor needs to exercise judgement on whose review takes precedence and/or how tensions are resolved. There is inevitably some cognitive bias and human error in this process.  

So - what does it mean to ask about the “level of evidence” for an assertion or a practice?

We can ask about the level of evidence behind an assertion, for example, “Vitamin D plays an important role in bone density” or about the level of evidence that can address a question about a practice, for example “Does a daily Vitamin D supplement improve bone density”? Corollaries in education might be “Knowledge of the alphabetic principle is important for beginning readers” and “How do we best establish knowledge of the alphabetic principle in beginning readers?”.

Astute readers will already be seeing the questions within questions in these examples.

• What do we mean by “important role”?
• What daily dosage of Vitamin D are we referring to? Which population(s) are we referring to? Children? Adolescents? Adult males? Adult females? Pre-menopausal females? Post-menopausal females?  People with particular medical conditions?
• What do we mean by the “alphabetic principle”?
• How does the alphabetic principle apply in a morphophonemic language like English? 
• What do we mean by children’s knowledge of the alphabetic principle? Which children? At what age/year level? How is this determined?

These questions within questions are often what leads academics, when asked about evidence to rightly respond by beginning with “Well it depends”. There’s not always a cut-and-dried answer for people schooled in the rigours of research critical appraisal and design. Yet we still need to respond to questions about whether current evidence supports a particular proposition or practice."It depends" is not code for "choose your own adventure". 

So, we must engage with the idea of levels of evidence.

This is a principle borrowed from health (medicine, nursing, allied health, public health), where it has gained wide acceptance in the last thirty years to guide policymakers and practitioners who must sieve through vast amounts of peer-reviewed (and increasingly, non-peer-reviewed) sources to guide their decision-making and optimise outcomes for their end-users.

Some readers may take exception right here, to me applying a framework derived from health to education. A gentle reminder is warranted then, that the idea of levels of evidence is equally applied in many other fields such as psychology, criminology and criminal justice, economics, aviation, engineering, road safety, environmental science and conservation, and forensic science – to name a few. As I have noted in a previous blog-post about education’s relationship with evidence, education is not special.

The evidence hierarchy is simply a means of organising, filtering, and appraising published evidence. It does not mean that research rigour only matters in experimental studies. Rigour should be maximised in all paradigms and across all study designs, to strengthen the trustworthiness of the data. 

The hierarchy of evidence does not say that one research paradigm (e.g., positivism Vs autoethnography) is inherently superior or inferior to another. You will see in the infographic above, that both quantitative and qualitative data sources can be used right across the hierarchy. Choice of research methodology should align with the study aims/questions posed. Many education researchers, me included, use mixed methods to address their research questions. This might mean (for example) collecting student data on standardised measures as well as carrying out in-depth qualitative interviews with teachers to understand their experiences of a new teaching approach.

You will note too, that the hierarchy is a "stack" that tapers towards the top, because higher-level evidence is more difficult to collect. RCTs, for example, are usually expensive, so there will be fewer of them, meaning that there will be fewer to draw on for systematic analyses. This applies in all fields, not just in education. Evidence at lower levels is often where new knowledge begins, as occurred in the early 1980s, with the appearance of case reports heralding the arrival of the illness we now know as AIDS. As fields advance, though, new hurdles emerge. We would not want to rely on case-reports alone to answer vital questions about effective treatments at scale. For those questions, we need studies with higher levels of experimental rigour built in. 

Being open to the idea of levels of evidence helps to guard against the many ways in which cognitive bias can influence the thinking of researchers, policy-makers and/or practitioners, as described by the late Daniel Kahnemann, in his 2012 text Thinking Fast and Slow. The levels of evidence framework also offers pushback against the "fads and fashions" state of play that education has normalised in recent decades.

There are many myths and misconceptions about research methodologies and levels of evidence in education, and I blogged about these earlier this year: The dog ate my homework: Education’s evidence-excuses echo-chamber. The ideas expressed in that blogpost were uncomfortable for some, particularly my reminder that education is no different from any other social science that examines the behaviour, attitudes, and beliefs of humans. Ultimately, we all need to be able to apply rigour in the laboratory of the real world, if we are to make meaningful advances for our end-users.

Evolving evidence needs to be monitored, weighed probabilistically, and considered alongside available resources and the reasoned judgement of professionals. It is not a black-box but nor is it a means of arguing, as did my fellow education academic some years ago, that “anything goes as long as I can find a published paper somewhere that vaguely suggests this might be OK”.

When education academics and policy-makers attend a medical practitioner, drive across a bridge, or buckle up in an aeroplane, they are privileged beneficiaries of the respect shown for the hierarchy of evidence in those disciplines.

Education - you owe your stakeholders the same respect.

© Pamela Snow (2026)