Sunday, 7 June 2026

The dog ate my homework: Education’s evidence echo-chamber

 


 Image source: ChatGPT

I am not the first to write about the fact that education has a fraught and not very pretty history when it comes to generating, critiquing, and applying rigorous evidence in order to maximise student academic and wellbeing outcomes. Recent examples of this commentary can be found here and here. Way back in 1996, David Hargreaves observed in a Teacher Training Agency Annual Lecture in London, that

“Teaching is not at present a research-based profession. I have no doubt that if it were, teaching would be more effective and more satisfying”.

In 2000, Dr Louisa Moats commented that

“Unfortunately, lack of rigor and respect for evidence in reading education are reinforced by the passivity of education leaders who feel that any idea that can muster a vigorous advocate is legitimate and deserves to be aired”.

Dr Moats was discussing reading instruction, but her words are no less applicable to other curriculum areas, most notably mathematics instruction.

Of course, there’s a bias from me right there in my opening paragraph – the idea that education exists to maximise student outcomes, academically and psychosocially. So yes, let’s be clear that that is, for me, a strong ethical and civic responsibility for those who work in education, across the university, policy, research, and classroom practice levels. And yes, I am of the view that transmission of knowledge as well teaching students how to use said knowledge, is core business for schools. For some, that immediately casts me as “neoliberal” in my political views. I’m not now, nor have I ever been, a member of any political party but if my position that accountability matters as much in education as it does in other professional fields makes me neoliberal, I’ll take that over being what I’ve termed previously, paleoprogressive.

On social media platforms, I see some astonishing approaches to evidence, sadly most often from other education academics, but also from education consultants whose steady-as-she-goes operating models cleverly create illusions of change while simultaneously minimising it (often with a great deal of fee-for-service “busy work” in the middle).

Meanwhile, classroom teachers are making it their business to learn the language of evidence, and often do so in their own time, via teacher-led platforms such as researchED around the globe. Others engage in further study and/or join closed communities of practice on platforms such as Facebook. They are not waiting for those in the academy to take the lead on prioritising evidence-based practice, and sadly, that’s probably just as well. I have blogged previously about the fact that much change is occurring on the metaphorical forest floor (classrooms), while the tall trees, whose branches thrive in rarefied air (education academics) are somewhere between oblivious and hostile to these efforts.

Evidence-based medicine” is a term that entered clinical parlance as recently as the 1990s, though it had antecedents that go back much further than that, such as the US Flexner Report in the early 20th century.  Critically, it was a call to medical practitioners (and rapidly, by extension, all health professionals) to move beyond blind and often uncritical faith in practitioner judgement, beliefs, and established practice, to a recognition that independent research might offer different, more effective, sometimes more efficient approaches, to patient care.

Evidence-based practice in health is not about

  • so-called "cookbook" practice (aka “one-size fits all approaches)
  • practitioners blindly following research findings
  • replacing/overriding professional judgement
  • ignoring the context of the individual
  • pretending that endless resources are available.

Medicine, like education is an interface of science, craft, and the complex vagaries of human nature and human behaviour. Claims by some in education that “it’s easy in medicine, you just do a clinical trial and find the answer” betray a phenomenal ignorance of the psychological, socio-cultural, cognitive, linguistic, and economic factors that introduce enormous variability into clinical trials and have to be taken account of in research analysis.

Education is not special. Like health, it deals with human beings in all their complex and unpredictable glory. Like health, too, its practitioners have ethical obligations to (a) do no harm and (b) deliver effective services that improve the human condition. The difference, however, lies in the misconceptions and misrepresentations of research that are perpetuated in education university lecture theatres, and by extension in classrooms around the globe.

I’ve compiled a list below of what I see most commonly on social media and have quoted to me by teachers who are grappling with change processes at school and sectors levels. 

Line of argument against evidence-based practice

My response

Teaching is too complex to be studied scientifically

 

Complexity does not take systematic investigation off the table.

Researchers in medicine, allied health, economics, psychology, nursing, and public health all study highly complex humans and human systems. They do so using research methodologies and tools that are appropriate for the questions being asked. They then triangulate findings across research methodologies and findings to find the best direction of travel, based on current evidence, knowing that this will need to change as the evidence evolves.This is not the same as "choose your own adventure". 

Research in education is not about achieving “perfect certainty” – that’s an unhelpful straw man argument. Rather, it should seek to identify practices that are more likely than alternatives to improve outcomes, for the majority of students.

Education research, like health research, should allow practitioners to play the probabilities in measured, careful ways.

Every child is different, therefore general findings are meaningless

Children are individuals, yes, as are patients. However, there are more similarities than there are differences between humans with respect to how they learn (see for example, the work of Dan Willingham and Stanislas Dehaene).

Human variation exists in every field where evidence is used; this is not unique to education.

Research identifies tendencies and probabilities within and across populations. In so doing, it is drawing on amounts of data that extend way beyond the bounds of individual practitioner experience.

The late Daniel Kahneman, author of Thinking, Fast and Slow reminded us that

“The confidence that individuals have in their beliefs depends mostly on the quality of the story they can tell about what they see, even if they see little.”

Experience is better than research

Experience is of course valuable, but it can also reinforce misconceptions and create blind spots where new evidence is concerned, especially if this challenges existing beliefs and practices.

People are often poor judges of cause and effect, partly because they do not have clean line of sight of all of the factors in play. This is one reason that we are all prone to various forms of cognitive bias (see here and here): mental heuristics that have a place in everyday thought-processes, but need to be kept in check to allow us to verify assumptions and revise beliefs and practices in response to new evidence, even when this creates cognitive dissonance or ideological discomfort.

Many educational practices that “felt” effective to teachers for a number of decades were later shown to have little or inadequate impact on student learning. Emily Hanford’s award-winning  Sold a Story podcast laid this bare in relation to the pernicious harm done to tens of thousands of children, around the globe, by balanced literacy and its antecedent, whole language. Teachers used these approaches in good faith and because they had not been presented with alternatives. In many cases, though, when alternatives are made available, teachers seize them with enthusiasm.

Evidence-based practice threatens teacher professionalism and autonomy

Professions that we hold in high esteem (e.g., medicine, aviation) actually have very low levels of autonomy, and high levels of accountability. I have blogged about this aspect of professionalism previously. 

There is a double-standard in the fact that folk in education expect evidence-based decision-making when they are the patient/client/passenger in a transaction but don’t see that the rest of the community expects the same rights as “consumers” of education.

What threatens teacher professionalism (and workforce longevity) is the creeping realisation that they have not been adequately equipped for classroom practice, using effective approaches for positive behaviour support and instructional success.

Evidence-based practice does not inherently require pre-prepared lesson plans but it’s difficult to understand opposition to practice that reduces variability and promotes consistency within and between classrooms. High variability and high quality cannot co-exist. Schools and sectors need to choose one.

In most professional fields, expertise involves integrating evidence with professional judgement. This can occur when teachers are well-equipped by their initial teacher preparation to critique and use research evidence.

The application of evidence does not remove the need for teacher decision-making. It enhances and refines teacher judgement and allows teachers to apply their pattern-recognition skills in decisions across the day.

Classroom practices should be dictated by the best interests of the students, not the personal preferences/ideologies of the teacher. Think about how this principle plays out in medicine, nursing, and aviation.

When teacher unions (mis)play the autonomy card, they are working against the best interests of their members.

Randomised controlled trials are not applicable / ethical in education.

Randomised controlled trials (RCTs) are sometimes described as the “gold standard” for establishing cause-and effect with respect to whether an intervention works. I won’t provide a tutorial on RCTs here as readers can easily learn more from online sources. Suffice to say that when they are well-designed, RCTs give us the greatest clarity about the efficacy of a teaching or intervention approach, as they control for the interference of many variables that can sit between cause (the teaching or intervention) and effect (student outcomes).

It is important to distinguish between efficacy (how well an approach works under well-controlled experimental conditions) and effectiveness (how well it performs “in the wild” of the real world). If an RCT supports efficacy, then the next step is an effectiveness trial.

A common misconception is that when RCTs are done in schools, some students receive “something” while those in the control arm receive “nothing”. This is incorrect. Those in the control arm may receive business-as-usual teaching, or they may receive another intervention that is considered worthy of testing. In some cases, there is a cross-over design, where students are shifted from one arm of the study to the other, and so are exposed to more than one approach.

RCTs, like all research carried out in schools, are subject to rigorous review by human research ethics committees.  

For a fascinating accounting of how RCTs have transformed our world, across health, criminology, education and economics, have a read of Andrew Leigh’s Randomistas.

Schools are no more or less complex as research environments than hospitals, community health settings, prisons or rehabilitation facilities. All must grapple with the vagaries of human nature and variation in all its guises.  

Research changes all the time, so teachers can’t rely on it

Revision in light of new evidence is a strength of science, not a weakness. Science is by definition self-correcting. The story of stomach ulcers is a good example of this. While once thought to be due to stress, ground-breaking and paradigm-shifting research by Australian researchers, Marshall and Warren, showed that bacterial infection, not stress, is the culprit, and management was overhauled at scale, via revised clinical care pathways that all practitioners were expected to adopt, and did so, because it is their ethical responsibility to do so.

The fact that some findings evolve / are modified over time does not mean that all findings are equally uncertain. It means that we need to be open to calibration and in some cases, de-implementation of current practices, as the evidence evolves.

It’s insulting to talk about a “hierarchy of evidence”. All methodologies are equally valid.

This is an unsophisticated play for egalitarianism in the face of both common sense and Research Methods 101 principles.

Research methodologies should be primarily selected for their goodness-of-fit with the question(s) under investigation. In many cases, researchers are faced with choices, e.g., if they are asking “What is the lived experience of teachers who have been abused and bullied by dissatisfied parents?”  they may elect to carry out qualitative, in-depth interviews, or they may elect to construct a survey that is framed in such a way that responses can be quantified, e.g., via Likert-type scales; or they may do a mix of both. In all cases, their study scope will be constrained by resource issues, and we cannot escape the reality that a large, representative survey sample will have higher validity and generalisability, than will a small, qualitative study of say, 6 to 8 teachers’ experiences. This does not “dismiss” the latter; it just means that one is going to add more weight to the evidence scales than the other.

Medicine is a so-called evidence-based discipline, and errors still occur in that field.

This is reminiscent of the school-yard defence that “other people were throwing stones too, so why am I getting into trouble?”.

Yes, errors and even malpractice and snake-oil interventions do occur in medicine (and other health fields) but that it is no reason for education, or any other discipline to not be applying the highest standards of research rigour and accountability.

Disrespect for the scientific method (by which I mean all methodologies, applied rigorously in ways that are appropriate for the questions under investigation) ultimately leads away from improvements in the human condition.

There’s conflicting evidence about X/Y/Z, so in the meantime teachers should just make up their own minds.

When we think about evidence, it’s most helpful to think of a set of scales and ask where does the current weight of evidence sit? We should not overlook the fact that some studies seem to be outliers, but we also can’t wait for every question, about every practice to be answered.  Staying curious and open-minded, while getting on with it in the meantime is what happens in schools that roll up their sleeves and go on change journeys towards improved practice. See examples here and here.

Quantitative research reduces complex phenomena in ways that make them overly simplistic. Just because you can count something, doesn’t mean it counts.

Like all arguments in this debate, there is some truth to this claim, but it is not the whole story. Yes, quantitative tools (like surveys that use numerical rating scales) by necessity, represent something complex (like anxiety level) as a number. This does not mean, however, that the data is somehow meaningless. If, on a 5-point scale, I tell the researchers that my anxiety level is “5”, that is a meaningful representation of my subjective experience. I may also find it easier to circle a 5 on an anonymous survey than to take part in an in-depth research interview. Large-scale quantitative studies often provide persuasive arguments to funders and policymakers, because they enable researchers to say things like “55% of Year 7 students experience anxiety associated with poor reading skills”.  

Education has a history of fads and fashions, and evidence-based practice is just another one. It too shall pass.

Sadly, it is true that education's history is one of fads and fashions, so teachers who have been around for a few decades do see many approaches come, go, and sometimes come again, slightly dressed up but not heavily enough disguised that they are not recognisable. It’s one thing to experience change fatigue when it leads to improvement, but change fatigue when the only outcome is change, is understandably frustrating and disheartening for teachers.

The fact that this is true is no tribute to education as a discipline and speaks to the fact that while other disciplines have moved forward over the last 50 years, education has changed, but has not necessarily progressed. This cycling and recycling approach to practice in education was summed up by Hewitt and Sachdeva in 2025 this way:

Education, on the other hand, lacks this cycle of progressive improvement. In spite of decades of reform and billions spent on improvement initiatives, it’s not clear that teachers are any more effective today than they were fifty years ago. While the curriculum has evolved over that period, it would be difficult to claim that the quality of education has significantly improved or that the gap in student outcomes has narrowed substantially.

Evidence-based practice is just a political ideology

This is essentially an ad hominem argument and flags a reluctance to engage with substance.

While stakeholders may have ideological/political alignments (as is their right), the central claim—that educational decisions should be informed by the best available evidence—should not be inherently ideological. We’re in big trouble if it is.

Criticising those who argue in favour of education’s better engagement in the generation, critiquing and application of evidence does not address the quality of the evidence itself. It’s just an attempt at deflection.

Pleasingly, as noted above, we’re seeing schools generate practice-based evidence that complements the body of scientific evidence about how children learn and optimal ways to teach them. See here and here. This is powerful evidence in its own right and is gaining a seat at the policy decision-making table. It also needs to start re-focussing debates in education faculties.

If there’s a paper to support an approach in a peer-reviewed journal, it must be evidence-based.

This is lazy reasoning and betrays ignorance of the academic publication food-chain in practice. Some journals only publish papers that adhere to rigorous methodological standards, and as a consequence, reject 90%-plus of the submissions they receive. Others are less fussy about methodological rigour and accept a wider array of manuscripts, inevitably of variable quality.

So no, being able to locate a peer-reviewed paper that says the earth is flat does not, in fact mean that if you walk out your back door and keep walking, you will eventually fall off the edge.

Facts are malleable – it’s a cultural imposition to say that “2+2=4”.

I know, I know, I know, but I saw an example of this just recently. I’ll just make the observation here that I’m pretty sure the academic who made this claim would not accept this position from their payroll department if a previously agreed salary loading was then reduced by their employer.

Approach XYZ was tried and failed

Implementation science is a complex field and claims that something was tried/implemented in a school or system often fall well short of what is required to create replicable and testable conditions that can shed light on whether something is

·         likely to have benefits, and if so for whom

·         likely to maintain the status quo, and if so for whom, or

·         likely to worsen outcomes, and if so for whom.

Fidelity of implementation and evidence of it are no trivial matters but are often underdone in policy shifts at sector level.

One of my great concerns in the current transition to more explicit teaching in Australia is that it goes without saying that implementation will be uneven, but we will have poor line of sight of this. This will result in all sorts of unsubstantiated claims about what did and did not work, and for whom.

Children’s academic achievement reflects their parents’ socio-economic status, and no research will change that.

This is a particular heart-sink one for me because it betrays an underlying belief that schools and the teachers who work in them are impotent; that teachers’ everyday instructional practices don’t matter, because biology-is-destiny.

Yes, family socio-economic status is a strong predictor of student academic achievement, but shouldn’t we be disrupting that trajectory through high-quality, impactful teaching and support? Fortunately, in many cases we are (see practice-based evidence, above). My colleagues and I have addressed this argument here as well.

I could do on, but hopefully this list gives readers a sense of the unproductive and in many cases, unsophisticated arguments that education does not need to hold itself to the same standards of evidence as other disciplines.

In health, we have the Therapeutic Good Administration in Australia (and cognate bodies in other industrialised nations) to systematically test the safety and efficacy of medicines, devices, and other therapeutic goods before they can be prescribed for or purchased by the public. Imagine the chaos, harm, and exploitation that would rain down on us if we were not the beneficiaries of such protection.

Where is this product and practice protection in education?

Teachers cannot be gatekeepers in

  • the adoption of practices that are consistent with current scientific evidence
  • the de-implementation of superseded approaches
  • the identification and avoidance of pseudoscientific approaches

where leaders (academics and policy makers) in their profession do not subscribe to the scientific method, as applies in other fields of human endeavour. I repeat, education is not special.

Education is, ironically though, a slow learner, it would seem of the maxim that when you stand for nothing, you fall for everything. 

 

© Pamela Snow (2026)