What are “levels of evidence” and why do we need to talk about them in education?

  

Image created using ChatGPT

Around ten years ago, when my colleague, Dr Caroline Bowen AM and I were writing a book on evidenced-based support for children and young people with additional needs I found myself going down all kinds of rabbit holes to examine what were sometimes extraordinary claims about equally extraordinary practices. Examples of such practices were cross-body physical exercises to improve reading and writing skills, using coloured lenses and overlays to improve reading skills, “Earducators” to provide auditory-integration training to children diagnosed with made-up disorders, psychoastrology for children with autism spectrum disorders, Chelation therapy, and neurolinguistic programming – to name a very small sample.

In early 2026, we released a “companion” text that is aimed more fairly and squarely at a parent readerhsip, to support them in their increasingly complex journey through the noisy, invariably expensive and often-alluring intervention marketplace.

No-one should be surprised that the “average parent” struggles to access, appraise and distil research evidence but it is disappointing that many teachers and allied health professionals find this daunting also – invariably because of a paucity of preparation for this in their pre-service education.

Regardless of their lack of preparation for the task, parents, teachers and clinicians* all have vested interests, however, in being able to answer questions such as:

• What are the hallmarks of effective interventions to support struggling early readers?
• Does the timing of speech-language pathology intervention matter for young children whose speech is dysfluent?
• Are children with pre-school language disorders at higher risk of later reading and writing difficulties?
• Do coloured lenses/overlays improve the reading of children who have a diagnosis of dyslexia?
• If I read to my toddler, will that make them a strong reader when they attend school?
• Should phonemic awareness be taught in isolation in early years classrooms, or should printed letters be present?
• Can children with an intellectual disability be taught to read? 
• Will a gluten-free diet lessen a child’s autism symptoms?

*Policy-makers also.

This is where the “fun” starts.

I’m reminded of an email exchange some years back with an education academic at a major Australian university who was promoting so-called “brain-based interventions” for children with special needs. The term “brain-based” struck me then (and still does) as odd, given that all learning (declarative and/or procedural) ultimately comes back to the brain and its complex inter-connected systems. It seems a little like what my late dad would have described as “selling ice-creams to Eskimos”.  When I wrote to the academic in question and asked for published evidence to support said “brain-based interventions” their reply rather glibly informed me that “you can find a journal article to support just about anything” – as if this was a good thing.

While it is true that peer-reviewed journals publish material of wide-ranging academic merit, this is a problem – for the scientific community and for parents, teachers and clinicians (not to mention policy-makers and taxpayers) who are the consumers of said research. It is not a loop-hole that should be exploited by those who want to argue some version of:

• If there’s a peer-reviewed journal article showing an association between two factors, those factors must be linked in some important way;
• If there’s a peer-reviewed journal article showing lack of an association between two factors, those factors must not be linked;
• If there’s a peer-reviewed journal article that reports positively on an intervention, the intervention is trustworthy;
• If there’s a peer-reviewed article saying parents/teachers/clinicians liked a particular assessment or therapy, it should be adopted;
• If there’s a peer-reviewed journal article saying parents/teachers/clinicians disliked a particular assessment or therapy, it should be discarded;
• All research methodologies are equally valid for answering all research questions;
• We should not appraise individual papers or bodies of evidence to assess the weight they can add to the decision-making scales.

One paradox we need to address right here is the fact that peer-review of new research is both (a) well-intentioned and preferable and (b) imperfect.

Peer review is well-intentioned and preferable because

• It means that manuscripts are critiqued, often in a “double-blind” way, by academic peers who can identify flaws and gaps in arguments and analyses and suggest ways in which the paper can be strengthened before it is published.
• It helps to uphold academic standards regarding coverage of prior literature, application of ethical principles to recruitment and data collection, selection and justification of research methodology, data analysis, and appropriate extrapolation without over-reach.
• It sets published research apart from other written sources, such as newspaper and magazine articles, blogposts and newsletters, and product websites.

Peer review is imperfect because

• Some journals are inundated with submissions and can be highly selective, based on rigorous criteria, about what even goes out for review, let alone what is ultimately accepted for publication. Others have editors who are sitting by the phone, waiting for copy, and necessarily need to lower the bar to meet publication schedules.
• Sometimes editors can locate reviewers (who do their work without payment, often in their own time) with the necessary content knowledge and research background, and sometimes they can’t. Since the COVID-19 pandemic, this seems to have become more challenging than ever.
• Two reviewers may arrive at markedly different assessments of a paper, and the editor needs to exercise judgement on whose review takes precedence and/or how tensions are resolved. There is inevitably some cognitive bias and human error in this process.  

So - what does it mean to ask about the “level of evidence” for an assertion or a practice?

We can ask about the level of evidence behind an assertion, for example, “Vitamin D plays an important role in bone density” or about the level of evidence that can address a question about a practice, for example “Does a daily Vitamin D supplement improve bone density”? Corollaries in education might be “Knowledge of the alphabetic principle is important for beginning readers” and “How do we best establish knowledge of the alphabetic principle in beginning readers?”.

Astute readers will already be seeing the questions within questions in these examples.

• What do we mean by “important role”?
• What daily dosage of Vitamin D are we referring to? Which population(s) are we referring to? Children? Adolescents? Adult males? Adult females? Pre-menopausal females? Post-menopausal females?  People with particular medical conditions?
• What do we mean by the “alphabetic principle”?
• How does the alphabetic principle apply in a morphophonemic language like English? 
• What do we mean by children’s knowledge of the alphabetic principle? Which children? At what age/year level? How is this determined?

These questions within questions are often what leads academics, when asked about evidence to rightly respond by beginning with “Well it depends”. There’s not always a cut-and-dried answer for people schooled in the rigours of research critical appraisal and design. Yet we still need to respond to questions about whether current evidence supports a particular proposition or practice."It depends" is not code for "choose your own adventure". 

So, we must engage with the idea of levels of evidence.

This is a principle borrowed from health (medicine, nursing, allied health, public health), where it has gained wide acceptance in the last thirty years to guide policymakers and practitioners who must sieve through vast amounts of peer-reviewed (and increasingly, non-peer-reviewed) sources to guide their decision-making and optimise outcomes for their end-users.

Some readers may take exception right here, to me applying a framework derived from health to education. A gentle reminder is warranted then, that the idea of levels of evidence is equally applied in many other fields such as psychology, criminology and criminal justice, economics, aviation, engineering, road safety, environmental science and conservation, and forensic science – to name a few. As I have noted in a previous blog-post about education’s relationship with evidence, education is not special.

The evidence hierarchy is simply a means of organising, filtering, and appraising published evidence.

The hierarchy of evidence does not say that one research paradigm (e.g., positivism Vs autoethnography) is inherently superior or inferior to another. You will see in the infographic above, that both quantitative and qualitative data sources can be used right across the hierarchy. Choice of research methodology should align with the study aims/questions posed. Many education researchers, me included, use mixed methods to address their research questions. This might mean (for example) collecting student data on standardised measures as well as carrying out in-depth qualitative interviews with teachers to understand their experiences of a new teaching approach.

You will note too, that the hierarchy is a "stack" that tapers towards the top, because higher-level evidence is more difficult to collect. RCTs, for example, are usually expensive, so there will be fewer of them, meaning that there will be fewer to draw on for systematic analyses. This applies in all fields, not just in education. Evidence at lower levels is often where new knowledge begins, as occurred in the early 1980s, with the appearance of case reports heralding the arrival of the illness we now know as AIDS. 

Being open to the idea of levels of evidence helps to guard against the many ways in which cognitive bias can influence the thinking of researchers, policy-makers and/or practitioners, as described by the late Daniel Kahnemann, in his 2012 text Thinking Fast and Slow. The levels of evidence framework also offers pushback against the "fads and fashions" state of play that education has normalised in recent decades.

There are many myths and misconceptions about research methodologies and levels of evidence in education, and I blogged about these earlier this year: The dog ate my homework: Education’s evidence-excuses echo-chamber. The ideas expressed in that blogpost were uncomfortable for some, particularly my reminder that education is no different from any other social science that examines the behaviour, attitudes, and beliefs of humans. Ultimately, we all need to be able to apply rigour in the laboratory of the real world, if we are to make meaningful advances for our end-users.

Evolving evidence needs to be monitored, weighed probabilistically, and considered alongside available resources and the reasoned judgement of professionals. It is not a black-box but nor is it a means of arguing, as did my fellow education academic some years ago, that “anything goes as long as I can find a published paper somewhere that vaguely suggests this might be OK”.

When education academics attend a medical practitioner, drive across a bridge, or buckle up in an aeroplane, they are privileged beneficiaries of the respect shown for the hierarchy of evidence in those disciplines.

Education - you owe your stakeholders the same respect.

© Pamela Snow (2026)