Thursday, 18 April 2019

Whole Language Unicorns at home at the University of Melbourne

I was disappointed this week to see that the University of Melbourne, one of Australia’s (and indeed the world’s) most highly-esteemed universities, was promoting the ideas of David Hornsby on the important subject of how we teach children to read. If you haven’t already done so, you can listen to the interview I am referring to (it takes about 25 minutes) here

I have some familiarity with David’s work, having read publications such as this one and have no doubt about the depth of his commitment to supporting teachers and improving children’s progress in learning to read. It is unfortunate, however, that his key ideas about the role of phonics instruction remain uninformed by the last 20 years of cognitive science research on how best to get all children across the bridge to successful reading, writing, and spelling, in the first three years of school. Instead, David’s ideas are pretty Whole Language unicorns that persist in the imaginations of those who believe that “authentic children’s literature” is the key driver of successful reading. This does not align at all with the recommendations of three national inquiries into the teaching of reading, in the US, the UK, and Australia all of which recommended an explicit focus on systematic phonics instruction as the starting point for novice readers - alongside development of all of the other skills essential for successful reading. 

I did not find myself in complete disagreement with David, however, as I listened to this podcast. 

Here are the points on which David and I agree:

1.      Teachers are not nearly knowledgeable enough about the structure of language. In order to be able to teach the written code, teachers need to be able to understand that English is a morpho-phonemic language, and the full implications of what this means for how written and spoken language map to each other. I have blogged previously about concepts that teachers need to understand in order to be able to do their jobs properly, about the fact that they currently lack this knowledge, and the fact that responsibility for this sits with education academics who have shown a disgraceful disregard for the reading science, and the importance of ensuring this is transmitted to classroom teachers. 

I note that A/Prof Misty Adoniou has also written about low teacher knowledge stating that “All the participants in this study had patchy content knowledge of the English language, and very little knowledge of how to apply what content knowledge they had, although they were convinced this kind of knowledge was important”. So we seem to be in agreement, David, Misty, and me, that teachers are being sold short by their pre-service education. Where you choose to direct your rage about this is up to you, but I would start with the damage done to young children, particularly those who come from disadvantaged backgrounds, and need the turbo-charging impact of highly effective teaching by highly knowledgeable teachers in the first three years of school. You might also want to consider the indignation of teachers who are paying thousands of dollars for courses that do not deliver the goods they thought they were buying – competence to go into a classroom and teach children how to read. 

2.      Low teacher knowledge means that teachers jump around from one seemingly attractive quick-fix commercial product to another. There’s an awful lot that goes on in schools that either lacks an evidence base, or for which the published evidence indicates a lack of efficacy. My colleague Dr Caroline Bowen and I have written about this here. This damages the professionalism and accountability of teachers, and yes, sorry, teachers, do need to be accountable for their decisions and practices, though the extent to which this plays out in practice is questionable. When was the last time a teacher was held to account by a registration body for failing to apply evidence-based approaches in their practice? 

3.      Programs don’t teach children how to read, teachers do. This is a point that is often made by Dr Louisa Moats, and speaks to some of the points made above about the importance of strong teacher knowledge. I understand though, why many teachers seek the security of a program, because it instils some confidence, given the weak knowledge base conferred by their pre-service education. Can you see a vicious cycle here?

These are the points on which I disagree with David:

1.      He makes a blanket declaration that “Programs don’t work”. That’s about as silly as saying “antibiotics don’t work”. Sometimes they do, sometimes they don’t. It depends on a range of factors. My experience in working with teachers in schools is that programs often create the platform on which teachers go on to extend their knowledge and develop their practice. Programs definitely work better in the context of high teacher knowledge (in which case, perversely, there is less need for them) and when they are administered with fidelity.  When teachers deliver their own version of Program X and see only mediocre results in return, it is highly problematic that they then report “Program X does not work”. If we’re going to deliver programs, let’s first:
a.      Check their alignment with the current evidence on how best to teach reading
b.      Ensure that teachers receive appropriate training and ongoing coaching support so the program is delivered as intended, i.e. with fidelity. 

2.      David appeals to the romantic notion that all that is needed to get children interested in reading, is exposure to “authentic texts” (this is, in itself a problematic term as it implies a non-existent binary distinction, but I won’t get distracted about that here). Again, I have blogged previously about the “authentic illusion” in teaching children how to read. The logical fallacy that is occurring here is the conflation of books we read to children, with books we use to teach children to read. We want to expose children to a wide range of rich texts, to
a.      Help them understand the power and “magic” of books
b.      Extend their receptive and expressive vocabularies
c.      Expose them to increasingly complex sentences (and the increasingly complex ideas they represent)
d.      Help them understand narrative structure (and later, the structure of other discourse genres).


My three and a half year old grandson can be seen here "reading" to his two-month old sister. Yes, it warms my heart, at so many levels to look at images such as this. But no-one in my family is under any illusion that any actual reading instruction is taking place here. My grandchildren are engaged in precursors to becoming successful readers, but my daughter and son-in-law should be able to rely on their children's teachers to do the actual business of teaching them how to read. Schools should not rely on parents to be the critical link here. Some are able in this respect and some are not. Let's meet the needs of all children who enter foundation-year classrooms, not just the more advantaged ones. 

Immersing children in text does not teach them how to read, any more than playing hours and hours of Mozart concertos turns them into pianists or violinists who can play Mozart. Instead, the hours and hours need to be invested into learning and practising sub-skills.  Listening to music may be motivating and enjoyable, but it won't in and of itself teach the complex skill of playing music. Why do we "get" this in relation to music, but not in relation to reading?

Further, the incidental embedding of phonics instruction into the process of reading beautiful literature to children is inefficient (see Stephen Parker's free books on phonics instruction for further expansion on this) and fails to systematically address the knowledge of phoneme-grapheme correspondences that children need to learn in an efficient manner in the early stages of reading, in order to use their knowledge of the code to access more and more complex texts and their meanings. Sure, some children will be fine with the incidental approach, but we should not be focussing our teaching methods on those more advantaged children who will get across the bridge almost irrespective of the reading instruction they experience. 

A strong Tier 1 is the best insurance against the creation of instructional casualties, but is often not in place.

Ironically, in order to teach in accordance with the recommendations of the three international inquires referenced above, teachers need to be more knowledgeable about language, in exactly the way that David Hornsby and Misty Adoniou are advocating. 

3.      David relies on the (almost comically) flawed definition of decoding in the Australian Curriculum:

A process of working out a meaning of words in a text. In decoding, readers draw on contextual, vocabulary, grammatical and phonic knowledge. Readers who decode effectively combine these forms of knowledge fluently and automatically, and self-correct using meaning to recognise when they make an error.

This may as well just be a definition of reading, as it fails to focus on the specific and essential sub-skill of mapping phonemes onto graphemes in order to first make sense of the black lines that we call letters on the page.

So – just because a definition has found its way into the Australian Curriculum, does not mean that it aligns with the science of reading. Anyone who is familiar with the Simple View of Reading, or with the Hollis Scarborough Reading Rope, will know how far short this definition falls of the skills beginning readers need to be taught in order to achieve decoding mastery. For my money, a child who can read aloud an isolated written word has decoded that word. Meaning comes next – the word is either in the child’s lexicon or needs to be added as a new word as a consequence of the oral language value-add of being able to read.

4.      David dials up the hoary old wind/wind chestnut to argue that “phonics alone isn’t enough”. Well, first of all, I am yet to meet anyone who claims that “phonics alone is enough”. 

       The debate as I see it, as around the role and type of initial instruction to get novices off the blocks as successful readers. Anyone familiar with the terrific Five from Five Project will know that no-one on the systematic phonics instruction side of the debate overlooks the importance of vocabulary, fluency, and comprehension – as well as inferencing and using contextual cues. The latter, however, should augment word identification rather than being the primary strategy relied upon by a novice who encounters an unfamiliar word.

My question for David, then is how does a child even get to the dilemma of wind Vs wind without first using phonics-based decoding skills?

5.      Finally, David makes the bizarre claim that systematic phonics instruction involves the teaching of nonsense words. Sigh. We don’t seem to be able to put this silly idea to bed. Firstly, the distinction between real words and non-words is dimensional rather than categorical. Just have a look at J.R. Tolkein, Lewis Carroll, J. K. Rowling, Dr Seuss, to name a few, if you don’t believe me. I am not aware of any evidence that says that novice readers approach unfamiliar words with a real-word – non-word filter.
Consider too, the ever-evolving nature of spoken and written language and the constant addition of what were once non-words, to the real-words lexicon. You can find examples of recent additions to English here. How can anyone (child or adult) read these new words if they are unable to decode in the truly get-the-word-off-the-page sense of decoding?

Think about this for a moment – every word is potentially a non-word to a novice, because their orthographic lexicon is still under construction. I have seen teachers “play” with non-words in Phonological Awareness activities and am completely comfortable with that – the children were experiencing rhyme and enjoyed talking about the meaning of the words. Consider this example: the teacher is using the words “wig, pig, and big” in a rhyming exercise and then introduces “rig” asking the class if it is a real word or a non-word. Chances are a lot of five-year olds will say it’s a non-word. What a great opportunity to wrap some vocabulary work into the Phonological Awareness activity. 

So can we be very clear about this please – non-words are extremely useful as a light-touch screening tool to determine the extent to which children are mastering essential phoneme-grapheme correspondences, so that they can decode in line with the scientific definition of decoding, not the politically sanitised version that is pleasing to Whole Language unicorns. 

I hope going forward that our esteemed educational institutions will apply careful quality and evidence-alignment filters over ideas they promote to teachers, who, more than anyone else, must be thoroughly tired of the contentious nature of something that they need to be able to do everyday - and across all year levels. 

In the last fifty years, we have put men on the moon, eradicated a number of terrifying childhood illnesses, and made major advances in engineering, aviation, construction, and manufacturing.  

But thanks to a massive knowledge-translation failure,  we cannot be relied upon to teach children how to read.  

(C) Pamela Snow (2019) 

Friday, 29 March 2019

Let’s all throw out the scientific method. It’s not perfect and it's too hard. AKA The soft bigotry of gendering research methodologies.

Last week, a piece entitled The problem with using scientific evidence in education (why teachers should stop trying to be more like doctors) appeared on the AARE Blog. This was co-authored by an education academic, Dr Lucinda McKnight (Deakin University) and a medical education academic Dr Andy Morgan (Monash University). The authors purport to mount an argument as to why the notion of evidence-based practice should be resisted in education. 
I believe the article is deeply flawed at a number of levels, and have provided a detailed response to it here.
By way of background, I worked in medical education for ten years, and have also taught teachers at postgraduate level, as well as having taught across some ten allied health professions.


 McKnight & Morgan  
For teachers to be like doctors, and base practice on more “scientific” research, might seem like a good idea. But medical doctors are already questioning the narrow reliance in medicine on randomised controlled trials that Australia seems intent on implementing in education.
 My response
Where is the evidence that all that is being recommended is randomised controlled trials (RCTs)?

It is right that evidence derived from RCTs be questioned, because it is right that evidence from all research be questioned.

In medicine, there is a sound understanding of an efficacy trial Vs an effectiveness trial. This difference should be considered and discussed in education as well.
In education, though, students are very different from each other.
They are no more different, nor similar to each other than are patients. Doctors, like teachers, rely on pattern recognition to form and test hypotheses. They could not do their jobs if this wasn’t the case.
Unlike those administering placebos and real drugs in a medical trial, teachers know if they are delivering an intervention. Students know they are getting one thing or another. The person assessing the situation knows an intervention has taken place.
Yes, but students do not necessarily know which teachers have been exposed to an intervention, e.g. a series of professional learning seminars. And researchers with overall responsibility for a trial can easily be blinded to allocation group - I speak from personal experience on this.

This statement betrays an unfortunate lack of understanding of the nature of RCTs in education.
Constructing a reliable educational randomised controlled trial is highly problematic and open to bias.
Yes, doing rigorous research is challenging.

Yes, all research is open to bias.

Skilled researchers make it their business to recognise, and minimise sources of bias, and to report their findings with caution.
Before Australia decides teachers need to be like doctors, we want to tell you what is happening and give you some reasons why evidence based medicine itself is said to be in crisis
The fact that researchers are questioning an approach does not mean it is being thrown out in its entirety. That’s exactly the kind of thinking that has plagued education for decades, as shown by the tendency to adopt fads and fashions, with zero research behind them, let alone any that has any supporting evidence. I have written about this here with my colleague, Dr Caroline Bowen.
Randomised controlled trials are just one kind of evidence
And this is news because??

Of course RCTs are only one kind of evidence. That’s like saying Toyota is only one make of car.
Medicine now recognises a much broader evidence base than just randomised controlled trials.
This is also not news. Medicine has always recognised a range of study designs. What seems to be overlooked in educational discourse, however is the notion of levels of evidence.

In health, it is recognised that different study designs have different degrees of strength in establishing the efficacy or effectiveness of an approach.
Other kinds of medical evidence include: practical “on-the-job” expertise; professional knowledge; insights provided by other research such as case studies; intuition; wisdom gained from listening to patient histories and discussions with patients that allow for shared decision-making or negotiation.
These are obviously important in all fields. They just don’t sit at the top of the hierarchy as to what can be established, replicated, and/or refuted, using the scientific method.
Privileging randomised controlled trials allows them to become sticks that beat practitioners into uniformity of practice, no matter what their patients want or need. Such practitioners become “cookbook” doctors or, in education, potentially, “cookbook” teachers. The best and most recent forms of evidence based medicine value a broad range of evidence and do not create hierarchies of evidence. Education policy needs to consider this carefully and treat all forms of evidence equally.
All forms of evidence are not “equal”. This does not mean that they should not all be considered, including expert opinion, but human beings are prone to all kinds of cognitive bias

Sometimes our intuitions tell us that something “should” work, or even that it “seems” to work, but the scientific evidence counters our intuitions. This is the subject of Andrew Leigh’s book Randomistas.
Teaching is a feminised profession, with a much lower status than medicine. It is easy for science to exert a masculinist authority over teachers, who are required to be ever more scientific to seem professional.  They are called on to be phallic teachers, using data, tools, tests, rubrics, standards, benchmarks, probes and scientific trials, rather than “soft” skills of listening, empathising, reflecting and sharing.  
Where to start with this one? Let me point out a few facts:
Medicine is rapidly becoming feminised, with more females enrolled to study medicine in many Australian universities than males. Does this mean that it will now abandon centuries of commitment to the scientific method? I certainly hope not.
Dark Ages, here we come, if it does.
      There’s an assumption here that the scientific method would be an imposition on those poor feeble women in teaching, who would not be able to cope with the rigours of its analytic tools.
How insulting.
     There is no connection between genitalia and the tools of scientific inquiry. This is just silly. What about all those women who conduct (and use) quantitative education research?
Where does this leave them?
      So-called “soft skills” are just as important in medicine as knowledge of human biosciences, pharmacology, and so on. No-one is suggesting otherwise.

Why can’t doctors and teachers be content experts AND competent consumers of new research? When did it become either-or?

A Western scientific evidence-base for practice similarly does not value Indigenous knowledges or philosophies of learning. Externally mandated guidelines also negate the concepts of student voice and negotiated curriculum.
Education and medicine both need to show a deep respect for and understanding of indigenous knowledge and practices. That does not mean that the Aboriginal man presenting to the Emergency Department with chest pain automatically wants to receive a different type of care from his non-indigenous counterpart. If the latter receives an immediate ECG and blood tests, then so should the Aboriginal patient. There is a place for the student/patient voice and there is a place for professionals to do what professionals are trained and paid to do.
While confident doctors know the randomised controlled trial-based statistics and effect sizes need to be read with scepticism, this is not so easy for many teachers. If randomised controlled trial-based guidelines are to rule teaching, teachers will also potentially be monitored for compliance with guidelines they may not fully understand or accept, and which may potentially harm their students.
If teachers are not confident in interpreting research studies (and I agree they are not), then education faculties need to step up and teach them how to be critical consumers of research – quantitative, qualitative, and mixed methods.

All professionals are monitored for compliance – that’s part of what being a professional means. It is a highly constrained form of public accountability. I have blogged about this previously.
Evidence based medicine is about populations, not people
The fallacy here of course, is that populations are not made up of people. Evidence-based medicine is about using robust study designs to control a range of sources of error so that appropriate conclusions are drawn. It is then up to the individual practitioner to consider the findings in the course of their clinical decision making on a case-by-case basis. As noted below, this entails consideration of evidence, patient values, and clinical resources. But the evidence part is non-negotiable.
While medical randomised controlled trials save lives by demonstrating the broad effects of interventions, they make individuals and their needs harder to perceive and respect.  Randomised controlled trial-based guidelines can mean that diverse people are forced to conform to simplistic ideals. Rather than starting with the patient, the doctor starts with the rule. Is this what we want for teaching?.
Well at least we have some acknowledgement here that RCTs can help to save lives!

It is not the role of an RCT to bring individuals into sharp focus. We have many other study designs that do that much better, and they are considered alongside the findings of RCTs in the development of treatment guidelines.

When medical guidelines are applied in rigid ways, patients can be harmed
When any guidelines are applied in rigid ways people can be harmed. There is nothing illuminating about this statement.

Anyone who is familiar with the pioneering evidence-based medicine work of Dr David Sackett and his colleagues will know that this model emphasises empirical research + patient values + clinical resources. It is not, and never has been, about research evidence alone. Right from the start in medicine, it was emphasised that evidence-based medicine is not a cook-book approach. This is just a straw man.

Interestingly, evidence-based practice was initially seen as an unnecessary imposition on medicine in its early days. Now it underpins the way we educate all health professionals, and the community is the beneficiary – both as patients and as tax-payers.
Trials cannot be done on every single kind of person and so inevitably, many individuals are forced to have treatments that will not benefit them at all, or that are at odds with their wishes and beliefs.
This is another nonsensical truism. No, we cannot include all kinds of people on planet earth in trials (clinical or educational). The whole purpose of research is that we sample from populations, in an effort to generalise back to the population as closely as possible.

Welcome to Research Methods 101.

Just because rigorous methodologies cannot answer every question, for every patient, every time, does not mean they are not the best horse in the race to back.
Educators need to ensure that teachers, not bureaucrats or researchers, remain the authority in their classrooms.
Well a good way to make a start on this would be for education faculties to equip pre-service teachers with scientifically-derived knowledge and skills on (for example) the teaching of literacy and numeracy, as well as the ability to read and critique new research and make decisions about how this should inform practice.

Teachers cannot speak with authority if they do not know the research behind an approach and the extent to which this is contested. This is why, for example, medical students are taught that prescribing antibiotics for children with middle ear infections is controversial. They know the ground will shift under them over time, as the science changes, and are primed to watch for new evidence as it arises, and adjust their practice accordingly.

This is called accountability.
Scientific evidence gives rise to gurus
A more critical and discerning teaching workforce would counter this in a flash – in the same way that it does in medicine. Gurus flourish where audiences can be easily wooed and charmed by pretty graphs and impressive looking numbers.
While medical-style guidelines may seem to have come from God, such guidelines, even in medicine are often multiple and contradictory. The “cookbook” teacher will always be chasing the latest guideline, disempowered by top-down interference in the classroom.
Yes, this is the nature of scientific evidence. It changes, and is sometimes contradictory. Rather than “chasing” the latest guideline, professionals need to avail themselves of new evidence and work out how it should influence their practice. 

This is called accountability.
In medicine, over five years, fifty percent of guideline recommendations are overturned by new evidence. A comparable situation in education would create unimaginable turmoil for teachers.
The paper linked to here states “This investigation sheds light on low-value practices and patterns of medical research”. Wouldn’t this be a good thing in education too? That way, we might never go down the Brain Gym, coloured lenses, learning styles, multiple intelligences, left brain-right brain, brain-based learning (etc) time wasting and expensive rabbit holes that education is so fond of.

One of the challenges of living in a knowledge economy is that information changes. We all have an obligation to keep up as much as possible. Choosing your own adventure, whether as a doctor or a teacher, is not acceptable to the community.
Evidence-based practice risks conflicts of interest
Then let’s be careful not to throw the baby out with the bath water. The more discerning and informed teachers (and doctors) are, the less prone they will be to commercial interests. This is part of the imperfect world in which we live and is not a reason to abandon the scientific method.

There are plenty of commercial interests at work in classrooms around the world today, regardless of the level of evidence underpinning the teaching that is occurring.
Randomised controlled trials in medicine routinely produce outcomes that are to the benefit of industry. Only certain trials get funded. Much unfavourable research is never published. Drug and medical companies set agendas rather than responding to patient needs, in what has been described as a guideline “factory”.
These are all legitimate concerns about health research that need to be managed. They are not reasons to abandon the scientific method. See babies and bath water, above.
Do we want what happens in classrooms to be dictated by profit driven companies, or student-centred teachers?
As noted above, there are plenty of profit-making companies doing their thing in classrooms around the world right now, cashing in on the fact that teachers are a soft target for approaches with a slick marketing spin, and a few researchy-sounding words in the glossy brochure and on the equally glossy box.

The purpose of having a more research-informed teaching workforce is being able to head off snake-oil sales people at the school gate.
We call for an urgent halt to the imposition of ‘evidence-based’ education on Australian teachers, until there a fuller understanding of the benefits and costs of narrow, statistical evidence-based practice. In particular, education needs protection from the likely exploitation of evidence-based guidelines by industries with vested interests.
Ironically, such a halt wouldn’t cause a great deal of disruption, given the limited extent to which evidence-based practice has genuinely found its way into education discourse.
Rather than removing teacher agency and enforcing subordination to gurus and data-based cults, education needs to embrace a wide range of evidence and reinstate the teacher as the expert who decides whether or not a guideline applies to each student.
Perhaps we need to consider the possibility of enhanced teacher agency, in a world where teachers are knowledgeable and confident consumers of new research, by virtue of their grasp of research methodologies and critical appraisal skills?

We can’t consider “a wide range of evidence” while disregarding evidence from RCTs, and not understanding the notion of levels of evidence means that equal weight is inappropriately assigned to a single case study and a meta-analysis of several RCTs. They all contribute to the understanding of an issue, but not necessarily equally, on a study-by-study basis.

One of the key differences between medicine and education, is that doctors frequently need to gain informed consent for their actions. In education, however, consent is implied. Students cannot give or withhold their consent for a particular instructional approach. It just comes their way, like it or not. This only serves to increase, not decrease the ethical burden on teachers to teach in ways that are supported by strong empirical evidence.
Pretending teachers are doctors, without acknowledging the risks and costs of this, leaves students consigned to boring, standardised and ineffective cookbook teaching. Do we want teachers to start with a recipe, or the person in front of them?
No-one is pretending teachers are doctors. But if they want to be afforded at least some professional autonomy, then they have to accept professional accountability, just like other professions do. We all need to acknowledge, however, that no profession is completely autonomous, least of all medicine. We all need to be accountable to our “consumers”, our employers, our professional bodies, and the community.

No, we don’t want recipes, but nor do we want random chaos, and the wild west of everyone choosing their own adventure, either. The Age of Enlightenment created some enduring legacies that we would all do well to hang on to.

As I have stated previously:

Education and medicine, for example, have a great deal in common; they both concern people, interactions between people, complex co-occurrences, and hard-to-control (actually impossible to control) variables, such as race, gender, ethnicity, religion, intelligence, empathy, sometimes unpredictable and seemingly inexplicable behaviour, resource limitations, and the need to establish trust and rapport.

Most importantly, both have to deal with uncertainty, coupled with a weight of responsibility and accountability to communities, peers, and policy-makers for outcomes

Perhaps some in education think that they can dispense with research rigour because children can't be harmed by sub-optimal or mis-directed classroom practices. 

They are wrong.

(C) Pamela Snow (2019)