We need an open dialogue about what 'evidence' is

Please note: the ATL website is no longer being updated and will be taken down soon.

Visit the new NEU website

Blog
12 December 2014 by Nansi Ellis
Tucked away in the ‘evidence check’ documents in the Select Committee webforum is something I have always suspected might be the case.

The evidence government uses to develop policies is sometimes no more than 'it became clear that there was logic to...' whatever the next step is. This is taken from the evidence check on the National College for Teaching and Leadership.

What has the evidence-check demonstrated?

  1. A big variation in research questions. The ‘summer born children’ evidence check looks at the impact of summer birth on test and exam attainment throughout schooling, as well as the relationship of the date of starting school with academic outcomes. On the other hand, while the evidence check for music education is very interesting, it’s impossible to know whether the DfE is looking at the quality or quantity of music education, or perhaps how it should be assessed. No research question has been defined.
  2. Quite obvious cherry-picking of evidence. Within the evidence on teachers’ pay, a single US study is cited to suggest that performance pay increases test scores. OECD PISA studies that show no relationship between average student performance in a country and the use of performance-based pay schemes, except in countries with relatively low overall salaries for teachers don’t feature in the evidence check.
  3. Little transparency on the issues that matter. DfE has paraded the usual evidence for teaching phonics. But no evidence is given for the phonics check, including the use of non-words, and how the ‘pass rate’ was determined. Teachers know that synthetic phonics is an important strategy. The evidence isn’t convincing on whether we should teach only synthetic phonics. But it’s the phonics check that really bothers teachers.
  4. Policy doesn’t necessarily grow from the evidence. The policy identified in the evidence check for summer born children is some flexibility in the School Admissions Code. But the evidence raises many more questions, particularly about the impact of early years education, and the possibility of age-weighted test scores.
  5. Sometimes, there really isn’t any evidence. Going back to the creation of the National College for Teaching and Leadership, I wonder whether this is a tiny cry for help. Sometimes policies are foisted upon us. But if policies are developed for ideological, financial, or administrative reasons, let’s not delude ourselves that there is any other reason. And, in the case of the NCTL, let’s use all the evidence we do have about effective teaching and leadership, to evaluate the impact of this policy.

We should welcome this burst of transparency from the DfE. I hope the Select Committee will explore the whole notion of using evidence to inform policy-making.

If we really want to build education policy on strong research foundations, then an open dialogue about the evidence is vital.

Tagged with: 
Educational reform