PISA – what’s the point?

Blog
15 January 2017 by Jill Stokoe
Broken pencil and test sheet
The Programme for International Student Assessment (PISA) aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-olds. But are the results meaningful?

The triennial international survey is run by the Organisation for Economic Co-operation and development (OECD). In 2015 over half a million students, representing 28 million 15-year-olds in 72 countries and economies, took the internationally agreed two-hour test. Students were assessed in science, mathematics, reading, collaborative problem solving and financial literacy.

The latest PISA tables show that England hasn’t improved its scores for almost a decade, although they haven’t fallen, which is the case in some other high-performing countries. There has been no significant change in England’s average maths, science or reading scores since 2006.

Although we remain far behind some countries in maths, we are average for the Organisation for Economic Co-operation and development (OECD). Seven south-east Asian countries are more than 20 points ahead of England in the table.

White working class pupils are not doing worse than ethnic minority working class pupils. White pupils score between 25 to 40 points more in PISA’s science, maths and reading tests than their black and Asian peers. It is advantaged white pupils that both the white working class and ethnic minority pupils perform significantly worse than.

Some highlights, which won’t have been a surprise to anyone who has been listening to what we have been saying time and time again are:

  • Our headteachers use data more than their colleagues abroad – 61% of head teachers in England ‘regularly use pupil’s performance results to develop their school’s educational goals’ – just 18% on average across the top 10 countries.
  • Lack of physical infrastructure is a problem for headteachers – half of head teachers said that a lack of good quality physical infrastructure was hindering learning.
  • Teacher shortages continue to impede learning – across the OECD on average English pupils are 15 percentage points more likely to be taught in schools where headteachers see recruitment as a problem. Only 2 high-performing countries complain of teacher shortages more than us – Japan and China.

It is shocking that almost half of secondary school students in England are taught in schools where the headteacher believes that staff shortages are hindering learning. Whether they believe it is a crisis or not, the Government must address the growing problem of teacher supply and retention.

Politicians around the world should avoid jumping to conclusions about the PISA results. The Trends in International Mathematics and Science Study (TIMSS) test results published the week before PISA were useful because those tests are designed to assess what students are learning in their schools in maths and science, whereas the PISA tests are based around OECD’s idea of what workforce skills students will need in the future. However, neither cover the full range of knowledge and skills our young people need.

Many countries achieve similar results to England and the OECD itself warns against a league table mentality. Research also suggests that national cultures have more impact on test scores than school systems. [1]

And Government clearly has selective hearing since it chooses to take notice of the PISA league tables, but ignores the OECD’s views on selection.

PISA, if it tells us anything at all, can only tell us about the education policies of the past. We caution UK governments against leaping into further changes which could increase workload and therefore the numbers leaving teaching, and to take time to evaluate the many and rushed through education reforms that have happened recently.

Isn’t it about time that the Government requested the OECD carry out a country report on England, so that we can find out how the Government’s education reform programme is really working? Scotland and Wales have already done so, so what’s the problem?


[1] See e.g. Jerrim J. (2014) Why do East Asian children perform so well in PISA? An investigation of Western-born children of East Asian descent, Department of Quantitative Social Science Working Paper No. 14-16, London University Institute of Education

Tagged with: 
Assessment