A couple of weeks ago I spent 3 hours with the infinitely patient Lucy Rimmington from Ofqual, trying to get under the skin of Progress 8, the new GCSEs and what it all means for teachers, children and parents. Thanks to her and to several teachers who helped me with questions and queries along that way, I’ve written this blog to try to explain to parents and teachers some of the central issues in our exam system. I should be clear that Lucy was simply explaining processes and language to me and that any opinions or conclusions drawn are mine alone.
My first question centred around what I referred to as “norm referencing” and what is more correctly termed “comparative outcomes.” Is it true, I asked, that the proportion of pupils passing GCSEs is set in advance, regardless of criteria or achievement? The answer is yes, well sort of. Exam boards can appeal the boundaries for individual subjects if they feel that the cohort was unusually able, but this is very rare and difficult to prove. So, in reality, it’s a yes. Already, it has been decided that this year, around 2/3rd of pupils will achieve a grade 4 or a C or above. In fact the proportions of 1s, 4s and 7s have been agreed in advance, regardless of what children do on the day. It means that although the government say that they have made exams harder and more rigorous, it hardly matters because the number of children passing will remain the same. So much for “raising standards.” It’s a clever PR ploy. Journalists and parents can look at the paper, say “ooh that’s hard,” look at the pass rate and assume that things are getting better. In reality, it simply means that children don’t have to score as highly to get the same grade. One might shrug if it weren’t for the pressure that the harder content puts on teachers and pupils and the extra worry it creates.
It seems mad, if you assume that having criteria means that children who reach it are rewarded with a corresponding grade, to find out that we have a system in which no-one can really improve – at least they can, but it must always be at the expense of someone else. But on the other hand, fixing the results in this way protects children from a catastrophic drop in results when government ministers have fiddled with the exam system. It creates stability. The alternative is what we saw with KS2 SATs last year – a criteria based system – where the % of children meeting expected standard fell from 80% to 53%. A drop like that at GCSE would be disastrous. As Lucy said, it seems like the fairest option in a flawed system.
This might not matter, if it were not for the fact that it creates a complete disconnect between reality and the expectations of government and Ofsted. If results are fixed in advance, then how can schools improve? If they are to be judged on data, how can it be fair to be expected not to “coast” when in fact the system is set up to ensure coasting? The fact is that every school that improves their data is doing so at the expense of another’s results. We are pitched against each other – child against child, teacher against teacher, school against school – in a fight to protect our position and to try to improve it, knowing full well that our Outstanding comes at a cost for someone else. No wonder so many schools are starting to select by excluding pupils who may skew their data. No wonder they are looking for ways to secure advantage over others. It actually makes the idea of sharing best practice an act of folly. Why should we collaborate when doing so could hurt our students’ chances of success?
The system is predicated on two central beliefs from Ofqual. One is that people don’t get more intelligent. Therefore, it is reasonable to assume that what they achieve at 11 is a fair indicator of what they will achieve at 16. So the proportion of pupils who will achieve 4s and above at GCSE is set in line with SATs results. All the research into growth mindsets and the evidence from MENSA that in fact human IQ is improving, is largely ignored. They point to the fact that data suggests that children move in a trajectory from KS2 to GCSE that is largely reliable. But that doesn’t allow for the possibility that our data tracking systems for the past twelve years or so have assumed this trajectory. That setting GCSE targets in line with KS2 results might create self-fulfilling prophecies where children achieve the grade that the adults around them expected them to. It doesn’t allow for the possibility that this thinking places a ceiling on achievement and potential.
The other belief, is that for some reason, teachers are not to be trusted – that they will cheat. And so the exam system has to be cheat proof. This lack of trust drives much decision making across the system. For example, reducing coursework; the number of resits; disallowing iGCSE from state school league table results; reducing the number of reviews (or remarks as teachers call them – a semantic tic that apparently irritates people at Ofqual) – all these things have been done to stop people “gaming” the system. Yet let’s take the last one – reviews. Last year, Ofqual announced that the right to have papers “reviewed” (or remarked) would be restricted. They claimed that this was to stop private schools from gaming the system by entering whole cohorts of students for review because they could afford it. By stopping that game, they made it harder for all the genuine applications to be successful. Yet IN THE SAME YEAR, their own research found that 50% of English Literature candidates and almost the same proportion of History candidates got the wrong mark. They publish evidence that exam marking is unreliable at the same time as they make it harder to have the paper reviewed. I don’t really need to say any more on that do I?
Similarly in an attempt to stop pesky teachers from “teaching to the tests” that will change the lives of the pupils they care about, they take care to ensure that the exams are increasingly unpredictable. That the questions cannot be guessed and that they will be written in such a way that children have to think laterally and apply their knowledge. On the one hand, that’s no bad thing – flexibility of mind is an important skill. On the other, when an exhausted child is sitting up to thirty exams in a three week period, it’s a farce to expect them to have the clarity of thinking that the test is designed to extract. At least, with the grade boundaries being fixed, it hardly matters what’s on the paper I suppose.
Having said all of this, schools do themselves no favours in challenging this perception of gaming. Take the ECDL (European Computer Driving Licence). Under Progress 8, schools are credited for a broader number of subjects than before under the old 5A-C accountability measure (for parents that means that schools used to be judged according to the number of pupils who achieved 5 grade As – C, now, they are judged across 8 subjects and the measure is not who gets Cs or above – but how much progress they make from SATs). That would seem fairer if it were not for the fact that the GCSE grades are set in line with SATs results. That means children are NOT EXPECTED to make progress that exceeds predictions at all by Ofqual, but they are by Ofsted. Indeed “good” progress is that which exceeds expectations – anything else is simply expected. And if they do make better than expected progress, someone else must do worse. I know, I know. Anyway, back to the ECDL.
There are three baskets for Progress 8. Schools must enter all pupils for English and Maths – the first basket – and these count for more points if you like than other subjects. They must also provide pupils with access to subjects in the second basket which are largely the EBacc subjects – Languages, Sciences, Humanities etc. The third basket is “other” this could include Arts subjects, vocational subjects and a host of others, like the ECDL. Although Ofqual states that these third basket subjects should involved a minimum number of hours of study, the fact that most pupils are already reasonably computer literate means that the ECDL can be taught and assessed in an intensive week or across a term. This Datalab blog post explains the gaming of ECDL in more detail. Putting all pupils in for ECDL could raise a school’s Progress 8 score by as much as 0.2 – a significant gain. So many schools, in line with the advice given to them by Regional Schools Commissioners (yes, that’s right – senior civil servants are actively encouraging schools to game the system) and MAT trustees, are now entering pupils for an assessment that has little value to them, but great value to the school. It’s no surprise then that the government have now announced that the ECDL will not count towards Progress 8 from 2019. And even before that, there is pressure on Ofsted to look carefully at how schools are creating their Progress 8 scores and whether their choices are made in the best interests of pupils. It’s another game of cat and mouse, and what it does is undermine further the credibility of the profession as well as placing more ethically minded schools at risk. Strategies such as these, while understandable, explain why it is that organisatons like Ofqual view us with suspicion.
It seems to me that we are stuck in a system of mistrust where each side is attempting to out manoeuvre the other in order to protect either themselves or the systems they create. And children get lost in the cross fire. One of the biggest problems leading to unreliability of exam marking is the poor quality of examiners. Struggling to recruit enough examiners to serve an enormously overloaded exam system, boards are turning to unqualified and inexperienced examiners, some of whom have never taught. Even without entering into the matter of subjectivity, this quality issue alone serves to undermine the validity of the whole system. Why are Ofqual and government not leaning on exam boards to ensure a basic level of qualification and experience for their examiners? They’d have to pay more; their profit margins would reduce. Why are government not giving experienced teachers incentives to examine – reduced timetables for example – and subsidising schools to encourage better quality examiners to come forward? Why is the cost of reviewing not met by the exam board if the quality of their service is so poor?
It seems to me that as long as Government allows the population to labour under the misapprehension that we live in a system of meritocracy where all can improve with effort, schools are in an impossible bind. It’s time for some open minded debate about how we best hold our schools to account; how we best assess our children and how we best communicate these aims to parents. I remain hopeful that there is a significant role to be played here by the Chartered College of Teachers to broker a relationship between the DfE, teachers, unions, Ofsted and Ofqual in a way that develops more realistic expectations, fairer systems of assessment and more open communication. What is clear, is that everyone is doing what they think is the best thing. Ofqual are trying to create a stable and reliable system. Ofsted are trying to create a fairer system of inspection. Teachers are trying to get children the best possible outcomes they can. All are working towards similar goals – to create an education system that equips young people for a successful future. But by working in competition and suspicion, they are undermining their stated purpose and the system is creaking under the pressure. We need to start again.