There is a fundamental problem with getting the hardness of exams right,  and having sat Exams (O's and A's in the 1980s) and also been involved in setting test questions, not for an exam board but within a university department, perhaps I can put another view.


Clearly we cannot set the exact same questions year on year, or for all the folk who boned up using last year's paper, the test is not really a proper test of understanding, but of memory.

So we must have new questions, but still based on the same syllabus, each year.  Now, if you make the question too similar to last year, you still have this problem. If you make it wackily different to anything ever seen before, and then  all the candidates who chose that question only get it less than half right, but do OK on other more familiar questions, was it too hard ?

You now cannot be sure if this is a good or bad crop of students this year, or that bit of the syllabus is not being taught as well as others, or indeed, just because magnetic spin waves is your thing and you think a question on it is a good vehicle to ask what is really a statistics question the students ought to know, it may not be anyone else's and so appears too hard...

Then, what are the exams supposed to achieve - well to show you have read around the subject or been lectured on it, and it has gone in, and you can recall it to order. In this age of the hollow expert who only needs to be good at using the internet to look up the answers, maybe the skill to be examined is also different.


IF the scores are so low, perhaps the questions should be easier ?


You cannot even be sure of this in reverse, as one sets questions for a few years running one sees there is a tendency for the examiners to also have a favourite sort of question too, and they cannot be sure that their ideas of difficulty are not shifting over time. Obviously last years questions look easy, because I saw the answers to them last year.



so deciding that fraction X% is a first class, y% is 2:1, z% is 2:2 is clearly flawed, but maybe as good as we can get. And the GCSE and A levels suffer similarly.

There is a fundamental problem with getting the hardness of exams right,  and having sat Exams (O's and A's in the 1980s) and also been involved in setting test questions, not for an exam board but within a university department, perhaps I can put another view.


Clearly we cannot set the exact same questions year on year, or for all the folk who boned up using last year's paper, the test is not really a proper test of understanding, but of memory.

So we must have new questions, but still based on the same syllabus, each year.  Now, if you make the question too similar to last year, you still have this problem. If you make it wackily different to anything ever seen before, and then  all the candidates who chose that question only get it less than half right, but do OK on other more familiar questions, was it too hard ?

You now cannot be sure if this is a good or bad crop of students this year, or that bit of the syllabus is not being taught as well as others, or indeed, just because magnetic spin waves is your thing and you think a question on it is a good vehicle to ask what is really a statistics question the students ought to know, it may not be anyone else's and so appears too hard...

Then, what are the exams supposed to achieve - well to show you have read around the subject or been lectured on it, and it has gone in, and you can recall it to order. In this age of the hollow expert who only needs to be good at using the internet to look up the answers, maybe the skill to be examined is also different.


IF the scores are so low, perhaps the questions should be easier ?


You cannot even be sure of this in reverse, as one sets questions for a few years running one sees there is a tendency for the examiners to also have a favourite sort of question too, and they cannot be sure that their ideas of difficulty are not shifting over time. Obviously last years questions look easy, because I saw the answers to them last year.



so deciding that fraction X% is a first class, y% is 2:1, z% is 2:2 is clearly flawed, but maybe as good as we can get. And the GCSE and A levels suffer similarly.