|
Please note that this is a general article; it is not an official document from, nor does it represent the detailed arrangements for, any particular examination Group. It relates only to Advanced GCE. All of the deliberations of all Groups are covered by the Code of Practice from QCA, the Qualifications and Curriculum Authority, so are therefore broadly similar.
It is written for those of you who ponder on what goes on in the marking process- perhaps because you wonder why the results seem to take so long to appear, or ask whether the criticism sometimes vented in the media is justified. So, here is a general outline of the proceedings which all Exam Groups will go through. Each Group has its own technical term for the various processes - you will get the drift.
Summary
Items in bold are elaborated below the table.
Time Process Outline of activity EXAM DAY
3 d After Exam Pre-coordination The Chief Examiner (or equivalent) meets with Team Leaders and Subject Officer(s) to discuss the markscheme in the light of having provisionally marked some papers. 4 d AE Coordination The Chief and the whole team of Assistant Examiners meet to discuss and finalise the markscheme, and mark some standard scripts. Assistant Examiners are asked to justify their marking of the standard papers. 6 - 21 d AE Marking During this time the work of all examiners is carefully monitored. 2 - 4 wk AE Checking and data input The marking is checked at the Board, the marks are keyed in and the statisticians get to work. 4 wk AE Awarding Papers are scrutinised, archive scripts consulted, standards discussed, and grade boundaries set. This takes several days. 5 wk AE until result day Analysis and data processing The Board has a huge quantity of data to process, check and print. RESULTS
The full-time officers of the Board are also going about their normal administrative tasks whilst the above is happening. The question writers and other people involved will also be thinking about the next session and the one after that, not to mention the 2001 exams when the syllabus changes.
So, the exams are done, the schools have sent off the papers, and the Examinations Officer(s) get a rest as well. Now the examining work begins....
Coordination meetings
For a paper with, say, 10,000 entries, a team of perhaps 30 or more examiners will be needed. This will be split into teams of 6 or so, each under the direction of a Team Leader, and all will be responsible to the Chief/Principal examiner who is running the paper.
The pre-coordination meeting.
At this meeting the Principal will discuss with Team Leaders in the light of having marked some papers the interpretation of the markscheme that came with the paper when it was initially written. The working markscheme that results from this meeting is the one used at the Coordination meeting. It will determine the standard of the paper, and although discussion with the whole examining team will produce extra scoring points or alternative interpretations, the standard of the paper will not be changed. The Team Leaders will then mark some standard scripts in order to be able to explain the interpretation of the markscheme to their teams.
The coordination meeting.
This meeting has one function - to be fair to the candidates.
This fairness is achieved by the Principal ensuring that every examiner goes out of the room intending to do the same thing. All of this might require a great deal of discussion, and perhaps for some examiners a realisation that marking examinations is not the same as marking homework or whatever. The scoring points have to be clearly learnt, and equally important is the understanding of the latitude allowed in interpretation of what candidates write. The facts of chemistry might not be disputed (though this is not always true - see my article on the effect of alkalis on solutions of metal ions) but the ways in which the facts can be expressed are legion. There is no opportunity for feedback or discussion with the candidate, so examiners have to make decisions. These are quite straightforward in most cases, but professional judgment is necessary from time to time. That is why exams have to be marked by chemists, rather than clerks equipped with a markscheme. The principle must be that correct chemistry scores. Again QCA representatives are often present.
Marking
It is vital that all the consumers of examinations, that is teachers, candidates, employers, candidates, candidates' parents, candidates, the DfEE, candidates, and the media, have confidence that examiners are doing their job.
The marking of examinations is a closely monitored process, and should an examiner fail to satisfy at any stage then that examiner's scripts will be re-allocated and partially or wholly re-marked. This can even happen after the marking period proper has passed, since a re-mark can be ordered as a consequence of the meeting at which the team leaders and principals discuss each examiner's performance. Examiners whose work is wholly re-marked are not re-appointed; those with partial re-marks undergo training and have a reduced load in the next session. The principles of monitoring are defined in the Code of Practice from QCA.
Although the details will vary from Group to Group, there will be three lots of monitoring. Team leaders are monitored by the principal, who is monitored by his immediate superior.
- The first sample will be within two days of the coordination meeting, and is used to ensure that the examiner has learnt the markscheme and is interpreting it correctly. The team leader remarks the scripts and will offer detailed advice to the examiner, who will re-mark all the scripts seen thus far in the light of that advice. If the results from these first ten are seriously adrift, a second 'first' sample will be requested.
- The second sample will be perhaps a week-and-a-half later; maybe around 40 scripts, from which the team leader will select a random sample. If this review is satisfactory, and it nearly always is, then the examiner will be allowed to submit the marksheets for the scripts marked thus far. If it is not, then the team leader has to decide whether to offer further advice and ask for more samples, or whether to advise a re-mark by another examiner. This is not a common outcome.
- The third sample will be sent near the end of the marking period, and is treated in the same way as the second one. It allows the team leaders to check for any drift in judgments over the marking period.
As stated above, there will be a meeting of all team leaders with the Chief and Principal examiners to check on the performance of all of the examiners, to grade their performance, and to ensure that any problems are addressed. Examiners have about three weeks to mark their allocation - it is important to avoid fatigue, so a reasonable time has to be allowed so that examiners are not too pressured.
Examiners then send the marks for processing, and the scripts back to the Board. These will be clerically checked to ensure that the transcription and addition of marks is error free.
The Award
Although standards are closely monitored year-on-year, it is not possible to make the grade boundaries for exams in one year the same as those in a succeeding year, or indeed to have the same grade boundaries for each paper in a single session. The award is the process that gives the grade boundaries from the raw scores, and which then determines the score in the uniform mark scheme (UMS).
The award is attended by all the Principals, the Chief, the Chair of examiners, officers of the Board, and representatives from QCA.
Consideration of a given paper will begin with an oral report - on the general standard, points of difficulty, bits that were answered well, bits that were answered badly.
The Award proceeds with a discussion of the standard required to achieve a grade E. The process is the same whatever the grade concerned, but E is the first. The Principal and his team will have come up with a range of marks that they think might be an E, so scripts are taken and scrutinised. The team might start at the high end of the suggested range and come down until they begin to think that a particular score is not of the required standard; then they go from the lower boundary and work up until the E grade seems to be in sight; then they look at the boundary and come to a conclusion on the actual mark that gets an E. Many scripts will be seen; and they will be compared with archive scripts from previous examinations to ensure comparability of standards.
Once this has been done, the A boundary is found in a similar way; and then the B. The other boundaries are done by interpolation.
During the award (several days of it) huge piles of scripts will have been scrutinised, jumbled up, and will have to be re-sorted. Spare a thought for the clerks involved.
The grade boundaries are then reported, and the processing of raw scores into final grades begins. The statistical and processing activities are immense, and the reason that exam results seem to take a long time to come out is because of the amount of data processing and checking needed. Believe me, the time is pretty tight even so, and the most heinous crime an otherwise brilliant examiner can commit in examining terms is to be late with the marksheets.
The results.And then, processing and a final check over, you get the results - in the words of my old school song, 'some to failure, some to fame'. Whether elated or disappointed, I hope that the candidates never feel that the Groups and their various officers take things lightly or casually. They do not; examining is a highly regulated affair, because behind every pile of scripts is a collection of people and their hopes and ambitions. That is never forgotten.