Has any research been done into the marks awarded by markers using different strategies for exams with multiple questions? I think anyone who's ever marked an exam with more than one question or section has wondered how to attack it, i.e.

Option 1) Mark all of the questions in a single script before moving onto the next student's script; or

Option 2) Go through each script and mark all the question 1s, then 2, and so on.

Idle rumination gives good points and bad points for each side (and possibly some more approaches), for example 1) means you can get through it faster since you're in 'question 1 mode', then 'question 2 mode', ... (i.e. less time spent switching your brain between different topics), and is likely more fair in the sense that you can consider all the answers to question i at the same time and make sure marks are distributed in a reasonable fashion.

On the other hand, marking a script at a time means you get used to a student's hand-writing, and may perhaps give benefit-of-the-doubt (or not) depending on the understanding that the student has shown previously in the same paper (but hopefully not on the basis of external bias from knowing anything about the student aside from what is on the script).

It would be interesting to see whether one technique or another tends to correlate with higher marks given by a marker, or fewer challenges once the student receives the script back (in cases where that happens), for example.

A brief search via scopus, arXiv and popular general-purpose search engines does not turn up anything particularly relevant. Does anyone here know of any published research on the matter?
The question we'd like to answer is:

How should exams with multiple essay-type questions be graded?

Note: It isn't possible to answer this definitively due to unavoidably large variations in grader performance and exam structure.

Each course topic, exam, and set of graders is going to be unique and your own experience and judgment are likely to yield better results than any prescription of what you should do.
This is a large part of the reason both teaching and assessment are challenging and generally require a lot of experience, testing, and adjustment to get right.



While you're own views should necessarily be more relevant for your teaching situation, here is One perspective to consider:

Ideally, we strive to consistently award the same amount of points for the same level of written work. However, it is likely that our impressions about the abilities of individual students (from lectures, office hours, etc.) actually have a negative impact on the overall consistency and reproducibility of the results when marking written exams.

Whitfield et al. compared oral evaluation results to written exam grades and found that facilitator impressions of student knowledge overestimate the performance of the weakest students.


  [Whitfield, 2002] "... there is little overlap in the factors which
  account for facilitator’s ratings and actual student performance ...
  Our conclusion from these studies is that an assessment given by a
  facilitator for student knowledge is not likely to be useful."


The take-away is:

knowing which student's answer you are grading will make the range of scores wider (lower reliability) for the same work, and generally lead to higher grades overall.

For the most reliable grading:

the grader should mark answers non-consecutively and randomly from the entire set of exams in order to:


reduce the chance of identifying the student that provided the answer
reduce the comparative influence from one marked answer to the next




The following articles look promising for further investigation, however, I don't have convenient access to these texts, so they are offered without comment:

"The interrelations of features of questions, mark schemes and examinee responses and their impact upon marker agreement," Assessment in Education: Principles, Policy &amp; Practice, Volume 18, Issue 3, 2011.

"Towards a model of the judgement processes involved in examination marking," Oxford Review of Education, Volume 36, Issue 1, 2010.

"Double-marking students' work," Assessment &amp; Evaluation in Higher Education, Volume 19, Issue 1, 1994.

"Responses to written work," Educational Review, Volume 30, Issue 2, 1978. 

"A further study of the reliability of English essays," British Journal of Statistical Psychology, Volume 7, Issue 2, pages 65–74, November 1954.



Related SE questions:

double marking
---- keeping focused
---- avoiding bias