Grading strategies for oral exams Several recent questions on oral examinations have caused me to reflect on whether or not this particular examination tactic would be worth further investigation for my courses.

From a practical standpoint, I am stumped as to how one would go about grading an oral exam. For example, one of the key benefits of an oral exam from the student's perspective seems to be that the exam can be optimized in real time according to the student's performance; however, a well-prepared student may end up getting a relatively more challenging exam, while the ill-prepared student may receive a relatively easier exam.

All of this has me wondering:


  How would one go about grading an oral exam in a manner which is both fair and accurate?
In all kinds of assessments, but especially oral exams, carefully prepared rubrics can help increase consistency and decrease subjectivity. From "A short guide to oral assessment":


  One of the advantages of oral assessment is that it can often be
  marked quickly on the spot. To support this, the use of a marking
  guide or rubric of some sort is usually essential. The use of rubrics
  in oral assessment has many benefits:
  
  
  It provides assessors with a common reference point for their
  judgments
  It reduces the likelihood that judgments will be based on
  extraneous factors
  Providing students with the marking guide in advance helps them
  understand the nature of good work and helps them to evaluate
  the quality of their own work in the assessment
  It provides a basis for peer evaluation/feedback
  It makes marking more efficient
  It provides a useful framework for feedback to students.
  


Further details depend on the format of the exam, and on whether you want to include things like presentation skills in the grade or not.

In [1], there is a predetermined set of questions that are graded as follows:


  Students are told that they will be asked a series of questions
  that are designed to evaluate their understanding of the
  material. Students are instructed to “think aloud” in order
  that their thought process can be observed (9). The grading
  system is explained and they are shown a scoring sheet. Four
  scores are possible for each answer: 3, 2, 1, or 0. Students
  answering a question correctly and without prompting earn
  3 points. Each prompt a student receives results in a deduction
  of 1 point. Mistakes made along the way have no consequence
  on the grade if they are self-corrected. Following
  the questions in the order they appear on the scoring sheet
  allows for consistency in the administration of the examination.


In [2], the content of the exam varied with each student. Therefore, 


  Developing a rigid rubric for an oral
  examination was not feasible when each student individually
  determined the content. This required the evaluation criteria to
  remain roughly defined to facilitate evaluation of all students,
  no matter which reaction they chose. The rubric shown in
  Table 2 was used to guide the evaluators when grading.
  Presentation skills included oral and nonverbal communication
  (such as writing a reaction mechanism) and delivering
  information in an organized manner. Student knowledge
  about their chosen subject was evaluated partly through
  reviewing the content they had prepared for the session and
  whether they presented the required information. The ability of
  each student to answer questions was used to assess depth of
  knowledge and the extent of the research undertaken and to
  gauge how capably a student could synthesize information
  quickly, problem-solve, and think critically. For example, one
  student mentioned a recent innovation of their selected
  reaction being undertaken via solid-phase synthesis. Their
  knowledge regarding solid-phase techniques was subsequently
  assessed through further questioning to determine the depth of
  their understanding.
  
  To limit inconsistencies and subjectivity, which are reported
  problems in oral testing, two of the course instructional
  staff were present for each examination
  
  


In the "poster exam" format described in [3],


  The grading of the poster exam occurs in two parts. First
  students are graded on the content, organization, and design
  of the poster itself, the oral presentation of the poster, and
  their ability to answer questions on the poster. The students
  receive a group grade, which we chose to score as 60% of
  the exam. The detailed breakdown of the poster evaluation
  is shown in Figure 1.
  
  
  
  ...
  
  Individual knowledge of the broad subject matter is
  probed in the second half of the test with an individual oral
  examination administered to each student in the poster group.
  Questions are drawn from a database constructed by the
  course designers and instructors, and labeled according to
  degree of difficulty. After each poster exam the database is
  refined and enlarged. The examiners are asked to draw questions
  randomly from the database and score the answers on a
  three-point scale of outstanding, acceptable, or unacceptable.
  Each student’s response is scored and entered on an answer
  sheet, which the students do not see, and returned to a folder
  that remains with the group. The next examiner then has access
  to the folder and does not repeat the prior questions asked.
  In this way each student receives at least two different oral
  exams, and most are given three. Examiners tend to conduct
  this aspect of the poster exam in one of two ways: some ask
  each student a different question, while others give the same
  question to all students.




[1] Roecker, L., 2007. Using Oral Examination as a technique to assess 
student understanding and teaching effectiveness. J. Chem. Educ, 84(10), p.1663.

[2] Dicks, A.P., Lautens, M., Koroluk, K.J. and Skonieczny, S., 2012. Undergraduate oral examinations in a university organic chemistry curriculum. Journal of Chemical Education, 89(12), pp.1506-1510.

[3] Marino, R., Clarkson, S., Mills, P.A., Sweeney, W.V. and DeMeo, S., 2000. Using poster sessions as an alternative to written examination—the poster exam. J. Chem. Educ, 77(9), p.1158.