We propose the PeerRank method for peer assessment. This constructs a grade for an agent based on the grades proposed by the agents evaluating the agent. Since the grade of an agent is a measure of their ability to grade correctly, the PeerRank method weights grades by the grades of the grading agent. The PeerRank method also provides an incentive for agents to grade correctly. As the grades of an agent depend on the grades of the grading agents, and as these grades themselves depend on the grades of other agents, we define the PeerRank method by a fixed point equation similar to the PageRank method for ranking web-pages. We identify some formal properties of the PeerRank method (for example, it satisfies axioms of unanimity, no dummy, no discrimination and symmetry), discuss some examples, compare with related work and evaluate the performance on some synthetic data. Our results show considerable promise, reducing the error in grade predictions by a factor of 2 or more in many cases over the natural baseline of averaging peer grades.
A closely related problem is the division of cash between a group of partners @cite_11 . Each partner cares selfishly about their share but is supposed to be disinterested about the distribution of the money that he or she does not get. Partners rate the relative contributions of the other partners. With four or more partners, there exist symmetric and impartial division rules. By comparison, whilst our PeerRank rule is symmetric, it is not designed to be impartial. The grades assigned by an agent can definitely influence their final grade.
@cite_11
2091152488
For impartial division, each participant reports only her opinion about the fair relative shares of the other participants, and this report has no effect on her own share. If a specific division is compatible with all reports, it is implemented. We propose a family of natural methods meeting these requirements, for a division among four or more participants. No such method exists for a division among three participants.