A Truthful Owner-Assisted Scoring Mechanism
Alice (owner) has knowledge of the underlying quality of her items measured
in grades. Given the noisy grades provided by an independent party, can Bob
(appraiser) obtain accurate estimates of the ground-truth grades of the items
by asking Alice a question about the grades? We address this when the payoff to
Alice is additive convex utility over all her items. We establish that if Alice
has to truthfully answer the question so that her payoff is maximized, the
question must be formulated as pairwise comparisons between her items. Next, we
prove that if Alice is required to provide a ranking of her items, which is the
most fine-grained question via pairwise comparisons, she would be truthful. By
incorporating the ground-truth ranking, we show that Bob can obtain an
estimator with the optimal squared error in certain regimes based on any
possible way of truthful information elicitation. Moreover, the estimated
grades are substantially more accurate than the raw grades when the number of
items is large and the raw grades are very noisy. Finally, we conclude the
paper with several extensions and some refinements for practical
considerations.