blackblackRecent, unbiased learning to rank approaches aim to learn reliable relevance signals from noisy and biased clickthrough data either with the help of click models or blackthrough counterfactual methods [1]}, [2]}, [3]}, [4]}.
These models typically require a large quantity of clicks, blackwhich makes them difficult to apply in systems where click data is highly sparse due to blackthe personalised nature of recommendation systems [5]}, [6]}.
On the other hand, these models are designed to tackle position/presentation bias in IR systems. blackHowever,
although blackboth position and presentation bias can influence the collected feedback from a deployed recommendation system, recommendation systems mostly suffer from selection bias [7]}, [8]}, [9]}.
blackIn this paper, we investigate the origin of selection bias in recommendation systems and provide a novel evaluation method that takes into account the role of the deployed recommendation system.
