We evaluate human-AI team performance in GuessWhich, for the setting where the questioner is a human and the answerer is an AI (that we denote ALICE). Specifically, we evaluate two versions of ALICE for GuessWhich: 1. ALICE SL which is trained in a supervised manner on the Visual Dialog dataset (Das et al. 2017a ) to mimic the answers given by humans when engaged in a conversation with other humans about an image, and 2. ALICE RL which is pre-trained with supervised learning and fine-tuned via reinforcement learning for an imageguessing task as in<cite> (Das et al. 2017b)</cite> .