For the color reference task, we use the data collected by<cite> Monroe et al. (2017)</cite> from human play on the color reference task through Amazon Mechanical Turk using the framework of Hawkins (2015). Each game consists of 50 rounds played by a human speaker and listener. In each round, the speaker describes a target color surrounded by a context of two other distractor colors, and a listener clicks on the targets based on the speaker's description (see Figure 1a) . The resulting data consists of 46, 994 rounds across 948 games, where the colors of some rounds are sampled to be more likely to require pragmatic reasoning than others. In particular, 15, 516 trials are close with both distractors within a small distance to the target color in RGB space, 15, 782 are far with both distractors far from the target, and 15, 693 are split with one distractor near the target and one far from the target.