Specifically, we evaluate a state-of-the-art coreference resolution system ) that makes use of ELMo's contextual embeddings on <cite>WinoBias</cite> (<cite>Zhao et al., 2018a</cite>) , a coreference diagnostic <cite>dataset that</cite> evaluates whether systems behave differently on decisions involving male and female entities of stereotyped or anti-stereotyped occupations.