[1]} extend WEAT to sentence encoders, calling it the Sentence Encoder Association Test (SEAT).
For a target term and its attribute, they create artificial sentences using generic context of the form "This is [target]." and "They are [attribute]." and obtain contextual word embeddings of the target and the attribute terms.
They repeat [2]}'s study using these embeddings and cosine similarity as the association metric but their study was inconclusive.
Later, [3]} show that cosine similarity is not the best association metric and define a new association metric based on the probability of predicting an attribute given the target in generic sentential context, e.g., [target] is [mask], where [mask] is the attribute.
They show that similar observations of [2]} are observed on contextual word embeddings too.
Our intrasentence CAT is similar to their setting but with natural context.
We also go beyond intrasentence to propose intersentence CATs, since language modeling is not limited at sentence level.
