We use two dependency relation extraction methods to extract dependency relations from each self-attention heads of BERT and RoBERTa. The first method-maximum attention weight (MAX)-designates the word with the highest incoming attention weight as the parent, and is meant to identify specialist heads that track specific dependencies like obj (in the style of <cite>Clark et al., 2019)</cite> .