Method 1: Maximum Attention Weights (MAX) Given a token A in a sentence, a selfattention mechanism is designed to assign high attention weights on tokens that have some kind of relationship with token A (Vaswani et al., 2017) . Therefore, for a given token A, a token B that has the highest attention weight with respect to the token A should be related to token A. Our aim is to investigate whether this relation maps to a universal dependency relation. We assign a relation (w i , w j ) between word w i and w j if j = argmax W [i] for each row (that corresponds to a word in attention matrix) i in attention matrix W . Based on this simple strategy, we extract relations for all sentences in our evaluation datasets. This method is similar to<cite> Clark et al. (2019)</cite> , and attempts to recover individual arcs between words; the relations extracted using this method need not form a valid tree, or even be fully connected, and the resulting edge directions may or may not match the canonical directions. Hence, we evaluate the resulting arcs individually and ignore their direction. After extracting dependency relations from all heads at all layers, we take the maximum UUAS over all relations types.