In the paper, Hierarchical Attention Networks for Document Classification, the authors use attention to compose words to sentences and then sentences to a document representation. They make use of a context vector $u_w$ to compute the attention weights for the annotation of each word in the sentence. The paper states in section 2.2, The context vector $u_w$ can be seen as a high level representation of a fixed query what is the informative word over the words. The word context vector $u_w$ is randomly initialized and jointly learned during the training process.This implies that the context vector is independent of the sentence input. It remains the same for every sentence and is learnt as a parameter of the neural network. If this is the case, how will $u_w$ accurately provide attention weights for words in a random sentence, given that sentences are so diverse in meaning.I'm not understanding the working of $u_w$ since it is independent of the sentence input. Please explain.
Attention using Context Vector: Hierarchical Attention Networks for Document Classification
machine learning;neural network;deep learning