Transformer [1]}, whose strong language modeling ability has been demonstrated on a variety of tasks [2]}, [3]}, [4]}, however, is relatively under-explored for personalized natural language generation.
Since IDs and words are in very different semantic spaces, it would be problematic to directly put them together for attention learning, because by doing so, the IDs are treated as words, but the IDs appear far less frequently than the words.
For example, a paragraph of review (and thus hundreds of words) on e-commerce platform only corresponds to a single pair of user ID and item ID.
As such, the IDs may be regarded as out-of-vocabulary tokens, to which the model is insensitive.
As shown in Fig. REF (a), when generating an explanation for a user-item pair, standard Transformer relies heavily on the special \(<\)bos\(>\)  token instead of the user or the item.
This would result in identical explanations over different user-item pairs (see USR score in Table REF ), deviating from our personalization goal.
