Recommender Transformers with Behavior Pathways
Sequential recommendation requires the recommender to capture the evolving
behavior characteristics from logged user behavior data for accurate
recommendations. However, user behavior sequences are viewed as a script with
multiple ongoing threads intertwined. We find that only a small set of pivotal
behaviors can be evolved into the user's future action. As a result, the future
behavior of the user is hard to predict. We conclude this characteristic for
sequential behaviors of each user as the Behavior Pathway. Different users have
their unique behavior pathways. Among existing sequential models, transformers
have shown great capacity in capturing global-dependent characteristics.
However, these models mainly provide a dense distribution over all previous
behaviors using the self-attention mechanism, making the final predictions
overwhelmed by the trivial behaviors not adjusted to each user. In this paper,
we build the Recommender Transformer (RETR) with a novel Pathway Attention
mechanism. RETR can dynamically plan the behavior pathway specified for each
user, and sparingly activate the network through this behavior pathway to
effectively capture evolving patterns useful for recommendation. The key design
is a learned binary route to prevent the behavior pathway from being
overwhelmed by trivial behaviors. We empirically verify the effectiveness of
RETR on seven real-world datasets and RETR yields state-of-the-art performance.