Learning Backward Compatible Embeddings
Embeddings, low-dimensional vector representation of objects, are fundamental
in building modern machine learning systems. In industrial settings, there is
usually an embedding team that trains an embedding model to solve intended
tasks (e.g., product recommendation). The produced embeddings are then widely
consumed by consumer teams to solve their unintended tasks (e.g., fraud
detection). However, as the embedding model gets updated and retrained to
improve performance on the intended task, the newly-generated embeddings are no
longer compatible with the existing consumer models. This means that historical
versions of the embeddings can never be retired or all consumer teams have to
retrain their models to make them compatible with the latest version of the
embeddings, both of which are extremely costly in practice. Here we study the
problem of embedding version updates and their backward compatibility. We
formalize the problem where the goal is for the embedding team to keep updating
the embedding version, while the consumer teams do not have to retrain their
models. We develop a solution based on learning backward compatible embeddings,
which allows the embedding model version to be updated frequently, while also
allowing the latest version of the embedding to be quickly transformed into any
backward compatible historical version of it, so that consumer teams do not
have to retrain their models. Under our framework, we explore six methods and
systematically evaluate them on a real-world recommender system application. We
show that the best method, which we call BC-Aligner, maintains backward
compatibility with existing unintended tasks even after multiple model version
updates. Simultaneously, BC-Aligner achieves the intended task performance
similar to the embedding model that is solely optimized for the intended task.