Factorization Machine (FM) is a supervised learning approach with a powerful capability of feature engineering. It yields state-of-the-art performance in various batch learning tasks where all the training data is made available prior to the training. However, in real-world applications where the data arrives sequentially in a streaming manner, the high cost of re-training with batch learning algorithms has posed formidable challenges in the online learning scenario. The initial challenge is that no prior formulations of FM could fulfill the requirements in Online Convex Optimization (OCO) -- the paramount framework for online learning algorithm design. To address the aforementioned challenge, we invent a new convexification scheme leading to a Compact Convexified FM (CCFM) that seamlessly meets the requirements in OCO. However for learning Compact Convexified FM (CCFM) in the online learning setting, most existing algorithms suffer from expensive projection operations. To address this subsequent challenge, we follow the general projection-free algorithmic framework of Online Conditional Gradient and propose an Online Compact Convex Factorization Machine (OCCFM) algorithm that eschews the projection operation with efficient linear optimization steps. In support of the proposed OCCFM in terms of its theoretical foundation, we prove that the developed algorithm achieves a sub-linear regret bound. To evaluate the empirical performance of OCCFM, we conduct extensive experiments on 6 real-world datasets for online recommendation and binary classification tasks. The experimental results show that OCCFM outperforms the state-of-art online learning algorithms.
The most significant difference between OCCFM and previous research is that OCCFM is an online machine learning model while most existing variants are batch learning models. Some studies attempt to apply FM in online applications @cite_25 @cite_44 . But these studies do not fit in the online learning framework with theoretical guarantees while OCCFM provides a theoretically provable regret bound.
@cite_44 @cite_25
2211399443 2474041037
Predicting ad click-through rates is the core problem in display advertising, which has received much attention from the machine learning community in recent years. In this paper, we present an online learning algorithm for click-though rate prediction, namely Follow-The-Regularized-Factorized-Leader (FTRFL), which incorporates the Follow-The-Regularized-Leader (FTRL-Proximal) algorithm with per-coordinate learning rates into Factorization machines. Experiments on a real-world advertising dataset show that the FTRFL method outperforms the baseline with stochastic gradient descent, and has a faster rate of convergence. Real-world item recommenders commonly suffer from a persistent cold-start problem which is caused by dynamically changing users and items. In order to overcome the problem, several context-aware recommendation techniques have been recently proposed. In terms of both feasibility and performance, factorization machine (FM) is one of the most promising methods as generalization of the conventional matrix factorization techniques. However, since online algorithms are suitable for dynamic data, the static FMs are still inadequate. Thus, this paper proposes incremental FMs (iFMs), a general online factorization framework, and specially extends iFMs into an online item recommender. The proposed framework can be a promising baseline for further development of the production recommender systems. Evaluation is done empirically both on synthetic and real-world unstable datasets.