Vertical Semi-Federated Learning for Efficient Online Advertising

As an emerging secure learning paradigm in leveraging cross-silo private data, vertical federated learning (VFL) is expected to improve advertising models by enabling the joint learning of complementary user attributes privately owned by the advertiser and the publisher. However, the 1) restricted applicable scope to overlapped samples and 2) high system challenge of real-time federated serving have limited its application to advertising systems. In this paper, we advocate new learning setting Semi-VFL (Vertical Semi-Federated Learning) as a lightweight solution to utilize all available data (both the overlapped and non-overlapped data) that is free from federated serving. Semi-VFL is expected to perform better than single-party models and maintain a low inference cost. It's notably important to i) alleviate the absence of the passive party's feature and ii) adapt to the whole sample space to implement a good solution for Semi-VFL. Thus, we propose a carefully designed joint privileged learning framework (JPL) as an efficient implementation of Semi-VFL. Specifically, we build an inference-efficient single-party student model applicable to the whole sample space and meanwhile maintain the advantage of the federated feature extension. Novel feature imitation and ranking consistency restriction methods are proposed to extract cross-party feature correlations and maintain cross-sample-space consistency for both the overlapped and non-overlapped data. We conducted extensive experiments on real-world advertising datasets. The results show that our method achieves the best performance over baseline methods and validate its effectiveness in maintaining cross-view feature correlation.


INTRODUCTION
Immediate auction (such as RTB [26] and the oCPC [27]) is the predominant trading mode of online advertising, where advertisers bid in real-time for every individual impression of the ad platform. The bidding price and the final ad ranking are both deeply influenced by the click-through rate (CTR) and the conversion rate (CVR). They have direct impacts on the final revenue for both entities and play key roles in advertising systems. Prior works have made significant progress in building accurate and efficient models for the two estimation tasks. But they mainly focus on the model design aspect and do not consider the data limit of a unilateral party (e.g., the advertiser or the platform). We emphasize that, as two symmetric roles in the advertising system, user features collected by the advertiser and the platform is intrinsically complimented and confidential.
The advertiser knows the "deep but narrow" interests while the platform knows the "wide but shallow" interests. Taking a smartphone user as an example, the advertiser (e.g. a game company) knows his usage and consumption activities on all its products and the platform (e.g. a social media platform) knows his browse and click history on all kinds of ad materials, products, and other contents impressed on its platform. The joint use of both parties' data features could generate more accurate CTR and CVR models, thereby increasing revenue for both organizations. However, both product consumption data and content interest data are core business assets of these companies and cannot be shared directly due to commercial confidentiality and privacy concerns [22].
As an emerging distributed learning paradigm for preserving privacy, vertical federated learning [9] (VFL) has been successfully applied in healthcare and finance to solve secure feature-sharing problems [3,21]. Inspired by this, recent works [11,24] started to adopt SplitNN and VFL in the recommendation and advertising problems. Although such explorations seem promising in theory, they face strict challenges in real industrial applications. Here we highlight two main challenges in applying splitNN-based VFL to advertising systems: • Restricted applicable scope in sample space: The training and inference of a VFL model can only be conducted on overlapped samples, which is usually a small portion of the whole dataset at each party. The performance drop caused by the reduced sample size would probably be greater than the performance lift led by the expanded number of features via federation. Besides, the overlapped sample naturally shows some special semantics in business logic to distinguish them from the nonoverlapped ones, thus inevitably incurring distribution shifts (e.g., a man buying both beer and napkin is probably a father who can not represent all beer buyers). These contradictions may render the data federation meaningless. • Difficulty of real-time federated serving: Compared to singleparty models, the inference process of VFL models brings extra time costs (caused by cross-agency feature transmission and security enhancement operations) and poses new system design challenges (due to the inconsistent network conditions and computational power of different parties). Such complex conditions are acceptable for offline model training but nearly infeasible for online model inference, since the federated inference system must meet the high throughput and real-time latency requirements of advertising systems (million-wise peak QPS, 10 ∼ 100ms process time per request [26]). These obstacles may cause the federation infeasible or cost too much [9,11].
To overcome these restrictions, We investigate a new learning paradigm that requires the participant to jointly utilize both the overlapped and non-overlapped samples but meanwhile be rid of cross-agency model inference. We term this learning fashion as "Semi-VFL" (Vertical Semi-Federated Learning), since only the training involves federated learning and the inference doesn't. Similarly, we identify the original VFL setting as "Full-VFL" and the unilateral-party learning setting as "No-VFL". We advocate Semi-VFL as a lightweight, practical solution to reap the benefits of data expansion. It is more efficient than No-VFL for better performance, and lighter than Full-VFL for easy deployment. Notably, there are two key points in implementing Semi-VFL: • Alleviating field missing: Only the overlapped samples have additional attributes from the passive party, and even though, they are banned to use in the inference stage. Maintaining and generalizing knowledge implied in these attributes is challenging but crucial for Semi-VFL to outperform No-VFL. • Adapting whole sample space: The final test samples comes from the whole sample space, and only the active party's data attributes are available. Compared to Full-VFL, it's important for Semi-VFL to jointly consider both the overlapped and nonoverlapped sample distribution to learn a unbiased model.
In this paper, we propose a joint privileged learning framework (JPL) to learn a single-party model reaps the privileged knowledge of the federated model and the non-overlapped data to make better predictions for the whole sample space but only conditioned on partial feature space. Specifically, we use both the overlapped and non-overlapped samples in an End2End fashion to consider the whole sample distribution, and explicitly learns the correlation between two-party's feature space to maintain federated knowledge. Our contributions can be summarized as follow: (1) We identify the vertical semi-federated learning setting for advertising systems as an important research problem. (2) We propose an effective privileged distillation framework JPL to implement Semi-VFL and achieves expected results. (3) We conducted extensive empirical evaluations of the proposed approach on real-world advertising datasets and achieves significant performance lift compared to baselines.
Our paper focus on the CTR and CVR prediction modeling in the two-party vertical federated learning problem of the advertiser and ad platform. Methods discussed here can be applied in other VFL scenarios with tabular data, such as LTV tasks and vertical federated recommendation problems. It can be also naturally extended to multi-party VFL problems.
The rest of the paper is organized as follows. We discuss related work in section 5 and introduce the detailed formulation of the Semi-VFL setting in section 2. Section 3 describes in detail the design of our proposed distillation framework. We present experiments in section 4 and conclude in section ??.

PRELIMINARY 2.1 Two-Party Vertical SplitNN
VFL enables multiple participants (usually two) collaboratively train a machine learning model with features distributed among them but labels owned by only one of them. The label owner party is called the active party, and the others are called the passive party. Since the label is usually the key asset of the active party and is highly sensitive, the passive party is not allowed to access the final prediction model (due to the label inference attack [6]). Thus, the typical target of building a vertical federation is to improve the performance of the active party. Only the active party is willing and granted to deploy the model and the passive party only asks for money in return.
In advertising systems, we have only two participants and the predominant model structure is neural networks, so in this paper, we focus on a typical two-party VFL setting and use the SplitNN [3,21] as the backbone model. For the sake of brevity, we use the term "local" as a special pronoun for the active party in the following sections. In splitNN, each party holds a bottom model for extracting hidden representations and the active party additionally holds a top model to fuse two sides of representations and make predictions. The fusion layer is called the cut layer and is usually implemented by concatenation. To finish the training, the passive party will send its hidden representations to the active party in the forward process, and receive gradients from the active party in the backward process. The security of intermediate data transmission can be satisfied by combining methods proposed by related works which are complementary to us, so we focus on the aspect of model training. Figure 1: The overall framework of JPL.
As common industrial choices, we use the hash embedding layer to process the categorical input fields and MLP for both the bottom and top models. However, our method is not restricted to these choices and is applicable to more complex and advanced structures. Using A and B to denote the active party and the passive party respectively,the federated model can be denoted as: where e = [e 1 , e 2 , . . . , e ] and e = [e 1 , e 2 , . . . , e ] are wide concatenated vectors of all fields' embedding vectors, and denote the number of input fields. and are bottom models, is the top model andˆdenotes the prediction. Besides, We'll use h = (e ), h = (e ) to denote the hidden representations for the overlapped data, and u = (e ′ ), u = (e ′ ) for the non-overlapped data. , where ( |x ) is the conditional label distribution regarding to the active party's feature. The core goal of Semi-VFL is to achieve better results than No-VFL while relying only on active party features. Compared with traditional settings (as shown in table 1), the advantages of Semi-VFL can be summarized as follow:

Problem Formulation
(1) Semi-VFL extends the applicable scope of Full-VFL from the overlapped sample space to the full sample space, but do not rely on knowing the passive party's feature. (2) Semi-VFL can utilize more data than both No-VFL and Full-VFL, but maintains the same level of inference cost as No-VFL. (3) The learning goal of Semi-VFL can meet the demand of the active party better than Full-VFL.

METHOD
In this section, we will firstly introduce the overall framework of JPL and then dive into the details of model structure and loss design.

The Overall Framework
JPL is designed as a two-stage learning framework: the federated pre-learning stage and the JPL stage, as shown in Figure 1. We firstly train the federated teacher on the overlapped dataset and then utilize it together with the full dataset to learn a local student model. During the JPL stage, the teacher model is fixed and acts as a regularizer to constrain features and predictions of the student. Once the JPL stage finished, all the auxiliary modules will be dropped and the inference of the student model is decoupled with the passive party. Compared with the basic splitNN, our difference only lies in the amount of data usage and model structure for the active party, without any new modification requirements for the passive party, and without any additional network transmission requirements, so no additional security risks are introduced. In other words, our security risk is the same as that of the base splitNN, and existing security enhancement methods for splitNNs are also applicable to our approach.

Model Structure
Let's firstly unify the notations. Bottom models are denoted as and top models are denoted as , here the top model may take one or more hidden representations from party A or party B as inputs. The superscript and identify whether a function belongs to the teacher or the student. The subscripts and denote a function's functionality in processing specific feature space, not its physical affiliation. As for model affiliation, among all parameterized functions and embedding, only and e belong to the passive party. As shown in figure 1, the student model has two bottom modules and three top modules during training. The two bottom modules and uses x to respectively learn hidden representations for x and x . , and are three classifiers respectively working in the feature space of party A, party B and both. Note that, these modules can take in both the overlapped and non-overlapped samples as input. All prediction results produced by classifiers will used together with the teacher's predictions to form the JPL loss. After JPL training, the auxiliary classifier will be dropped, the final prediction is the ensemble of the rest classifiers.

Imitating Full Feature Space
The performance improvement of the federated model is brought by the additional use of B-side features, but they are banned to use in the inference stage of Semi-VFL, this seems to be an insoluble paradox. So how can we still leverage the knowledge of x to make predictions, with only x available? From a probabilistic view, the only possible way is to learn the intrinsic correlation between x and x (e.g., the conditional distribution (x |x ). And fortunately, the B-side feature for the overlapped samples is available during the training process. Based on these motivations, we design a partial encoder (x ) to learn the mapping function from the A-side input space to the B-side feature space. Since there is no available B-side feature for the non-overlapped data, we firstly learn from the overlapped part and further transfer its knowledge to the nonoverlapped part.
3.3.1 For the Overlapped Sample. Lets' use h = (x ), ∀x ∼ (x ) to denote the imitation feature of samples from the overlapped distribution (x ). We expect h to imitate the functionality of h and design learning objectives from two aspects: • equivalence in feature representation: In the ideal case, we expect imitation features to be numerically equal to the original features, but this is difficult and unrealistic due to the intrinsic difference of A and B, so we only require to maximize their similarity in some transformed space as a looser constraint. As a common and practical choice, we use the same measurement as in CL, that is cos-similarity or dot-product in the vector 2 hypersphere. • equivalence in label discrimination: the imitation feature should manifest the same discriminative functionality as the raw one, including its cooperative discriminative utility with h in the federated classifier (h , h ) and the independent discriminative utility in the B-side classifier (h ). These two classifiers have different inductive biases, so we take both of them into consideration to promote a comprehensive modeling. For the representation equivalence, we design the cross-correlation matrix error (CME). Despite of point-wise value identity restriction in AEs, we choose to reconstruct the similarity of B-side features as an easier and looser constraint. Firstly, the features are transformed to a metric space and measured by cosine similarity: where is a shared projection function. To be succinct, we use s = (h )/∥ (h ) ∥ 2 and t = (h ) ⊤ /∥ (h )∥ 2 as abbreviation of the transformed features. Respectively using the matrix S ∈ × and T ∈ R × to denote a batch of features in the metric space for the student and teacher, the cross-correlation error matrix is Regarding to the importance of negative student-teacher pairs and labeled negative samples, we can write three version of CME: CME, single-balanced CME(bCME) and double-balanced CME (dCME), as show in follow: where Y + and Y − denotes the index sets of the positive and negative samples, | · | denotes the cardinality. denotes the batch size, is the identity matrix and (·) means the diagonal vector of a matrix. The original CME treats all similarity pairs with equal importance, while the bCME balance the importance of the correct and incorrect student-teacher pair. The dCME further consider the skewness of binary label. The degree of "balance" between the three losses gradually increases. The so-called "balance" takes into account two facts: 1) positive labels are scarce and more important than negative labels, and 2) the right pair is scarce and more important than the wrong pair. We choose dCME as the B-side feature reconstruction loss.
For the discrimination equivalence, we propose the Fed-Head consistency loss (FHC) and the B-Head consistency loss (BHC): where CE denotes the binary cross-entropy loss and KL is the KL-divergence. In BHC loss, we constrain the discriminative performance of B-side student features on the newly learned B-side classifier to be consistent with that of the teacher. The FHC loss follow the similar principle, but the well-learned teacher classifier is reused. As we observed empirically in Figure x, the single-side head and federated head maintain different inductive-bias of label discrimination and cover different crowd, we take both of losses into account to maintain this diversity. Finally, we sum up all above losses as the "A to B" transformation loss for the overlapped data: where is a hyper-parameter controlling the loss effect of the federated teacher.

3.3.2
For the Non-overlapped Sample. Lets' use u = (x ) to denote the imitation feature of samples x ∼ (x ) from the non-overlapped distribution. Analogous to the purpose for the overlapped data, We still hope that u can imitate the missed Bside feature information in both representation equivalence and discrimination equivalence. However, it is more challenging for the non-overlapped data, due to the absence of corresponding groundtruth from the teacher (u do not exist).
For the representation equivalence, we propose the cross-space similarity isomorphic (CSI) restriction, motivated by the principle of collaborative filtering (CF). CF assumes that similar users prefer the same items, i.e., if two users are similar in the user feature space, then they should also be similar in the item space. CSI inherits this principle, it assumes that the similarity of samples in both feature space A and B should be numerically close or equal (socalled isomorphic). Based on CSI assumption, we can use the A-side similarity between the overlapped samples and the non-overlapped sample to guide the learning of u . Respectively using capital letter U and H to denote a batch of features (with 2 -normed) from and , the CSI loss is: where denotes the softmax function with a temperature parameter . Here we select the teacher's feature space as the anchor space, for U to achieve the purpose of feature imitation and meanwhile acquire stable supervision signal. So that, in CSI loss, only U is learnable and all other matrices are already known. The discrimination equivalence regularization for u is similar to h , that is: where we use the superscript " " (abbreviation for non-overlapped) to distinguish the same losses for the overlapped data. The losses are so-designed according to the fact that: the optimal feature u * would maintain good performance in the B-side head and both the student and teacher federated classifiers. Finally, we sum up all above losses as the "A to B" transformation loss for the non-overlapped data:

Adapting Full Sample Space
After learning the cross-view partial encoder, we are do capable to make predictions in the whole sample space, however, the learnt A-side representation is only guided by the overlapped data, which is not well-suitable to the non-overlapped sample. To tackle this issue, we design an A-side module to specifically learn in the whole sample space and further integrate it with the federated module to make comprehensive predictions. The A head and Federated head have complementary advantages: the former adapts well to the full-sample space (but partial feature space) and the latter adapts well to the full-feature space (but partial sample space). To fuse the advantages of both, we firstly let them learn mutually to enhance their own prediction, and then combine their diverse predictions as the final result. We achieve this goal via the proposed ranking consistency loss and adopting the logit ensemble.
The inherent differences in the training data result in the federation head learning better from the overlapped data (because of additional B-side fields), while the A head learns better from the non-overlapped data (because of B-side field missing). Therefore, for the overlapped data, the A head should learn from the federated head, while for non-overlapped data, the federated head should learn from the A head. In this way, the two heads can be mutually reinforcing. Since the magnitudes of the predicted scores of two heads are probably different (due to data difference), directly forcing their score to be in the same scale may harm to the prediction diversity. For this reason, we choose to maximizing their ranking consistency by using the scale-invariant pair-wise ranking information.
Inspired by the pair-wise ranking loss, we use a partial order matrix (POM) to fully describe a prediction vector's ranking order. Each element of POM is denoted by the partial order of a sample pair. We use the probability (ˆ>ˆ) to denote the partial order and model it with the logistic function (ˆ−ˆ). It's obvious that, two arrangements for the same element set is identical, when and only when they hold the same partial order for any two elements. Thus we can draw closer two ranking order via drawing closer their POMs. Lets' use R to denote the POM for a batch of overlapped samples' prediction vectorŷ. Permuting the element order according to sample label, we can use three sub-matrices to denote the full POM: The superscripts "+" and "-" of the submatrix indicate the binary label value of elements, for example The POM loss for the overlapped sample is defined as: where (·) denotes the "stop_gradient" operation, indicating that the gradient generated by this element will not be involved in backpropagation. The subscripts "A" and "Fed" denote which prediction head's logit is used in the ranking matrix. The loss consists of three restriction terms imposed on three sub-matrices. For the sub-matrix generated by samples belongs to the same category (R ++ and R −− ), we minimize the gap between the A head and the Fed head (the Fed head does not back propagate the gradient) and balance the effect of label scale. For the sub-matrix generated by cross-category comparing (R +− ), the ground-truth value should always be 1, so we maximize its element value. For the non-overlapped data, we use V to represent its order matrix, its POM loss is similar to that of the over-lapped data: The only difference is that in L , the learning direction is from the A head to the Fed head, and the A head does not back-propagate the gradient. Finally, we integrate the predictions from both heads using averaged logits:ˆ=

The Final Learning Objective
By combining above two losses, we get the final multi-objective JPL training loss: where the hyper-parameter is set to balance the effect of the non-overlapped data, because it's scale is usually larger than the overlapped ones. Θ denotes the collection of all parametrized embedding tables. Following [7], we use 2 regularization to prevent overfitting. JPL is trained in an End2End way, allowing different batch sizes to leveraging more non-overlapped data in training.

EXPERIMENTAL STUDIES
We evaluate the performance of JPL on two real-world datasets to validate its efficiency.

Experimental Setup
4.1.1 Datasets and Tasks. We collect one real federated datasets from Tencent, "the world's largest video game publisher" called by the Economist [5]. It is inherently vertically partitioned, multiviewed, and privacy-sensitive. We also use a well-known public    [4] and manually partition its attributes to simulate a vertical federated data setting.
• Tencent-Ad CVR Dataset comes from Tencent game and a major social media platform. The social media platform provides rich user portrayals extracted from massive social behaviors. The task is also to predict the conversion rate of users for game ads, but for multiple games. • Criteo CTR Dataset is a public CTR benchmark dataset. We manually split its fields into two parts with equal number of numerical fields and categorical fields.
More details of datasets are summarized in Table 2. The two partitioned field sets in each dataset are respectively called the "left part" and the "right part" by their natural order. Notice that, their contribution to label discrimination differs, so we alternatively select one of them as the active party (accompanied with the label) to validate JPL's robustness against field importance.

Evaluation Settings.
Following related works in advertising [7], we use Area-Under-Curve (AUC) to measure model performance.To validate the superiority of JPL, we consider the basic "Non-VFL" Local baseline and related prior works, as follow: • Local Model (Local): The raw model trained in the active party (corresponding to "No-VFL"). It is the basic lower-bound performance for a qualified Semi-VFL method to outperform. • Federated Privileged Distillation (FPD) [12,18]: Two pioneer works [12,18] have adopted privileged distillation to learn the student model in their methods, so we consider them closely related to the Semi-VFL setting. But they have some slight differences in using the non-overlapped data and do not explicitly formalize it in the optimization objective. So we collectively call them FPD, and use L = L + L + (1 − )L as its training loss to make a clarified comparation.
The size ratio between the overlapped and non-overlapped data is usually different in practise, we use a varying setting to simulate this: we fix the size of non-overlapped data and increasing the one of the overlapped data.  Table 2. We use the Adam optimizer with 2 regularization to avoid overfitting. The batch size is 5 for both the overlapped data and non-overlapped data. We use 1/20 of training data as the validation set to conduct early stopping. We tune hyper-parameters for all methods under the same random seed and report the best result among them. We use the selected best teacher models for all the methods. The learning rate and are chosen following ∈ {1 −3 , 5 * 1 −4 }, ∈ {1 −4 , 1 −5 }, and the distillation weight ∈ {0.1, 0.3, 0.5, 0.7, 0.9},. All experiments are conducted via an VFL simulation platform(with a Tensorflow backend [1]).

Results and Analysis
The main results are shown in Figure 2. We can analysis it in three aspects: • Superiority: JPL achieves the best performance in all cases, no matter the strengthen of the active partys' feature sets or the data volume of overlapped size. It shows the strong generalizability of JPL in adapting the field partition setting and dataset size changing. • Adaptation of field importance: It's obvious that the "right" part field set is more important than the left one. the former gets more than 76% point AUC performance in almost cases but the latter never get that point. The hardness of recovering Bside feature and the prediction quality from A-side classification head is different in two settings. Taking the data point at 1500 thousand as example, JPL achieves higher AUC improvemnt on the left feature sets (0.74%) than it in the right feature set (0.45%) compared against the raw local model. • Adaptation of data volume: The performance of all methods gets improved by the ascending of data volume. This validate the importance of data volume in affecting model performance. Though this result seems to be natural, it is not that trival to get such result (for both FPD and JPL). we have tried four types of training mode to get such "good" result, details will be discussed in appendix. In general, JPL can benefit from further data growth with correct training technics, it's a critical point for it to use in reality.
In summary, JPL achieves efficiently the goal of Semi-VFl and outperform existing baseline methods with significant improvement. More details about experiments will be provided in appendix.

RELATED WORK
As one of the most noticeable task for internet companies, more and more efforts have been taken to apply federated learning in recommendation tasks [14, 23? , 24]. However, these works varies a lot in data scene (sample-wise partition or feature-wise partition) [13,17] and model type. Another line of work focus on improving the security of the federated learning system [2,8,15,20], especially the problem of label leakage [6,16,19,25]. These ideas are complementary to ours and can be combined to enhance security. While our paper focus on improving the utility of split neural network models [3] in the vertical federated learning setting for advertising tasks, considering both the high inference cost problem and data restrction problem. This motivation clearly distinguish our work from many of related works. As few works with similar purposes, FedMVT [10] also uses splitNN model and consider to use all the non-overlapped data, it foucs on designing learning mechanism to complement missed data fields and labels, but do not consider the high cost of inference and not validated on recommendation tasks. [18] and [11] are pioneer works realizing the importance of inference decoupling, They "force" to transfer the knowledge of the federated model into the active party's local model. Though the student model missed half of input fields, it do maintains noticeable performance lift over raw local models, showing the huge potential on the "information-asymmetric" distillation style. However, they only utilize the soft label as the distillation signal and do not take actions to alleviate the negative impact of field missing and sufficiently consider student's adoption to the non-overlapped data. While we formally formalize the vertical semi-federated learning problem in this paper and provide a more comprehensive algorithm which consider both the full sample space and the full feature space. And they [11,18] can also be treated as degraded special versions of our solution.