The solution we developed and have deployed to Facebook App's Feed ranking system is based on a two step-process. First, to find the optimum user satisfaction in our ecosystem we run an A/B test ([1]}) that increases the distribution of randomly selected producers' posts. Then in a second step we use machine learning to find heterogeneous treatment effects in this experiment in terms of posting/engagement-received, ie we estimate a model about which content producers are most likely to increase their production and create further valuable content if they receive engagement. We can model this through heterogeneous effects machine learning such as a three-tree model, which we use at Meta and we lay out the details in the paper below. After the model is verified to work in a test set, we deploy the heterogeneous effects machine learning model in our ranking system, rewarding content producers that create good content in the future, in order to give an optimum amount of long-run user happiness to our users. The model, which scores all content producers, can be continuously retrained on a small producer holdout where producers do not receive treatments.
