One way of overcoming the above limitations is to distribute synthetic data to the parties as rewards (in short, synthetic data rewards) instead of trained models. It has been demonstrated that augmenting real data with synthetic data can improve model performance: For example, some works [1]}, [2]}, [3]} have used generative adversarial networks (GANs) for data augmentation to improve classification performance on various medical imaging applications such as liver lesions or brain scan segmentations.
Distributing synthetic data as rewards is less likely to violate data privacy regulations, unlike sharing real data directly.
Also, there is no assumption on whether all parties share a common downstream learning task, the task of interest to each party (e.g., supervised or unsupervised, classification or regression), or the type of ML model used by each party. In particular, with the synthetic data reward, each party can now optimize over model architectures and hyperparameters, train new model architectures emerging in the future, and train separate ML models for different learning tasks.
