Given the much smaller sizes of two of the datasets, transfer learning is the method that is recommended by previous studies ([1]}, [2]}). In Table 3, we compare the results obtained using different transfer learning trials with the results obtained using multi-task learning on heterogeneous data, i.e., training a single model on all three datasets. In each of the transfer learning trials, we first trained the model to convergence on one of the datasets. We then fine-tuned it to convergence on another dataset. We then further fine-tuned the model that had been trained on the second dataset on the remaining dataset. Our definition of convergence is the same as defined in Section REF . Our fine-tuning strategy was “deep fine-tuning" [1]}; we reduced the initial learning rate by half and fine-tuned all model layers. We also experimented with shallow fine-tuning as well as keeping the initial learning rate, but the results were inferior.
<TABLE>