While there are many sub-topics of TL, we are deeply interested in domain adaptation (DA) [1]}. While there are many approaches to measure and reduce the disparity between the distributions of these two domains, Maximum Mean Discrepancy (MMD) [2]} and Kullback-Leibler divergence (KL) [3]} are widely used in the literature. Our approach uses KL to approximate the representation of learning distributions between the teachers and the student, differing from the original knowledge amalgamation method [4]}, where the MMD approach is applied.
