PATE [1]} is based on knowledge aggregation and transfer from “teacher” models, trained on disjoint data, to a “student” model whose attributes may be made public.
The approach involves transferring knowledge of multiple models trained on sensitive disjoint datasets. Since these models rely directly on sensitive data, they are not published, but are used as “teachers” for a “student” model. The student learns to predict an output chosen by noisy voting among all of the teachers, and cannot directly access an individual teacher or the underlying data or parameters.
The student is trained on a publicly available, unlabeled dataset, where the labels come from the aggregate votes of the teachers. That is why PATE is considered a semi-supervised approach, and it cannot be applied as it is in as setup where we do not have public data (even small in number) available.
<FIGURE><FIGURE><FIGURE>