Detectability of trojans in poisoned AI models:
While we showed a minimum tensor-state overlap between clean and poisoned class encodings in Figure REF  for semantically equivalent traffic signs of STOP pedestrian crossing sign, it is not clear whether detecting encodings of hidden classes is computationally feasible without a priori knowledge about types of triggers, types of foreground, and types of background by the trojan detector designers. One can view trojan detection as an outlier detection problem [1]} and aim at establishing probably approximately correct (PAC) style learning bounds on the outlier detection under general assumptions. However, such limited guarantees on a successful trojan detection pose vulnerability risks as reported by Sun et al. [2]} by demonstrating a patch attack in backdoored “broken” classifiers. Furthermore, Goldwasser et al. [3]} has shown two frameworks of planting undetectable backdoors with incomparable guarantees (i.e., finding a trojan cannot be solved in polynomial time) for a class of ReLU networks. This topic of trojan detectability remains an open research problem.
