In this experiment, we follow the procedure specified in
[1]}, by first training A-PixelHop with real and
ProGAN-generated fake images, and then evaluating its detection performance on real images or fake images generated by the aforementioned eleven generative models. The
performance comparison between A-PixelHop and two existing
methods, Auto-Spec and the method proposed by Wang et al. in [1]}, is shown in
Table REF . It is worthwhile to emphasize three points.
First, since we do not include augmentation in the training of
A-PixelHop, we compare against [1]}
under the no augmentation setting. Second, we evaluate the performance in terms of
average precision (AP) so as to be
consistent with [1]}. Third, we collect a total of
10%, 20% and 30% of samples from two ends for the image-level ensemble
and show the corresponding mean AP (mAP) values. We see from the table that
A-PixelHop outperforms both Auto-Spec and the method from [1]} in
all three cases (i.e., 10%, 20% and 30% of samples) in terms of mAP. A-PixelHop outperforms both
Auto-Spec and the method from [1]} by a large margin in the case of
Gau-GAN. A-PixelHop performs worse in the case of Big-GAN, SITD and
SAN, indicating a weaker transferability from ProGAN to these three
generative models. It demands further exploration. SITD and SAN generate images of very high resolution (e.g., 3Kx4K pixels), which does not match well with that of the training images generated by ProGAN.
One possible fix is to rescale these large images to smaller ones in the
pre-processing step.
