One of the important design decisions of Poly-GAN is the placement of skip connections.
From previous works [1]}, it is understood that coarser spatial resolution results in higher level spatial change, such as pose and shape.
We use skip connections to pass the information from the encoded coarse layers (4x4, 8x8, 16x16) to the respective decoder layers (4x4, 8x8, 16x16) and concatenate to augment the feature representation.
We observed that if we use skip connections between encoder and decoder at spatial resolution above (16x16), then the generated image is not deformable enough to be close to the ground truth. If we don't use skip connections above spatial resolution of (16x16), then the problem of missing minute details arises in the generated image. This is illustrated in the results of Figure REF .
Using skip connection to connect all layers of the encoder to their respective decoder layer would help in passing minute details from the encoder side, but would also hinder learning and generating new images effectively.
