GAN-based generation of realistic 3D data: A systematic review and taxonomy
Data has become the most valuable resource in today's world. With the massive
proliferation of data-driven algorithms, such as deep learning-based
approaches, the availability of data is of great interest. In this context,
high-quality training, validation and testing datasets are particularly needed.
Volumetric data is a very important resource in medicine, as it ranges from
disease diagnoses to therapy monitoring. When the dataset is sufficient, models
can be trained to help doctors with these tasks. Unfortunately, there are
scenarios and applications where large amounts of data is unavailable. For
example, in the medical field, rare diseases and privacy issues can lead to
restricted data availability. In non-medical fields, the high cost of obtaining
a sufficient amount of high-quality data can also be a concern. A solution to
these problems can be the generation of synthetic data to perform data
augmentation in combination with other more traditional methods of data
augmentation. Therefore, most of the publications on 3D Generative Adversarial
Networks (GANs) are within the medical domain. The existence of mechanisms to
generate realistic synthetic data is a good asset to overcome this challenge,
especially in healthcare, as the data must be of good quality and close to
reality, i.e. realistic, and without privacy issues. In this review, we provide
a summary of works that generate realistic 3D synthetic data using GANs. We
therefore outline GAN-based methods in these areas with common architectures,
advantages and disadvantages. We present a novel taxonomy, evaluations,
challenges and research opportunities to provide a holistic overview of the
current state of GANs in medicine and other fields.