<jupyter_start><jupyter_text>StyleGAN2-ADA: training a model from scratchReferences:- this notebook: https://github.com/woctezuma/steam-stylegan2-ada- the original StyleGAN2-ADA repository: https://github.com/NVlabs/stylegan2-ada- my fork of StyleGAN2-ADA: https://github.com/woctezuma/stylegan2-ada  Mount Google Drive<jupyter_code>%pip install Google-Colab-Transfer
import colab_transfer

gd = colab_transfer.get_path_to_home_of_google_drive()
lm = colab_transfer.get_path_to_home_of_local_machine()

colab_transfer.mount_google_drive()<jupyter_output>Mounted at /content/drive/<jupyter_text>Switch to Tensorflow 1.x > TensorFlow 2.x is not supported.Reference: https://github.com/NVlabs/stylegan2-adarequirements> Colab uses TensorFlow 2.x by default, though you can switch to 1.x by the method shown below.Reference: https://colab.research.google.com/notebooks/tensorflow_version.ipynb<jupyter_code>%tensorflow_version 1.x<jupyter_output>TensorFlow 1.x selected.<jupyter_text>Install my fork of StyleGAN2-ADASwitch to the branch `google-colab`.<jupyter_code>%cd /content/

%rm -rf stylegan2-ada/
!git clone https://github.com/woctezuma/stylegan2-ada.git

%cd stylegan2-ada/
!git checkout google-colab

%cd /content/<jupyter_output>/content
Cloning into 'stylegan2-ada'...
remote: Enumerating objects: 17, done.[K
remote: Counting objects: 100% (17/17), done.[K
remote: Compressing objects: 100% (15/15), done.[K
remote: Total 112 (delta 7), reused 10 (delta 2), pack-reused 95[K
Receiving objects: 100% (112/112), 1.39 MiB | 29.06 MiB/s, done.
Resolving deltas: 100% (48/48), done.
/content/stylegan2-ada
Branch 'google-colab' set up to track remote branch 'google-colab' from 'origin'.
Switched to a new branch 'google-colab'
/content<jupyter_text>Prepare datasets (once)Reference: https://github.com/NVlabs/stylegan2-adapreparing-datasets<jupyter_code>folder_name = 'datasets/steam/'<jupyter_output><empty_output><jupyter_text>Import image data from Google Drive Data consists of 14,000 vertical Steam banners, resized from 300x450 to 256x256 resolution.<jupyter_code>archive_name = 'resized_vertical_steam_banners_14k.tar'
# colab_transfer.copy_file(file_name=archive_name,
#                          source = gd + 'data/')
# !echo tar -xf {archive_name}
# !tar -xf {archive_name}<jupyter_output><empty_output><jupyter_text>Prepare datasets for StyleGAN2<jupyter_code>custom_dataset = '/content/' + folder_name
custom_images = '/content/data/resized_vertical_steam_banners'
# !python stylegan2-ada/dataset_tool.py create_from_images {custom_dataset} {custom_images}<jupyter_output><empty_output><jupyter_text>Archive prepared datasets to Google Drive<jupyter_code># colab_transfer.copy_folder(folder_name,
#                            source=lm,
#                            destination=gd)<jupyter_output><empty_output><jupyter_text>Import previously prepared datasets from Google Drive<jupyter_code>colab_transfer.copy_folder(folder_name)<jupyter_output>Copying /content/drive/My Drive/datasets/steam/ to /content/datasets/steam/<jupyter_text>Train new networks  Computation time A tick requires 4 minutes of computation time, so we save a snapshot every 6 ticks (`--snap=6`) in order to save every 24 minutes.Processing 1 kimg requires 1 min of computation time, so:-   the total training time for **5 Mimg** should be 3.5 days of computation,-   the total training time for **25 Mimg** should be 2.5 weeks of computation.  Documentation for transfer learning ```transfer learning:  --resume RESUME       Resume from network pickle (default: noresume)  --freezed INT         Freeze-D (default: 0 discriminator layers)``````transfer learning source networks (--resume):  ffhq256        FFHQ trained at 256x256 resolution.  ffhq512        FFHQ trained at 512x512 resolution.  ffhq1024       FFHQ trained at 1024x1024 resolution.  celebahq256    CelebA-HQ trained at 256x256 resolution.  lsundog256     LSUN Dog trained at 256x256 resolution.    Custom network pickle.```  Training recommendations -   about `--gamma`:> The training configuration can be further customized with additional arguments. Common examples:> * `--gamma=10` overrides R1 gamma. We strongly recommend trying out at least a few different values for each new dataset.-   about `--cfg`:> Please note that `--cfg=auto` only serves as a reasonable first guess for the hyperparameters &mdash; it does not necessarily lead to optimal results for a given dataset. For example, `--cfg=stylegan2` yields considerably better FID  for FFHQ-140k at 1024x1024 than illustrated above. We recommend trying out at least a few different values of `--gamma` for each new dataset.-   about `--metrics`:> By default, `train.py` will automatically compute FID for each network pickle. We strongly recommend inspecting `metric-fid50k_full.txt` at regular intervals to monitor the training progress. When desired, the automatic computation can be disabled with `--metrics none` to speed up the training.  Training settings Resolution is automatically determined from the dataset.Other settings are chosen as follows:- horizontal mirroring, via `--mirror=true` (instead of default to False),- no EMA rampup, via `--cfg=auto_no_ramp`,- cfg map = 8 (instead of 2), via `--cfg_map=8`,- freezed D (the first 10 layers, as in the article for resolution 256), via `--freezed=10`.References:-   https://github.com/NVlabs/stylegan2-adatraining-new-networks-   https://raw.githubusercontent.com/NVlabs/stylegan2-ada/main/docs/train-help.txt  TrainNB: to ensure your Colab session stays connected, follow: https://stackoverflow.com/a/58275370/  Initial run (transfer learning) Start with transfer learning from a model trained on a **diverse** dataset (`lsundog256`).Caveat: the upstream model was trained with cfg map = 8.<jupyter_code># !python stylegan2-ada/train.py \
#  --outdir='/content/drive/My Drive/training-runs' \
#  --snap=6 \
#  --data='/content/datasets/steam' \
#  --mirror=true \
#  --metrics=none \
#  --cfg=auto_no_ramp \
#  --cfg_map=8 \
#  --kimg=5000 \
#  --resume=lsundog256 \
#  --freezed=10 \<jupyter_output><empty_output><jupyter_text>Following runs (resume from latest snapshot) Then,  **automatically** resume from the latest pickle snapshot.Caveat: you will have to **manually** specify the augmentation strength, as found in `log.txt`.<jupyter_code>!python stylegan2-ada/train.py \
 --outdir='/content/drive/My Drive/training-runs' \
 --snap=6 \
 --data='/content/datasets/steam' \
 --mirror=true \
 --metrics=none \
 --cfg=auto_no_ramp \
 --cfg_map=8 \
 --kimg=5000 \
 --resume=latest \
 --p=0.713 \
 --freezed=10 \<jupyter_output>tcmalloc: large alloc 4294967296 bytes == 0x8250000 @  0x7f9e8d0c3001 0x7f9e8a2ec765 0x7f9e8a350bb0 0x7f9e8a352a4f 0x7f9e8a3e9048 0x50a4a5 0x50cc96 0x507be4 0x508ec2 0x594a01 0x549e8f 0x5515c1 0x5a9dac 0x50a433 0x50cc96 0x507be4 0x588e5c 0x59fd0e 0x50d256 0x507be4 0x588e5c 0x59fd0e 0x50d256 0x507be4 0x588e5c 0x59fd0e 0x50d256 0x5095c8 0x50a2fd 0x50beb4 0x507be4
tcmalloc: large alloc 4294967296 bytes == 0x7f9cbac20000 @  0x7f9e8d0c11e7 0x7f9e8a2ec5e1 0x7f9e8a350c78 0x7f9e8a350f37 0x7f9e8a3e8f28 0x50a4a5 0x50cc96 0x507be4 0x509900 0x50a2fd 0x50cc96 0x507be4 0x509900 0x50a2fd 0x50cc96 0x507be4 0x509900 0x50a2fd 0x50cc96 0x5095c8 0x50a2fd 0x50beb4 0x507be4 0x509900 0x50a2fd 0x50beb4 0x507be4 0x509900 0x50a2fd 0x50cc96 0x507be4
tcmalloc: large alloc 4294967296 bytes == 0x7f9bb9c1e000 @  0x7f9e8d0c11e7 0x7f9e8a2ec5e1 0x7f9e8a350c78 0x7f9e8a350f37 0x7f9e483560c5 0x7f9e47cd9902 0x7f9e47cd9eb2 0x7f9e47c92c3e 0x50a12f 0x50beb4 0x5095c8 0x50a2fd 0x50beb4 0x507be4 0x588c8b 0x59fd0e 0x50d256 0x507b[...]
{'fraction_non_alphanumeric': 0.09649468294604174, 'fraction_numerical': 0.09610082709728239, 'mean_word_length': 4.946916471506635, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 42, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '486343', 'license': 'MIT', 'n_tokens_mistral': 3147, 'n_tokens_neox': 2693, 'n_words': 711, 'path': 'StyleGAN2_ADA_training.ipynb', 'repo_name': 'woctezuma/steam-stylegan2-ada', 'size': '7617'}