<jupyter_start><jupyter_text>wav2vec-u CV-sv - GAN> "GAN training for wav2vec-u on Common Voice Swedish"- toc: false- branch: master- badges: true- comments: true- categories: [kaggle, colab, wav2vec-u] The original attempt on [Kaggle](https://www.kaggle.com/jimregan/wav2vec-u-cv-swedish-gan) won't run because of an issue with CuDNN, but this notebook runs fine on Colab.  Preparation<jupyter_code>!pip install condacolab
import condacolab
condacolab.install()
%%capture
!conda install -c pykaldi pykaldi -y
!git clone https://github.com/jimregan/fairseq/ --branch issue3581
!git clone https://github.com/kpu/kenlm
%%capture
!apt-get -y install libeigen3-dev liblzma-dev zlib1g-dev libbz2-dev
%%capture
%cd /content/kenlm
!python setup.py install
%cd /tmp
import os
os.environ['PATH'] = f"{os.environ['PATH']}:/content/kenlm/build/bin/"
os.environ['FAIRSEQ_ROOT'] = '/content/fairseq'
%cd /content/fairseq/
%%capture
!python setup.py install
os.environ['HYDRA_FULL_ERROR'] = '1'
%%capture
!pip install editdistance<jupyter_output><empty_output><jupyter_text>https://colab.research.google.com/github/corrieann/kaggle/blob/master/kaggle_api_in_colab.ipynb<jupyter_code>%%capture
!pip install kaggle
from google.colab import files

uploaded = files.upload()

for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(
      name=fn, length=len(uploaded[fn])))
  
# Then move kaggle.json into the folder where the API expects to find it.
!mkdir -p ~/.kaggle/ && mv kaggle.json ~/.kaggle/ && chmod 600 ~/.kaggle/kaggle.json
%cd /content
!kaggle datasets download "jimregan/w2vu-cvsv-prepared-text"
%%capture
!unzip /content/w2vu-cvsv-prepared-text.zip
!kaggle datasets download -d jimregan/w2vu-cvsv-precompute-pca512-cls128-mean-pooled
%%capture
!unzip w2vu-cvsv-precompute-pca512-cls128-mean-pooled.zip
!rm *.zip<jupyter_output><empty_output><jupyter_text>GAN<jupyter_code>import torch
torch.version.cuda
torch.backends.cudnn.version()
%cd /content/fairseq
from google.colab import drive
drive.mount('/content/drive')
%%writefile rungan.sh
PREFIX=w2v_unsup_gan_xp
TASK_DATA=/content/precompute_pca512_cls128_mean_pooled
TEXT_DATA=/content/preppedtext/phones/
KENLM_PATH=/content/preppedtext/phones/lm.phones.filtered.04.bin

PREFIX=$PREFIX CUDA_LAUNCH_BLOCKING=1 fairseq-hydra-train \
	-m --config-dir fairseq/config/model/wav2vecu/gan \
	--config-name w2vu \
	task.data=${TASK_DATA} \
	task.text_data=${TEXT_DATA} \
	task.kenlm_path=${KENLM_PATH} \
	checkpoint.no_epoch_checkpoints=true \
	checkpoint.save_dir=/content/drive/MyDrive/w2vu \
	'common.seed=range(0,5)'
!bash rungan.sh<jupyter_output>[2021-06-04 00:06:14,189][fairseq.tasks.unpaired_audio_text][INFO] - REF: ɛ n f œ ʂ ə n a d ɵ ʂ ə k t f øː r d eː t s ɔ m h ɛ n d ə p oː ɕ œ r k ɔ n s ɛ t ə n
[2021-06-04 00:06:14,192][fairseq.tasks.unpaired_audio_text][INFO] - HYP: oː b iː ʃ œ m ɕ m œ ɕ ɪ ɵ ɕ ɵ m ɵ s ɵ uː ɵ s ɵ ɛ ʂ a tː sx
[2021-06-04 00:06:14,198][fairseq.tasks.unpaired_audio_text][INFO] - LM [REF]: -53.44462585449219, 0.05339602260269112
[2021-06-04 00:06:14,198][fairseq.tasks.unpaired_audio_text][INFO] - LM [HYP]: -61.104984283447266, 0.006571721232914821
[2021-06-04 00:06:14,844][valid][INFO] - {"epoch": 8, "valid_loss": "0.93", "valid_ntokens": "3039.79", "valid_nsentences": "144.214", "valid_lm_score_sum": "-71760.8", "valid_num_pred_chars": "28972", "valid_vocab_seen_pct": "0.949477", "valid_uer": "92.9812", "valid_weighted_lm_ppl": "229.386", "valid_lm_ppl": "206.793", "valid_wps": "15426", "valid_wpb": "3039.8", "valid_bsz": "144.2", "valid_num_updates": "128", "valid_best_weighted_lm_ppl": "189.002"}
[2021-0[...]
{'fraction_non_alphanumeric': 0.15179796870710952, 'fraction_numerical': 0.0771342300301949, 'mean_word_length': 5.304498269896194, 'pattern_counts': {'":': 15, '<': 12, '<?xml version=': 0, '>': 13, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '615696', 'license': 'Apache-2.0', 'n_tokens_mistral': 1714, 'n_tokens_neox': 1550, 'n_words': 312, 'path': '_notebooks/2021-05-30-wav2vec-u-cv-swedish-gan.ipynb', 'repo_name': 'jimregan/notes', 'size': '3643'}