<jupyter_start><jupyter_text>Downloading Flicker 8 Dataset an their text<jupyter_code>!nvidia-smi
!wget https://github.com/jbrownlee/Datasets/releases/download/Flickr8k/Flickr8k_Dataset.zip
# !wget https://github.com/jbrownlee/Datasets/releases/download/Flickr8k/Flickr8k_text.zip<jupyter_output>--2020-11-18 12:59:58--  https://github.com/jbrownlee/Datasets/releases/download/Flickr8k/Flickr8k_Dataset.zip
Resolving github.com (github.com)... 172.16.17.32
Connecting to github.com (github.com)|172.16.17.32|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/124585957/47f52b80-3501-11e9-8f49-4515a2a3339b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20201118%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20201118T125958Z&X-Amz-Expires=300&X-Amz-Signature=77bc93f7a5f7abd81c3fbbb2799213f3cf6041279f94dc8a8f73f401ce1fb53a&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=124585957&response-content-disposition=attachment%3B%20filename%3DFlickr8k_Dataset.zip&response-content-type=application%2Foctet-stream [following]
--2020-11-18 12:59:58--  https://github-production-release-asset-2e65be.s3.amazonaws.com/124585957/47f52b80-3501-11e9-8f49-4515a2a3339b?X-Amz-Algorithm=AWS4[...]<jupyter_text><jupyter_code># unzip flicker 8 dataset
!unzip -qq '/content/Flickr8k_Dataset.zip' -d .
# Lowering pytorch to stable version of lambda
!pip install --upgrade torch==1.5.1+cu101 torchvision==0.6.1+cu101 -f https://download.pytorch.org/whl/torch_stable.html
# imread problem in previous version
!pip install --upgrade scipy==1.2.1<jupyter_output>Collecting scipy==1.2.1
[?25l  Downloading https://files.pythonhosted.org/packages/7f/5f/c48860704092933bf1c4c1574a8de1ffd16bf4fde8bab190d747598844b2/scipy-1.2.1-cp36-cp36m-manylinux1_x86_64.whl (24.8MB)
[K     |████████████████████████████████| 24.8MB 1.3MB/s 
[?25hRequirement already satisfied, skipping upgrade: numpy>=1.8.2 in /usr/local/lib/python3.6/dist-packages (from scipy==1.2.1) (1.18.5)
[31mERROR: umap-learn 0.4.6 has requirement scipy>=1.3.1, but you'll have scipy 1.2.1 which is incompatible.[0m
[31mERROR: tensorflow 2.3.0 has requirement scipy==1.4.1, but you'll have scipy 1.2.1 which is incompatible.[0m
[31mERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.[0m
Installing collected packages: scipy
  Found existing installation: scipy 1.4.1
    Uninstalling scipy-1.4.1:
      Successfully uninstalled scipy-1.4.1
Successfully installed scipy-1.2.1<jupyter_text>Connecting to Drive<jupyter_code>from google.colab import drive
drive.mount('/content/drive')<jupyter_output>Mounted at /content/drive<jupyter_text>Importing and Appending Paths<jupyter_code>import sys
S12_PROJECT_PATH = '/content/drive/My Drive/TSAI/TSAI-2/S12'
sys.path.append(S12_PROJECT_PATH+'/S12_Modular_Code/')
# Adding python path to env variables
!export PYTHONPATH="${PYTHONPATH}:/content/drive/My Drive/TSAI/TSAI-2/S12/S12_Modular_Code/"
# moving caption file to local
%cp -av '{S12_PROJECT_PATH}/Save_Model/dataset_flickr8k.json' ./
# Creating data folder for processing the data
!mkdir data
!python3 $'{S12_PROJECT_PATH}'/S12_Modular_Code/create_input_files.py --dataset 'flickr8k' --json '/content/dataset_flickr8k.json' --image_folder '/content/Flicker8k_Dataset/' --output_folder '/content/data/'
!python3 $'{S12_PROJECT_PATH}'/S12_Modular_Code/train.py
# Copy checkpoints to drive
%cp -av BEST_checkpoint_flickr8k_5_cap_per_img_5_min_word_freq.pth.tar '{S12_PROJECT_PATH}/Save_Model/'
%cp -av checkpoint_flickr8k_5_cap_per_img_5_min_word_freq.pth.tar '{S12_PROJECT_PATH}/Save_Model/'
!mkdir checkpoints
!mv *.pth.tar checkpoints<jupyter_output><empty_output><jupyter_text>Results Check<jupyter_code>import json
import torch

from caption import caption_image_beam_search, visualize_att
with open('./data/WORDMAP_flickr8k_5_cap_per_img_5_min_word_freq.json', 'r') as j:
    word_map = json.load(j)

rev_word_map = {v: k for k, v in word_map.items()}  # idx2word
checkpoint = torch.load('./checkpoints/BEST_checkpoint_flickr8k_5_cap_per_img_5_min_word_freq.pth.tar', map_location='cuda')
decoder = checkpoint['decoder']
decoder = decoder.to('cuda')
decoder.eval()
encoder = checkpoint['encoder']
encoder = encoder.to('cuda')
encoder.eval()
image = './Flicker8k_Dataset/2786299623_a3c48bd318.jpg'
seq, alphas = caption_image_beam_search(encoder, decoder, image, word_map, 5)
alphas = torch.FloatTensor(alphas)

print(f'Caption:\n{" ".join([rev_word_map[ind] for ind in seq][1:-1])}\n')

visualize_att(image, seq, alphas, rev_word_map)<jupyter_output>/pytorch/aten/src/ATen/native/BinaryOps.cpp:81: UserWarning: Integer division of tensors using div or / is deprecated, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.<jupyter_text>Save the model state dict for deploying to lambda<jupyter_code>ecpu = encoder.to('cpu')
torch.save(ecpu.state_dict(), 'encoder_cpu.pt')

dcpu = decoder.to('cpu')
torch.save(dcpu.state_dict(), 'decoder_cpu.pt')

# Save to google drive
%cp -av encoder_cpu.pt '{S12_PROJECT_PATH}/Save_Model/'
%cp -av decoder_cpu.pt '{S12_PROJECT_PATH}/Save_Model/'
%cp -av './data/WORDMAP_flickr8k_5_cap_per_img_5_min_word_freq.json' '{S12_PROJECT_PATH}/Save_Model/'<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13803284738881713, 'fraction_numerical': 0.0843329027495848, 'mean_word_length': 6.374149659863946, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 24, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '528214', 'license': 'MIT', 'n_tokens_mistral': 2437, 'n_tokens_neox': 2173, 'n_words': 336, 'path': 'S12/ImageCaptioning.ipynb', 'repo_name': 'pankaj90382/TSAI-2', 'size': '5419'}