<jupyter_start><jupyter_text>Image taging using Google Cloud VisionIn this notebook, we will use the [Google Cloud Vision](https://cloud.google.com/vision) to 'tag' out the elements in each image of our dataset. And make a histogram of these tags and find out the most representative tags for a certain genre.Integrates Google Vision features, including image labeling, face, logo, and landmark detection, optical character recognition (OCR), and detection of explicit content, into applications.  Authentication and Configuration * [Set up a Cloud Vision API project](https://cloud.google.com/vision/docs/quickstartset_up_a_google_cloud_vision_api_project) in the Google Cloud Platform Console.* Set up your environment for using [Application Default Credentials](https://cloud.google.com/vision/docs/auth). Python* Install [Python](https://www.python.org/)* [Install PIP](https://pip.pypa.io/en/latest/installing/)* [Install the Google Cloud Client Library](https://cloud.google.com/vision/docs/librariesinstalling_the_client_library) and the [Python Imaging Library](http://python-pillow.org/).* Imports the Google Cloud client library* Add environment variables with the JSON file applied in step 2* Instantiates a client<jupyter_code>import io
import os
import pandas as pd
# Imports the Google Cloud client library
from google.cloud import vision
from google.cloud.vision import types

# Instantiates a client
os.environ["GOOGLE_APPLICATION_CREDENTIALS"]= 'Your creditial'
client = vision.ImageAnnotatorClient()<jupyter_output><empty_output><jupyter_text>**Test on tagging one Image*** Load the image into memory* Convert image into acceptable type<jupyter_code># The name of the image file to annotate
file_name = 'C:/Users/qyao7/INFO7390/Final_MindaChen_QiuminZhang/dataset_merged/training/ambient/a0952779729_2.jpg'

# Loads the image into memory
with io.open(file_name, 'rb') as image_file:
    content = image_file.read()

image = types.Image(content=content)<jupyter_output><empty_output><jupyter_text>**Performs label detection on the image file**<jupyter_code>response = client.label_detection(image=image)
labels = response.label_annotations

print('Labels:')
for label in labels:
    print(label.desciption)<jupyter_output>Labels:
musical instrument
keyboard
organist
keyboard player
piano
musical keyboard
electronic device
electronic instrument
musical instrument accessory
pianist<jupyter_text>**get score for certain label**<jupyter_code>labels[0].score<jupyter_output><empty_output><jupyter_text>**Tag all images**We have tagged all images of the training set and it took quite a long time. Please check tags_all.csv for entire results.<jupyter_code>dir = './dataset/training'

for obj in os.listdir(dir):
    if obj!='.DS_Store':
        tags = []
        print('working on %s' %(obj))
        try:
            for img in os.listdir(os.path.join(dir,obj)):
                if img!='.DS_Store' and img!='tags.csv':
                    try:
                        with io.open(os.path.join(dir,obj,img), 'rb') as image_file:
                            content = image_file.read()
                        image = types.Image(content=content)
                        response = client.label_detection(image=image)
                        labels = response.label_annotations
                        temp = {'file_name': img,
                                'genre': obj}
                        label_count = 0
                        for label in labels:
                            temp['Label'+str(label_count)] = label.description
                            temp['possibility'+str(label_count)] = label.score
                            label_count+=1
                        tags.append(temp)
                    except:
                        continue
        except:
            continue
        pd.DataFrame(tags).to_csv(os.path.join(dir,obj,'tags.csv'),header=True)<jupyter_output>working on .DS_Store
working on metal
working on hiphop_rap
working on ambient
working on rock
working on folk
working on punk
working on soul
working on dubstep
working on jazz<jupyter_text>**Plot first 20 tags that appear most in each genre**<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

fig=plt.figure(figsize=(24, 12))
columns = 4
rows = 3
count = 1

for obj in os.listdir(dir):
    try:
        tags = pd.DataFrame.from_csv(os.path.join(dir, obj, 'tags.csv'))
        tags = tags.fillna(np.nan)
        
        list = []
        
        for i in tags.iloc[:,0:10]:
            for label in tags[i]:
                list.append(label)
        
        df = pd.DataFrame(list)
        v = df.iloc[:,0].unique()
        g = df.groupby(by = 0)[0].count().sort_values(ascending=False)
        r = range(min(len(v), 20))
        
        fig.add_subplot(rows,columns,count)
        plt.barh(r, g.head(20))
        plt.yticks(r, g.index)
        plt.title(obj)
        plt.show()
        count+=1
    except:
        print(obj+' failed')
        continue
plt.show()
fig.savefig('tags_hist.png')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0993103448275862, 'fraction_numerical': 0.007881773399014778, 'mean_word_length': 2.9874312647289867, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 6, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '217231', 'license': 'MIT', 'n_tokens_mistral': 1516, 'n_tokens_neox': 1448, 'n_words': 412, 'path': 'Neural Network/7390_Final/Tag_images.ipynb', 'repo_name': 'qiuminzhang/INFO7390_Final_Final', 'size': '5075'}