<jupyter_start><jupyter_text>Word2Vec for Sentiment Analysis in RussianWord2vec [1] is a computationally-efficient predictive model for learning low-dimensional word embeddings from raw textual data.  1. Loding sentiment dataThe corpus of short texts in Russian based on Twitter messages is available at http://study.mokoron.com/ (and also described in [2]). The entire database of unlabeled tweets was used in order to train Wor2Vec model.<jupyter_code>import re

def preprocess_text(text):
    text = text.lower().replace("ё", "е")
    text = re.sub('((www\.[^\s]+)|(https?://[^\s]+))','URL', text)
    text = re.sub('@[^\s]+','USER', text)
    text = re.sub('[^a-zA-Zа-яА-Я1-9]+', ' ', text)
    text = re.sub(' +',' ', text)
    return text.strip()<jupyter_output><empty_output><jupyter_text>In Python it is easier to work with SQLite databases rather then with SQL databases, so the original SQL file was converted into SQLite using mysql2sqlite script.<jupyter_code>import sqlite3

conn = sqlite3.connect('mysqlite3.db')
c = conn.cursor()

with open('tweets.txt', 'w', encoding='utf-8') as f:
    for row in c.execute('SELECT ttext FROM sentiment'):
        if row[0]:
            tweet = preprocess_text(row[0])
            print(tweet, file=f)
tweets = open('tweets.txt').read().splitlines()
import pandas as pd
%matplotlib inline

preprocessed_data = pd.Series(tweets).apply(lambda x: len(x.split()))
import numpy as np
import matplotlib.pyplot as plt

plt.style.use('ggplot')

plt.figure(figsize=(16, 9)) 
# facecolor='g'
n, bins, patches = plt.hist(preprocessed_data.values, 50, density=True)
plt.xlabel('Number of words in a text')
plt.ylabel('Share of texts')
plt.axis([0, 40, 0, 0.12])
plt.grid(True)
plt.show()
for i in range(20, 35, 1):
    preprocessed_data>i
    print("%0.5f for sentence_length=%r" % (1-(preprocessed_data>i).sum()/preprocessed_data.shape[0], i))<jupyter_output>0.94599 for sentence_length=20
0.96172 for sentence_length=21
0.97419 for sentence_length=22
0.98353 for sentence_length=23
0.99021 for sentence_length=24
0.99457 for sentence_length=25
0.99712 for sentence_length=26
0.99849 for sentence_length=27
0.99924 for sentence_length=28
0.99959 for sentence_length=29
0.99975 for sentence_length=30
0.99984 for sentence_length=31
0.99988 for sentence_length=32
0.99990 for sentence_length=33
0.99992 for sentence_length=34<jupyter_text>2. Training the modelGensim [3] was used for obtaining vector representations of Russian words. The model was trained on the entire dataset, which was collected and pre-processed at the previous steps.<jupyter_code>import logging
import multiprocessing
import gensim
from gensim.models import Word2Vec

logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)

data = gensim.models.word2vec.LineSentence('tweets.txt')
model = Word2Vec(data, size=200, window=5, min_count=3, workers=multiprocessing.cpu_count())
model.save("w2v/tweets_model.w2v")<jupyter_output>2018-06-26 13:09:14,817 : INFO : saving Word2Vec object under w2v/tweets_model.w2v, separately None
2018-06-26 13:09:14,818 : INFO : storing np array 'vectors' to w2v/tweets_model.w2v.wv.vectors.npy
2018-06-26 13:09:16,552 : INFO : not storing attribute vectors_norm
2018-06-26 13:09:16,554 : INFO : storing np array 'syn1neg' to w2v/tweets_model.w2v.trainables.syn1neg.npy
2018-06-26 13:09:22,268 : INFO : saved w2v/tweets_model.w2v<jupyter_text>3. Visualizationt-SNE was used to plot a subset of similar words from trained Word2Vec model. Firstly, similar words were found and appended each of the similar words embedding vector to the matrix. Secondly, t-SNE was applied to the matrix in order to project each word to a 2D space (i.e. dimension reduction). Finally, the 2D position of each word with a label was plotted.<jupyter_code>from gensim.models import Word2Vec

model = Word2Vec.load('w2v/tweets_model.w2v')
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import numpy as np
% matplotlib inline


def tsne_plot(labels, tokens, classes, clusters):
    tsne_model = TSNE(perplexity=15, n_components=2, init='pca', n_iter=3500, random_state=33)
    new_values = tsne_model.fit_transform(tokens)

    x = []
    y = []

    for value in new_values:
        x.append(value[0])
        y.append(value[1])

    colors = cm.rainbow(np.linspace(0, 1, clusters))
    plt.figure(figsize=(16, 9))
    for i in range(len(x)):
        plt.scatter(x[i], y[i], c=colors[classes[i]], alpha=0.75)
        plt.annotate(labels[i], alpha=0.75, xy=(x[i], y[i]), xytext=(5, 2), 
                     textcoords='offset points', ha='right', va='bottom', size=10)
    plt.grid(True)
    plt.show()
labels = []
tokens = []
classes = []

samples = 15
for i, word in enumerate(['программирование', 'инструкция', 'рубашка', 'аэропорт', 'фильм', 'iphone', 'обидел', 'красивый', 'нравится', 'домашка']):#random.sample(list(model.wv.vocab), samples)):#['ссора', 'iphone', 'обидел', 'красивый', 'нравится', 'домашка']):#random.sample(list(model.wv.vocab), 10):
    tokens.append(model.wv[word])
    labels.append(word)
    classes.append(i)
    for similar_word, similarity in model.wv.most_similar(word, topn=20):
        tokens.append(model.wv[similar_word])
        labels.append(similar_word)
        classes.append(i)
tsne_plot(labels, tokens, classes, samples)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1322574661472825, 'fraction_numerical': 0.05639028009645706, 'mean_word_length': 4.860869565217391, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '428979', 'license': 'MIT', 'n_tokens_mistral': 2083, 'n_tokens_neox': 1896, 'n_words': 496, 'path': 'Word2Vec for Sentiment Analysis in Russian.ipynb', 'repo_name': 'sismetanin/sentiment-analysis-of-tweets-in-russian', 'size': '5391'}