<jupyter_start><jupyter_text>Text Analysis of Top 4 Google Play Yoga and Meditation AppsRead the file and see what is inside for the first 5 lines.<jupyter_code>import pandas as pd

reviews = pd.read_csv('TopAppReviews.csv')
reviews.head(5)<jupyter_output><empty_output><jupyter_text>Making the string to lower case and removing punctuation is the first step for pre-processing the text data.<jupyter_code># Make the string lower case

reviews['Translated_Review'] = reviews['Translated_Review'].str.lower()

# Remove punctuation
reviews['Translated_Review'] = reviews['Translated_Review'].str.replace('[^\w\s]','')

reviews.head(5)<jupyter_output><empty_output><jupyter_text>Now, remove the stop words.<jupyter_code>reviews_nostop = reviews.copy()
from nltk.corpus import stopwords
import nltk
nltk.download('stopwords')

stop = stopwords.words('english')
reviews_nostop['Translated_Review'] = reviews_nostop['Translated_Review'].apply(lambda x: " ".join(x for x in x.split() if x not in stop))

reviews_nostop.head()<jupyter_output>[nltk_data] Downloading package stopwords to
[nltk_data]     /Users/lizawood/nltk_data...
[nltk_data]   Unzipping corpora/stopwords.zip.<jupyter_text>Finding n-gramsn-gram means frequent sequences of n words that are occurring together. Below is a table showing 1-Grams, 2-Grams, 3-Grams, and 4-Grams and the number of occurrence in each. This table can be imported into Tableau for further visualization.<jupyter_code>from sklearn.feature_extraction.text import CountVectorizer
import nltk

# Using the reviews_nostop data frame created previously...

# Count the 1-grams
word_vectorizer1 = CountVectorizer(ngram_range=[1,1], analyzer='word')
sparse_matrix = word_vectorizer1.fit_transform(reviews_nostop['Translated_Review'])
frequencies = sum(sparse_matrix).toarray()[0]
one_grams = pd.DataFrame(frequencies, index=word_vectorizer1.get_feature_names(), columns=['frequency']).sort_values('frequency', ascending = False).reset_index()

one_grams.columns= ["1-Gram", "1-Gram Occurence"]


# Count the 2-grams
word_vectorizer2 = CountVectorizer(ngram_range=[2, 2], analyzer='word')
sparse_matrix = word_vectorizer2.fit_transform(reviews_nostop['Translated_Review'])
frequencies = sum(sparse_matrix).toarray()[0]
two_grams = pd.DataFrame(frequencies, index=word_vectorizer2.get_feature_names(), columns=['frequency']).sort_values('frequency', ascending = False).reset_index()

two_grams.columns= ["2-Gram", "2-Gram Occurence"]

# Count the 3-grams
word_vectorizer3 = CountVectorizer(ngram_range=[3, 3], analyzer='word')
sparse_matrix = word_vectorizer3.fit_transform(reviews_nostop['Translated_Review'])
frequencies = sum(sparse_matrix).toarray()[0]
three_grams = pd.DataFrame(frequencies, index=word_vectorizer3.get_feature_names(), columns=['frequency']).sort_values('frequency', ascending = False).reset_index()

three_grams.columns= ["3-Gram", "3-Gram Occurence"]

# Count the 4-grams
word_vectorizer4 = CountVectorizer(ngram_range=[4, 4], analyzer='word')
sparse_matrix = word_vectorizer4.fit_transform(reviews_nostop['Translated_Review'])
frequencies = sum(sparse_matrix).toarray()[0]
four_grams = pd.DataFrame(frequencies, index=word_vectorizer4.get_feature_names(), columns=['frequency']).sort_values('frequency', ascending = False).reset_index()

four_grams.columns= ["4-Gram", "4-Gram Occurence"]

# Put the above together in one table:

all_grams = [one_grams, two_grams, three_grams, four_grams]

review_grams = pd.concat(all_grams, axis=1, join='outer')

export_csv = review_grams.to_csv (r'review_grams.csv', index = None, header=True) 

review_grams.head(20)<jupyter_output><empty_output><jupyter_text>WordCloudUsing **pillow** and **wordcloud** packages, we can generate a word cloud directly in PythonYou can play with the coloring, fonts, even the shape of the cloud. There is a really good DataCamp article on it: https://www.datacamp.com/community/tutorials/wordcloud-python<jupyter_code>from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator
import matplotlib.pyplot as plt

# Exporting the text with the stopwords removed and reading it back in as one continuous block of text.

cleanreviews = reviews_nostop.loc[:,'Translated_Review']
export_csv = cleanreviews.to_csv (r'cleanreviews.txt', index = None, header=False) 

text = ""
f = open('cleanreviews.txt')
for line in f:
    text = text + line.strip()

# Generate word cloud:
wordcloud = WordCloud(background_color="white").generate(text)

# Display the generated image:
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis("off")
plt.show()

#save it to file: 
wordcloud.to_file("reviews.png")<jupyter_output><empty_output><jupyter_text>Frequency DistributionYou can show the frequency distribution of 1-grams. The simplest way is to install the **yellowbrick** package to visualize the frequency of the words in the data. Remember to remove the stop words.<jupyter_code>from yellowbrick.text import FreqDistVisualizer  #had to do some wrestling to get this to import!
from sklearn.feature_extraction.text import CountVectorizer

# Visualizing 'Translated_Review' in the data frame where the stop words were removed.

vectorizer = CountVectorizer()
#complete the following line: 
docs = vectorizer.fit_transform(reviews_nostop['Translated_Review'])
features = vectorizer.get_feature_names()

visualizer = FreqDistVisualizer(features=features)
visualizer.fit(docs)
visualizer.poof()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11225616488774383, 'fraction_numerical': 0.008833271991166729, 'mean_word_length': 7.387345679012346, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '664864', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1815, 'n_tokens_neox': 1720, 'n_words': 425, 'path': 'Yoga_Meditation_Apps/Yoga_Meditation_Apps_Review_Analysis.ipynb', 'repo_name': 'lizawood/apps-and-games', 'size': '5434'}