<jupyter_start><jupyter_text>Playstore Data Analysis Import libraries<jupyter_code>import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Import Data<jupyter_code>data = pd.read_csv("playstore-analysis.csv")
data.head()<jupyter_output><empty_output><jupyter_text>Shape of data<jupyter_code>data.shape<jupyter_output><empty_output><jupyter_text>1. Data clean up – Missing value treatment a. Drop records where rating is missing since rating is our target/study variable<jupyter_code>#Display data information

data.info()
#Count the total null values 

data.isnull().sum()
#Drop records where rating is missing

data.dropna(subset=['Rating'], inplace=True)
#Now chack how many record after drop records where rating is missing

data.isnull().sum()
data.shape<jupyter_output><empty_output><jupyter_text>b. Check the null values for the Android Ver column.<jupyter_code>data[data['Android Ver'].isna()]<jupyter_output><empty_output><jupyter_text>b(i) Are all 3 records having the same problem?  Yes all three records having same problem with Andriod version is NaN<jupyter_code># Index number of 3rd record i.e. record for “Life Made WIFI …”

data[data['Android Ver'].isna()].index[-1]
# b(ii) Drop the 3rd record i.e. record for “Life Made WIFI …”

data.drop(10472, inplace=True)
data[data['Android Ver'].isna()]
#Find out the Mode value

data['Android Ver'].value_counts()
# b(iii) Replace remaining missing values with the mode

data["Android Ver"].fillna('4.1 and up', inplace = True) 
# Find the mode 
data['Android Ver'].value_counts()
# C. Current ver – replace with most common value

data["Current Ver"].fillna(data['Current Ver'].mode()[0], inplace = True)<jupyter_output><empty_output><jupyter_text>2. Data clean up – correcting the data types<jupyter_code>#Display data types 

data.dtypes<jupyter_output><empty_output><jupyter_text>2 (a). Which all variables need to be brought to numeric types?Ans: Reviews and Installs need to be brought to numeric types.<jupyter_code># 2.(b) b. Price variable – remove $ sign and convert to float

data['Price'] = [x.lstrip('$') for x in data['Price']]
data['Price'] = pd.to_numeric(data['Price'], downcast='float')
data['Reviews'] = pd.to_numeric(data['Reviews'])
data.dtypes
import re
data['Installs']
# 2(c). Installs – remove ‘,’ and ‘+’ sign, convert to integer

data['Installs'] = [re.sub('[+,]', '', x) for x in data['Installs']]
data['Installs'] = pd.to_numeric(data['Installs'])
data['Installs']
# 2(d) Convert all other identified columns to numeric

#data['Reviews']= data['Reviews'].astype('int')<jupyter_output><empty_output><jupyter_text>3. Sanity checks –check for the following and handle accordingly<jupyter_code>print(data['Rating'].min())
print(data['Rating'].max())
data.loc[data.Rating < 1] & data.loc[data.Rating > 5]<jupyter_output><empty_output><jupyter_text>3 a. Avg. rating should be between 1 and 5, as only these values are allowed on the playstore.i. Are there any such records? Drop if so.Ans:- There are no such records with rating less than 1 or greater than 5.<jupyter_code>#b. Reviews should not be more than installs as only those who installed can review the app.
# i. Are there any such records? Drop if so.

data[data['Reviews'] > data['Installs']]<jupyter_output><empty_output><jupyter_text>Yes, there are 7 records where Review is greater than Installs.<jupyter_code># Drop all 7 records.

data.drop(data[data['Reviews'] > data['Installs']].index, inplace=True)
data[data['Reviews'] > data['Installs']]<jupyter_output><empty_output><jupyter_text>Droped all 7 records  4. Identify and handle outliers –<jupyter_code># 4 (a). Price column
# i. Make suitable plot to identify outliers in price

sns.boxplot(data['Price'])
# 4 (ii). Do you expect apps on the play store to cost $200? Check out these cases

data.drop(data[data['Price']>70].index, inplace=True)
# 4 (iii). After dropping the useless records, make the suitable plot again to identify outliers

sns.boxplot(data['Price'])
# 4.(iv). drop Limit data to records with price < $30

data.drop(data[data['Price']>=30].index, inplace=True)
sns.boxplot(data['Price'])<jupyter_output><empty_output><jupyter_text>b. Reviews columni. Make suitable plot<jupyter_code># Try with bar plot
sns.barplot(y = data['Reviews'])
# Try with Violinplot 

sns.violinplot(x = data['Reviews'])
#ii. Limit data to apps with < 1 Million reviews

data[data['Reviews']>=1000000]
data.drop(data[data['Reviews']>=1000000].index, inplace=True)
# Dropped 704 rows 

data[data['Reviews']>=1000000]
#C(i). What is the 95th percentile of the installs?

data[data['Installs'].quantile(0.95) < data['Installs']]
# C (ii). Drop records having a value more than the 95th percentile

data.drop(data[data['Installs'].quantile(0.95) < data['Installs']].index, inplace=True)
data[data['Installs'].quantile(0.95) < data['Installs']]<jupyter_output><empty_output><jupyter_text>5. Data analysis to answer business questions 5. What is the distribution of ratings like? (use Seaborn) More skewed towards higher/lowervalues?<jupyter_code>data.isnull().sum()
sns.distplot(data['Rating'])
# a. How do you explain this?

print('The skewness of this distribution is',data['Rating'].skew())
print('The Median of this distribution {} is greater than mean {} of this distribution'.format(data.Rating.median(),data.Rating.mean()))
# b. What is the implication of this on your analysis?

data['Rating'].mode()<jupyter_output><empty_output><jupyter_text>Since mode = median > mean, the distribution of Rating is Negatively Skewed.Therefore distribution of Rating is more Skewed towards lower values.  6. What are the top Content Rating values?<jupyter_code># 6(a) a. Are there any values with very few records?

data['Content Rating'].value_counts()<jupyter_output><empty_output><jupyter_text>Adults only 18+ and Unrated are values with very few records so we drop them.<jupyter_code>data[data['Content Rating'] == 'Unrated']
data[data['Content Rating'] == 'Adults only 18+']
#Replacing unwanted values with NaN

A = []
for i in data['Content Rating']:
    A.append(i.replace('Adults only 18+','NaN').replace('Unrated','NaN'))

data['Content Rating']=A
# Droping the NaN values.

B = data[data["Content Rating"] == 'NaN'].index
data.drop(labels=B, inplace=True)
print('droped cols',B)
#Check 

data['Content Rating'].value_counts()<jupyter_output><empty_output><jupyter_text>7. Effect of size on rating  a. Make a joinplot to understand the effect of size on rating<jupyter_code>sns.set(style="white", color_codes=True)
sns.jointplot(x='Size', y='Rating', data=data)<jupyter_output><empty_output><jupyter_text>b. Do you see any patterns?Yes, patterns can be observed between Size and Rating ie. their is correlation between Size and Rating. c. How do you explain the pattern?Generally on increasing Rating, Size of App also increases. But this is not always true ie. for higher Rating, their is constant Size. Thus we can conclude that their is positive correlation between Size and Rating.  8. Effect of price on rating<jupyter_code>data.dtypes
# a. Make a jointplot (with regression line)
sns.jointplot(x='Price', y='Rating', data=data, kind='reg')<jupyter_output><empty_output><jupyter_text>b. What pattern do you see?Ans: Generally on increasing the Price, Rating remains almost constant greater than 4. c. How do you explain the pattern?Ans:- Since on increasing the Price, Rating remains almost constant greater than 4. Thus it can be concluded that their is very weak Positive correlation between Rating and Price.<jupyter_code>data.corr()<jupyter_output><empty_output><jupyter_text>d. Replot the data, this time with only records with price > 0<jupyter_code>P = data.loc[data.Price>0]
sns.jointplot(x='Price', y='Rating', data=P, kind='reg')
plt.show()<jupyter_output><empty_output><jupyter_text>e. Does the pattern change?Yes, On limiting the record with Price > 0, the overall pattern changed a slight i.e their is very weakly Negative Correlation between Price and Rating.<jupyter_code>P.corr()<jupyter_output><empty_output><jupyter_text>f. What is your overall inference on the effect of price on the ratingGenerally increasing the Prices, doesn't have signifcant effect on Higher Rating. For Higher Price, Rating is High and almost constant ie greater than 4  9. Look at all the numeric interactions together – a. Make a pairplort with the colulmns - 'Reviews', 'Size', 'Rating', 'Price'<jupyter_code>sns.pairplot(data, vars=['Reviews', 'Size', 'Rating', 'Price'])<jupyter_output><empty_output><jupyter_text>10. Rating vs. content rating a. Make a bar plot displaying the rating for each content rating<jupyter_code>sns.set(style="whitegrid")
data.groupby(['Content Rating'])['Rating'].count().plot.bar(color="darkblue")<jupyter_output><empty_output><jupyter_text>b. Which metric would you use? Mean? Median? Some other quantile?We must use Median in this case as we are having Outliers in Rating. Because in case of Outliers , median is the best measure of central tendency.<jupyter_code>plt.boxplot(data['Rating'])
plt.show()<jupyter_output><empty_output><jupyter_text>c. Choose the right metric and plot<jupyter_code>data.groupby(['Content Rating'])['Rating'].median().plot.barh(color="darkblue")
plt.show()<jupyter_output><empty_output><jupyter_text>11. Content rating vs. size vs. rating – 3 variables at a time a. Create 5 buckets (20% records in each) based on Size<jupyter_code>bins=[0, 20000, 40000, 60000, 80000, 100000]
data['Bucket Size'] = pd.cut(data['Size'], bins, labels=['0-20k','20k-40k','40k-60k','60k-80k','80k-100k'])
pd.pivot_table(data, values='Rating', index='Bucket Size', columns='Content Rating')<jupyter_output><empty_output><jupyter_text>b. By Content Rating vs. Size buckets, get the rating (20th percentile) for each combination<jupyter_code>C = pd.pivot_table(data, values='Rating', index='Bucket Size', columns='Content Rating', aggfunc=lambda x:np.quantile(x,0.2))
C<jupyter_output><empty_output><jupyter_text>c. Make a heatmap of thisi. Annotatedii. Greens color map<jupyter_code># Annotated

f,ax = plt.subplots(figsize=(7, 7))
sns.heatmap(C, annot=True, linewidths=.7, fmt='.1f',ax=ax)
plt.show()
# ii.Greens color map

f,ax = plt.subplots(figsize=(7, 7))
sns.heatmap(C, annot=True, linewidths=.5, cmap='Greens',fmt='.1f',ax=ax)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1300789524359715, 'fraction_numerical': 0.016079337569805507, 'mean_word_length': 6.515918958031838, 'pattern_counts': {'":': 0, '<': 119, '<?xml version=': 0, '>': 126, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '28', 'dir': 'jupyter-structured-clean-dedup', 'id': '514660', 'license': 'MIT', 'n_tokens_mistral': 3554, 'n_tokens_neox': 3347, 'n_words': 1052, 'path': 'pyhton1_Assignment.ipynb', 'repo_name': 'rekha364/Python', 'size': '10386'}