<jupyter_start><jupyter_text>Learning Pandas and Matplotlib Pandas is pythons library that enables broad possibilities for data analysis.By using Pandas it is very easy to upload, manage and analyse data from different tables by using SQL-like commands. Moreover, in connection with the libraries Matplotlib and Seaborn, Pandas gives broad opportunities to visualise the data.<jupyter_code>import pandas as pd
import numpy as np

from matplotlib import pyplot as plt
from matplotlib.pyplot import rcParams
rcParams['figure.figsize'] = 8, 5
import seaborn as sns<jupyter_output><empty_output><jupyter_text>We study the main methods of packages Pandas and Matplotlib by working with the dataset that describes the churn rate of the customers of a telecom company.  Exercise 1:Read the data from the file 'telecom_churn.csv' and display the first 5 rows by using the method 'head'.  Hint : method read_csv may be useful<jupyter_code>### Write Your code here ###
tel_churn = pd.read_csv('telecom_churn.csv')
tel_churn.head()<jupyter_output><empty_output><jupyter_text>Exercise 2:Display the size of the data array, information about it, and its main statistical characteristics. Hint: use methods shape, info, describe.<jupyter_code>### Write Your code here ###
tel_churn.info()

tel_churn.shape
tel_churn.describe()<jupyter_output><empty_output><jupyter_text>Exercise 3:Convert column 'Churn' to the int64 type.  Hint: use method astype.Plot the distribution of the churn and  loyal clients in the bar plot. The figure should look like this:<jupyter_code>### Write Your code here ###
tel_churn.Churn =tel_churn.Churn.astype(int)

sns.countplot(x='Churn',data=tel_churn)<jupyter_output><empty_output><jupyter_text>SortingExercise 4:Sort the dataframe you have obtained in exercise 3 by the value in 'Total day charge' in descending/ascending order.Also sort it by using the column 'Churn' as the primary key and 'Total eve calls' as the secondary key. Try different combinations of ordering.  Hint: use the method sort_values<jupyter_code>### Write Your code here ###
tel_churn.sort_values(by=['Total day charge'])
#, ascending=False
tel_churn.sort_values(by=['Churn','Total eve calls'])<jupyter_output><empty_output><jupyter_text>Indexing and extracting information from the dataframeBy using Pandas dataframes we are able to index and extract information from the dataset. You can index the information stored in dataframe either by names or by indices. In the first case you use the command loc, in the second iloc.  Hint:  Use logical indexing for the columns and the groupby method to solve the tasks.Exercise 5:* Display the mean churn rate of the clients.* Now we want to analyse statistical information only for the clients, which are or aren't loyal to their telecom-company (field 'Churn' in dataframe). Extract the loyal and non-loyal clients from the table seperately and display the means of their charateristics in a single dataframe.* How long do the non-loyal users talk during the day (on average)?* What is the maximum length of the international calls for the loyal users that do not use the international plan?<jupyter_code>### Display the mean churn rate of the clients. ###
tel_churn.Churn.mean()
tel_churn.groupby('Churn').groups
#Now we want to analyse statistical information only for the clients,
#which are or aren't loyal to their telecom-company (field 'Churn' in dataframe). 
# Extract the loyal and non-loyal clients from the table seperately and display the means of their charateristics in a single dataframe.

tel_churn.groupby('Churn').mean()
#How long do the non-loyal users talk during the day (on average)?
#users[(users.sex == 'F') | (users.age < 30)].head(3)
non_loyals =tel_churn[(tel_churn.Churn == 1)]
#non_loyals.head()
non_loyals['Total day minutes'].mean()
#What is the maximum length of the international calls for the loyal users that do not use the international plan?
tel_churn =tel_churn.rename(columns={"International plan": "int_plan"})
loyals =tel_churn[(tel_churn.Churn == 0) & ((tel_churn.int_plan) == 'NO')]
loyals.head()<jupyter_output><empty_output><jupyter_text>Distribution of the featuresExercise 6:Plot the distribution of the features that have numerical values.  Hint: use the method hist which can also be applied from the pandas dataframe. It should look like that:What do you observe? From which probability distribution could each feature be generated?<jupyter_code>tel_churn.hist()
### Write Your code here ###<jupyter_output><empty_output><jupyter_text>Pivot tables and graphicsWe want to see how the instances are distributed between two categories: 'International plan' and 'Churn'. Exercise 7:* Build the cross table between the features using the method crosstab.* Visualize the distribution for the feature 'Churn', depending on the value of the features 'International plan', 'Voice mail plan', and 'Customer service calls'.Hint: Use commands plt.sublot and sns.subplots.What do you see? What conclusions can be drawn? What feature (intuitively) can be more important for Churn prediction?<jupyter_code>### Write Your code here ###
pd.crosstab(tel_churn.int_plan, tel_churn.Churn, margins=True)

sns.jointplot(x='Churn',y='Customer service calls',data=tel_churn)

sns.barplot(x='Churn',y='Customer service calls',data=tel_churn)
sns.pairplot(tel_churn,
            y_vars=['Churn'],
                  x_vars=['Customer service calls', 'Voice mail plan','int_plan'])
g = sns.FacetGrid(tel_churn, col="Churn")
g.map(plt.scatter, "Customer service calls", "int_plan", alpha=.7)
g.add_legend();<jupyter_output><empty_output><jupyter_text>Exercise 8:Add a new feature to the dataframe which will describe whether or not the user has done more than 3 service calls. Investigate how this feature describes the churn rate.<jupyter_code>### Write Your code here ### data.date.apply(lambda x: datetime.strptime(x, "%Y-%m-%d"))
#tel_churn = tel_churn.rename(columns={"Customer service calls":"cscalls"})
tel_churn['threeServCall'] =tel_churn.cscalls.apply(lambda x: 1 if (x > 3) else 0)
tel_churn.head()
pd.crosstab(tel_churn.threeServCall, tel_churn.Churn, margins=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08729514846665586, 'fraction_numerical': 0.0038942073665422685, 'mean_word_length': 5.949267192784667, 'pattern_counts': {'":': 2, '<': 38, '<?xml version=': 0, '>': 38, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '6836', 'license': 'MIT', 'n_tokens_mistral': 1892, 'n_tokens_neox': 1797, 'n_words': 723, 'path': 'Lab1/.ipynb_checkpoints/LearningPandasandPylab-checkpoint.ipynb', 'repo_name': 'pratik98/Machine-LearningSummer2020', 'size': '6163'}