<jupyter_start><jupyter_text>Logistic Regression<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Loading the data<jupyter_code>df = pd.read_csv('ChurnData.csv')
df.head()<jupyter_output><empty_output><jupyter_text>Data selection and preprocessing<jupyter_code>X = df[['tenure', 'age', 'address', 'income', 'ed', 'employ', 'equip']]
X.head()
y = df['churn'].astype('int')
y.head()<jupyter_output><empty_output><jupyter_text>Normalizing the data<jupyter_code>from sklearn import preprocessing
ss_X = preprocessing.StandardScaler()
ss_X.fit(X)
X = ss_X.transform(X)
X[0:5]<jupyter_output><empty_output><jupyter_text>Train-test split<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)<jupyter_output><empty_output><jupyter_text>Training the model<jupyter_code>from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
model<jupyter_output><empty_output><jupyter_text>Lets build our model using __LogisticRegression__ from Scikit-learn package. This function implements logistic regression and can use different numerical optimizers to find parameters, including ‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’ solvers. You can find extensive information about the pros and cons of these optimizers if you search it in internet.The version of Logistic Regression in Scikit-learn, support regularization. Regularization is a technique used to solve the overfitting problem in machine learning models.__C__ parameter indicates __inverse of regularization strength__ which must be a positive float. Smaller values specify stronger regularization. Now lets fit our model with train set:<jupyter_code>model.C = 0.1      # Changing the value of C to make it more regularized
model.solver = 'liblinear'
model
model.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Prediction<jupyter_code>y_hat = model.predict(X_test)
y_hat[0:5]<jupyter_output><empty_output><jupyter_text>**predict_proba**  returns estimates for all classes, ordered by the label of classes. So, the first column is the probability of class 1, P(Y=1|X), and second column is probability of class 0, P(Y=0|X):<jupyter_code>y_hat_proba = model.predict_proba(X_test)
y_hat_proba[0:5]<jupyter_output><empty_output><jupyter_text>Evaluation<jupyter_code>from sklearn import metrics
print('Accuracy of the Logistic Model using accuracy_score is %.9f' % metrics.accuracy_score(y_hat, y_test))<jupyter_output>Accuracy of the Logistic Model using accuracy_score is 0.700000000<jupyter_text>Jaccard-Index Score<jupyter_code>from sklearn import metrics
print('Accuracy of the Logistic Model using jaccard_similarity_score is %.9f' % metrics.jaccard_similarity_score(y_hat, y_test))<jupyter_output>Accuracy of the Logistic Model using jaccard_similarity_score is 0.700000000<jupyter_text>Confusion Matrix Confusion Matrix is way representing the result in a 2d-matrix.                     Predicted No        Predicted Yes                Actual No        a        b                Actual Yes        c        d<jupyter_code>from sklearn import metrics
print(metrics.confusion_matrix(y_test, y_hat))
print(metrics.confusion_matrix(y_test, y_hat, labels=[1, 0]))  # Tweaking the positions of rows and columns<jupyter_output>[[ 6  5]
 [ 7 22]]<jupyter_text>Log Loss<jupyter_code>from sklearn import metrics
print('Accuracy of the Logistic Model using log_loss is %.9f' % metrics.log_loss(y_test, y_hat_proba))<jupyter_output>Accuracy of the Logistic Model using log_loss is 0.490226788
{'fraction_non_alphanumeric': 0.08655162969049575, 'fraction_numerical': 0.015064365927143249, 'mean_word_length': 6.640167364016737, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '75152', 'license': 'MIT', 'n_tokens_mistral': 1246, 'n_tokens_neox': 1186, 'n_words': 321, 'path': 'Week 3/Logistic Regression.ipynb', 'repo_name': 'Razdeep/Coursera-ML-with-Python-notebooks', 'size': '3651'}