<jupyter_start><jupyter_text>0. Get some data and choose some features<jupyter_code>df = pd.read_excel('./data/churn.xls', sheet_name='churn')
df.head()
df.columns
y = df['Churn']
X = df[['Account Length', 'VMail Message', 'Day Mins', 'Eve Mins', 'Night Mins',
       'Intl Mins', 'CustServ Calls',  "Int'l Plan", 'VMail Plan',
       'Day Calls', 'Day Charge', 'Eve Calls', 'Eve Charge', 'Night Calls',
       'Night Charge', 'Intl Calls', 'Intl Charge']]<jupyter_output><empty_output><jupyter_text>1. Train test split<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y, random_state=123, train_size=0.8)<jupyter_output><empty_output><jupyter_text>2. Choose a model and train it<jupyter_code>from sklearn.linear_model import LogisticRegression
logreg = LogisticRegression()
logreg.fit(X_train, y_train)<jupyter_output>C:\Users\HP\Anaconda3\envs\a1\lib\site-packages\sklearn\linear_model\logistic.py:432: FutureWarning: Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.
  FutureWarning)<jupyter_text>3. What is this model doing? Evaluate on the test set!<jupyter_code>y_pred = logreg.predict(X_test)
y_pred[:5]
y_test[:5]
from sklearn.metrics import confusion_matrix
confusion_matrix(y_true=y_test, y_pred=y_pred)
y_proba = logreg.predict_proba(X_test)
y_proba
threshold = 0.2
y_new_pred = [1 if val>threshold else 0 for val in y_proba[:,1]]
confusion_matrix(y_true=y_test, y_pred=y_new_pred)<jupyter_output><empty_output><jupyter_text>A perfect model would have non-zero entries on the diagonal, but zero outside (no false positives / no false negatives).  How to improve the model?- Use raw probabilities for calculating churn/no churn.- Select some of the variables => more or less automated.- Adding some preprocessing, e.g. clustering.- Do we need more data or do we need better algorithm? => Learning curve.  Coming soon... - What makes the model work? Why some predictions are made (both globally and individually). To calculate feature importance in logistic regression we use the coefficients * standard deviation in the corresponding column.<jupyter_code>X.shape
import numpy as np
# Feature importances
np.std(X)*logreg.coef_[0]
logreg.coef_[0]<jupyter_output><empty_output><jupyter_text>Look at correlations (using a heatmap)<jupyter_code>import seaborn as sns
sns.heatmap(X.corr())<jupyter_output><empty_output><jupyter_text>Univariate feature selection<jupyter_code>from sklearn.feature_selection import SelectKBest
sel = SelectKBest(k=5) # Defines that we want k features from our data
sel.fit(X,y) # .fit() makes the algorithm aware of your data
X_new = sel.transform(X)
chosen_features = sel.get_support()
chosen_features
X.columns[chosen_features] # This is some pandas-specific magic<jupyter_output><empty_output><jupyter_text>Retry with X_new ?<jupyter_code>X_train_new, X_test_new, y_train, y_test
 = train_test_split(X_new, y, random_state=123, train_size=0.8)
logreg.fit(X_train_new, y_train)
y_pred_new = logreg.predict(X_test_new)
confusion_matrix(y_test, y_pred_new)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10586734693877552, 'fraction_numerical': 0.01020408163265306, 'mean_word_length': 6.688725490196078, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '401812', 'license': 'Apache-2.0', 'n_tokens_mistral': 1126, 'n_tokens_neox': 1083, 'n_words': 288, 'path': '06 Logistic Regression.ipynb', 'repo_name': 'jpmaldonado/np-dl-telco', 'size': '3136'}