<jupyter_start><jupyter_text>BackgroundThis statistical model is used when there are one or more independent variables to predict a dichotomous or binary outcome. Additionally, the model will assign a probability that the prediction is correct. Logistic regression relies on the following assumptions:* Outcome is binary* Predictors are continuous or categorical* Predictors are independent of one another* Sample size is adequate  Loading the Breast cancer dataset<jupyter_code>from sklearn.datasets import load_breast_cancer
cancer_data = load_breast_cancer()
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Examining the data<jupyter_code>print(cancer_data.data.shape)<jupyter_output>(569, 30)<jupyter_text>We see that there are 569 records with 30 different attributes. If we want to access to the description of the dataset we can do:<jupyter_code>print(cancer_data.DESCR)<jupyter_output>Breast Cancer Wisconsin (Diagnostic) Database
=============================================

Notes
-----
Data Set Characteristics:
    :Number of Instances: 569

    :Number of Attributes: 30 numeric, predictive attributes and the class

    :Attribute Information:
        - radius (mean of distances from center to points on the perimeter)
        - texture (standard deviation of gray-scale values)
        - perimeter
        - area
        - smoothness (local variation in radius lengths)
        - compactness (perimeter^2 / area - 1.0)
        - concavity (severity of concave portions of the contour)
        - concave points (number of concave portions of the contour)
        - symmetry 
        - fractal dimension ("coastline approximation" - 1)

        The mean, standard error, and "worst" or largest (mean of the three
        largest values) of these features were computed for each image,
        resulting in 30 features.  For instance, field 3 is Mean Radius, field
        13 is [...]<jupyter_text>If we want to access the names of the attributes:<jupyter_code>print(cancer_data.feature_names)<jupyter_output>['mean radius' 'mean texture' 'mean perimeter' 'mean area'
 'mean smoothness' 'mean compactness' 'mean concavity'
 'mean concave points' 'mean symmetry' 'mean fractal dimension'
 'radius error' 'texture error' 'perimeter error' 'area error'
 'smoothness error' 'compactness error' 'concavity error'
 'concave points error' 'symmetry error' 'fractal dimension error'
 'worst radius' 'worst texture' 'worst perimeter' 'worst area'
 'worst smoothness' 'worst compactness' 'worst concavity'
 'worst concave points' 'worst symmetry' 'worst fractal dimension']<jupyter_text>And now let's access the name of the labels:<jupyter_code>print(cancer_data.target_names)<jupyter_output>['malignant' 'benign']<jupyter_text>Splitting data into training and test setsLet's split the cancer data into a training set that will be used in order to build our logistic regression model and the test set that will be used to assess the performance of our model<jupyter_code>from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(cancer_data.data, cancer_data.target, test_size=0.25, random_state=0)<jupyter_output><empty_output><jupyter_text>The test dataset will have 25% of the observations  Building our model<jupyter_code>from sklearn.linear_model import LogisticRegression<jupyter_output><empty_output><jupyter_text>We make an instance of the Logistic Regression model with the default parameters<jupyter_code>logisticRegr = LogisticRegression(verbose=1)<jupyter_output><empty_output><jupyter_text>Now, we train the model with our cancer data<jupyter_code>logisticRegr.fit(x_train, y_train)<jupyter_output>[LibLinear]<jupyter_text>Measuring model performanceFirst, we are going to make predictions using our new model and the test data<jupyter_code>predictions = logisticRegr.predict(x_test)<jupyter_output><empty_output><jupyter_text>We can use the score function in order to calculate the mean accuracy on the test data and labels<jupyter_code>score = logisticRegr.score(x_test, y_test)
print(score)<jupyter_output>0.958041958042<jupyter_text>In order to visualize the correctness of our predictions, we can also create a confusion matrix<jupyter_code>import matplotlib.pyplot as plt
import seaborn as sns
from sklearn import metrics

cm = metrics.confusion_matrix(y_test, predictions)
print(cm)<jupyter_output>[[52  1]
 [ 5 85]]<jupyter_text>The matrix above can be embellished by using seaborn:<jupyter_code>plt.figure(figsize=(9,9))
sns.heatmap(cm, annot=True, fmt=".0f", linewidths=.5, square = True, cmap = 'Blues_r');
plt.ylabel('Actual label');
plt.xlabel('Predicted label');
all_sample_title = 'Accuracy Score: {0}'.format(score)
plt.title(all_sample_title, size = 15);<jupyter_output><empty_output><jupyter_text>Observing the table we can see that there were 85 True Positives, 52 True Negatives, 5 False Negatives and 1 False Positive   Assessing the influence of each predictor on the outcome<jupyter_code>import numpy as np    
from sklearn.linear_model import LogisticRegression

x1 = np.random.randn(100)
x2 = 4*np.random.randn(100)
x3 = 0.5*np.random.randn(100)
y = (3 + x1 + x2 + x3 + 0.2*np.random.randn()) > 0
X = np.column_stack([x1, x2, x3])

m = LogisticRegression()
m.fit(X, y)

# The estimated coefficients will all be around 1:
print(m.coef_)

# Those values, however, will show that the second parameter
# is more influential
print(np.std(X, 0)*m.coef_)<jupyter_output>[[ 1.12742653  1.13311707  0.46075686]]
[[ 1.20964183  4.3823029   0.20870382]]<jupyter_text>Now, let's practice by creating a binary classification dataset using the `make_classification` method<jupyter_code>from sklearn.datasets import make_classification

X, y = make_classification(n_samples=5000, n_features=2, n_informative=1,
                           n_redundant=1, n_repeated=0, n_classes=2,
                           n_clusters_per_class=1,
                           class_sep=0.8, random_state=0)<jupyter_output><empty_output><jupyter_text>In this case, the dataset has only one informative predictor.  Now, let's build the logistic regression model:<jupyter_code>import numpy as np    
from sklearn.linear_model import LogisticRegression

m = LogisticRegression()
m.fit(X, y)

# Those values, however, will show that the second parameter
# is more influential
print(np.std(X, 0)*m.coef_)<jupyter_output>[[-0.14809515  4.78741821]]
{'fraction_non_alphanumeric': 0.08937763219466542, 'fraction_numerical': 0.02729683356730619, 'mean_word_length': 5.348514851485149, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 57, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '399749', 'license': 'Apache-2.0', 'n_tokens_mistral': 2127, 'n_tokens_neox': 1950, 'n_words': 633, 'path': 'MACHINE_LEARNING/Logistic_Regression/.ipynb_checkpoints/Example of a Binary classifier using Logistic Regression-checkpoint.ipynb', 'repo_name': 'elowy01/PYTHON_NOTES', 'size': '6411'}