<jupyter_start><jupyter_text>Import Data<jupyter_code>processed_data_path = os.path.join(os.path.pardir, 'data','processed')
train_file_path = os.path.join(processed_data_path, 'train.csv')
test_file_path = os.path.join(processed_data_path,'test.csv')
df_train = pd.read_csv(train_file_path, index_col='PassengerId')
df_test = pd.read_csv(test_file_path, index_col='PassengerId')
df_train.info()
# Survived is the output and rest is the input

df_test.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 418 entries, 892 to 1309
Data columns (total 33 columns):
Survived              418 non-null int64
Age                   418 non-null float64
Fare                  418 non-null float64
FamilySize            418 non-null int64
IsMother              418 non-null int64
IsMale                418 non-null int64
Deck_A                418 non-null float64
Deck_B                418 non-null float64
Deck_C                418 non-null float64
Deck_D                418 non-null float64
Deck_E                418 non-null float64
Deck_F                418 non-null float64
Deck_G                418 non-null float64
Deck_Z                418 non-null float64
Pclass_1              418 non-null float64
Pclass_2              418 non-null float64
Pclass_3              418 non-null float64
Title_Lady            418 non-null float64
Title_Master          418 non-null float64
Title_Miss            418 non-null float64
Title_Mr              418 non-null float[...]<jupyter_text>Data preparation machine learning algorithms expect numerical arrays.We will seperate output features and input features of the train data. Output feature is 'Survived' columnInput feature is all the other column after Survived column<jupyter_code>X = df_train.loc[:, 'Age':].as_matrix().astype('float')
y = df_train['Survived'].ravel()  #ravel is a numpy function for creating one dimensional array
print X.shape, y.shape
!pip install sklearn
# we will divide the data into test and traninig data

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size = 0.2, random_state =0)
print X_train.shape, y_train.shape
print X_test.shape, y_test.shape
print 'Mean survival in train: {0:.3f}'.format(np.mean(y_train))
print 'Mean survival in test: {0: .3f}'.format(np.mean(y_test))<jupyter_output>Mean survival in train: 0.383
Mean survival in test:  0.385<jupyter_text>Check Scikit Learn Version for making Baseline Model as we will use DummyClassifier which is available after 0.19 version<jupyter_code>import sklearn
sklearn.__version__<jupyter_output><empty_output><jupyter_text>Baseline Model<jupyter_code># import function
from sklearn.dummy import DummyClassifier
# create model
model_dummy = DummyClassifier(strategy='most_frequent', random_state=0)
#   Train mode;
model_dummy.fit(X_train,y_train) # in fit function , we provide two things, 1: input data, 2:output data<jupyter_output><empty_output><jupyter_text>Pass test data to evaluate the performanceThe model will first predict the output on X_test and the it will compare the predicted output(X_test) with the original output (y_test).For classification problem, this score is called accuracy<jupyter_code>print 'score for baseline model : {0:.2f}'.format(model_dummy.score(X_test, y_test))<jupyter_output>score for baseline model : 0.61<jupyter_text>The above score means without any machine learning algorithm, if we always predict the value of 0 or nonsurvived then we get the 61% accuracy  Performance Matrics<jupyter_code>from sklearn.metrics import accuracy_score, confusion_matrix, precision_score, recall_score<jupyter_output><empty_output><jupyter_text>The method signature for all these matrix are also samefirst parameter = actual outputsecond parameter = predicted output<jupyter_code># accuracy score
print 'accuracy for baseline model: {0:.2f}'.format(accuracy_score(y_test, model_dummy.predict(X_test)))
# confusion matrix
print 'confusion matrix for baseline model: \n{0}'.format(confusion_matrix(y_test, model_dummy.predict(X_test)))
# precision and recall scores
print 'precision for baseline model: {0:.2f}'.format(precision_score(y_test, model_dummy.predict(X_test)))
print 'recall for baseline model: {0:.2f}'.format(recall_score(y_test, model_dummy.predict(X_test)))<jupyter_output>precision for baseline model: 0.00
recall for baseline model: 0.00<jupyter_text>Logistic Regression Model<jupyter_code># import function
from sklearn.linear_model import LogisticRegression
# create model
model_lr_initial = LogisticRegression(random_state=0)
# train model
model_lr_initial.fit(X_train, y_train)
# evaluate model
print "Score for logistic model :{0:.2f}".format(model_lr_initial.score (X_test, y_test)
                                                )<jupyter_output>Score for logistic model :0.83<jupyter_text>Performance metrics<jupyter_code># accuracy
print 'accuracy for logistic regression - initial version: {0:.2f}'.format(accuracy_score(y_test, model_lr_initial.predict(X_test)))
# confusion matrix for logistic regression
print 'confusion matrix for logistic regression -initial version: \n {0}'.format(confusion_matrix(y_test, model_lr_initial.predict(X_test)))
# precision
print 'precision for logistic regression - initial version : {0:.2f}'.format(precision_score(y_test, model_lr_initial.predict(X_test)))
# recall for logistic regression
print 'recall for logistic regression {0: .2f}'.format(recall_score(y_test, model_lr_initial.predict(X_test)))<jupyter_output>recall for logistic regression  0.78<jupyter_text>Model coefficients<jupyter_code># model coefficients or model weights or parameters
model_lr_initial.coef_<jupyter_output><empty_output><jupyter_text>Logistic regresion model with Hyperparameter optimization<jupyter_code># logistic model
model_lr = LogisticRegression(random_state=0)<jupyter_output><empty_output><jupyter_text>Using GridSearch technic for hyperparameter optimization<jupyter_code># importing GridSearchCV
from sklearn.model_selection import GridSearchCV
# you can chose C as much as you want and the values
parameters = {'C':[1.0, 2.0,90.0,200.0,500,800.0, 950.0],'penalty':['l1','l2'] }

# creating grid search object
# this function takes, base model, gridparameters, and crossvalidation fold, here we will give cv=3 means 3 fold cross validation
clf = GridSearchCV(model_lr, param_grid= parameters, cv=3)

# traning the model
clf.fit(X_train, y_train)
# for finding the best combination we can use best_params 
clf.best_params_
# best score
print 'best score : {0:.2f}'.format(clf.best_score_)
# evaluate
print 'evaluate the model: {0:.2f}'.format(clf.score(X_test, y_test))<jupyter_output>evaluate the model: 0.83<jupyter_text>Feature Normalization and Standardization<jupyter_code># MinMaxScaler for normalization
# StandardScaler function for standardization
from sklearn.preprocessing import MinMaxScaler, StandardScaler<jupyter_output><empty_output><jupyter_text>feature normalizationFeature normalization means all the features in the data should be on the same scale.There are two types of scaling.Scale Type 1 : 0  to 1Scale Type 2 : -1  to 1By default it will scale between 0 to 1<jupyter_code># create scaler object and pass the data into fit_transform function
scalar = MinMaxScaler()
X_train_scale = scalar.fit_transform(X_train)
X_train_scale[:,0].min(), X_train_scale[:,0].max()<jupyter_output><empty_output><jupyter_text>We need to scale our test data also because we train the model with scale data<jupyter_code># normalize test data
X_test_scale = scalar.transform(X_test)<jupyter_output><empty_output><jupyter_text>Feature Standardizationstandardize the feature in such a way that all of the features have 	*	Mean = 0	* Variance = 1<jupyter_code>scaler_stand = StandardScaler()
X_train_scale = scaler_stand.fit_transform(X_train)
X_test_scale = scaler_stand.transform(X_test)<jupyter_output><empty_output><jupyter_text>Create model after feature scaling and standardization<jupyter_code>model_lr = LogisticRegression(random_state=0)
# you can chose C as much as you want and the values
parameters = {'C':[1.0, 2.0,90.0,200.0,500,800.0, 950.0],'penalty':['l1','l2'] }
clf = GridSearchCV(model_lr, parameters, cv=3)
clf.fit(X_train_scale, y_train)
print 'Best score {0:.5f}'.format(clf.best_score_)
print 'Score for logistic regression {0:.5f}'.format(clf.score(X_test_scale, y_test))<jupyter_output>Score for logistic regression 0.84358
{'fraction_non_alphanumeric': 0.08710594927707987, 'fraction_numerical': 0.030931500355534488, 'mean_word_length': 5.4126139817629175, 'pattern_counts': {'":': 0, '<': 62, '<?xml version=': 0, '>': 62, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '203275', 'license': 'MIT', 'n_tokens_mistral': 2839, 'n_tokens_neox': 2630, 'n_words': 752, 'path': 'notebooks/Titanic Datasets Building Predictive Models.ipynb', 'repo_name': 'sdrsiddiqui/titanic_kaggle_challenge', 'size': '8438'}