<jupyter_start><jupyter_text>Assignment-2 | <NAME> RA1911003010464  18CSE398J Machine Learning - Core Concepts with Applications  Performing Logistic Regression on Equity Apartments Dataset from Assignment-1 ***  Data Collection<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from matplotlib.axes._axes import _log as matplotlib_axes_logger
matplotlib_axes_logger.setLevel('ERROR')
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

file = 'Equity_Apartments_Data.csv'
df = pd.read_csv(file, nrows=1000)<jupyter_output><empty_output><jupyter_text>Data cleaning<jupyter_code>to_drop = ['Northern_Exposure',
           'Southern_Exposure',
           'Eastern_Exposure',
           'Western_Exposure',
           'City_Skyline',
           'Kitchen_Island',
           'Stainless_Appliances']

df.drop(to_drop, inplace = True, axis = 1)<jupyter_output><empty_output><jupyter_text>Columns that were not of much use are dropped.  Data Exploration<jupyter_code>Beds = df[df.Beds==1]
Beds.shape
NoBeds = df[df.Beds==0]
NoBeds.shape
df.groupby('Beds').mean()
df.plot.scatter('Beds','Floor',color='red', figsize=(16, 10))
plt.scatter(df.Beds,df.Balcony,marker='+',color='red')
df.plot.scatter('Units', 'Floor', figsize=(16, 10), color='purple')
df.Beds.hist()
plt.title('Histogram of Bedrooms')
plt.xlabel('Bedrooms')
plt.ylabel('Frequency')<jupyter_output><empty_output><jupyter_text>It is observed that properties with a single bedroom are most common.<jupyter_code>pd.crosstab(df.Beds,df.Balcony).plot(kind='bar')<jupyter_output><empty_output><jupyter_text>It is observed that properties with a double bedroom have a higher chance of also having a balcony.<jupyter_code>pd.crosstab(df.Balcony,df.Baths).plot(kind='bar')
pd.crosstab(df.City,df.Baths).plot(kind='bar')<jupyter_output><empty_output><jupyter_text>It is observed that in New York, most properties have alteast a single bathroom.<jupyter_code>df2 = df.rename(columns = {'sq.ft': 'sqft'}, inplace = False)
pd.crosstab(df.Renovated,df.Beds).plot(kind='bar')
pd.crosstab(df.City,df.Beds).plot(kind='bar')<jupyter_output><empty_output><jupyter_text>It is observed that in New York, properties with a single bedroom are most common.<jupyter_code>table=pd.crosstab(df.City,df.Beds)
table.div(table.sum(1).astype(float), axis=0).plot(kind='bar', stacked=True)
plt.title('Stacked Bar Chart of City vs Beds')
plt.xlabel('City')
plt.ylabel('Proportion')<jupyter_output><empty_output><jupyter_text>Data Selection<jupyter_code>subdf = df[['Baths','Floor','Beds','Office_Space','Balcony','City','Price']]
subdf.head()
City_dummies = pd.get_dummies(subdf.City, prefix="City")
df_with_dummies = pd.concat([subdf,City_dummies],axis='columns')
df_with_dummies.head()
df_with_dummies.drop('City',axis='columns',inplace=True)
df_with_dummies.head()<jupyter_output><empty_output><jupyter_text>Model Building<jupyter_code>X = df_with_dummies
X.head()
y = df.Beds<jupyter_output><empty_output><jupyter_text>y is our target variable that is the number of bedrooms.<jupyter_code>print(y)<jupyter_output>0      0
1      1
2      2
3      0
4      1
      ..
995    2
996    2
997    2
998    2
999    2
Name: Beds, Length: 1000, dtype: int64<jupyter_text>Model Fitting<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y,train_size=0.3)
from sklearn.linear_model import LogisticRegression
model = LogisticRegression(solver='lbfgs', max_iter=10000)
model.fit(X_train, y_train)
model.predict(X_test)
y_pred = model.predict(X_test)
y_pred
model.coef_
model.intercept_
fig = plt.figure(figsize=(10, 6))
plt.scatter(y_test,y_pred, color='r')
plt.xlabel('Y Test')
plt.ylabel('Predicted Y')<jupyter_output><empty_output><jupyter_text>Confusion Matrix<jupyter_code>from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)
cm<jupyter_output><empty_output><jupyter_text>Accuracy<jupyter_code>print('Accuracy of logistic regression classifier on test set: {:.2f}'.format(model.score(X_test, y_test)))<jupyter_output>Accuracy of logistic regression classifier on test set: 0.96<jupyter_text>Model evaluation<jupyter_code>from sklearn import metrics

print('MAE:', metrics.mean_absolute_error(y_test, y_pred))
print('MSE:', metrics.mean_squared_error(y_test, y_pred))
print('RMSE:', np.sqrt(metrics.mean_squared_error(y_test, y_pred)))
import seaborn as sns

sns.distplot((y_test-y_pred),bins=10, color='r');<jupyter_output><empty_output><jupyter_text>K-fold model validation<jupyter_code>from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X_train, y_train, scoring='r2', cv=5)
scores
len(X_train.columns)
from sklearn.feature_selection import RFE
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import KFold

folds = KFold(n_splits = 5, shuffle = True, random_state = 100)

hyper_params = [{'n_features_to_select': list(range(1, 12))}]

model = LogisticRegression(solver='lbfgs', max_iter=10000)
model.fit(X_train, y_train)
rfe = RFE(model)             

model_cv = GridSearchCV(estimator = rfe, 
                        param_grid = hyper_params, 
                        scoring= 'r2', 
                        cv = folds, 
                        verbose = 1,
                        return_train_score=True)      

# fit the model
model_cv.fit(X_train, y_train)  
cv_results = pd.DataFrame(model_cv.cv_results_)
cv_results
from sklearn import model_selection
from sklearn.model_selection import cross_val_score
kfold = model_selection.KFold(n_splits= 5)
modelCV = LogisticRegression(solver='lbfgs', max_iter=10000)
scoring = 'accuracy'
results = model_selection.cross_val_score(modelCV, X_train, y_train, cv=kfold, scoring=scoring)
print("K-fold cross validation average accuracy: %.3f" % (results.mean()))
from sklearn.metrics import classification_report
print(classification_report(y_test, y_pred))
plt.figure(figsize=(12,4))

plt.plot(cv_results["param_n_features_to_select"], cv_results["mean_test_score"])
plt.plot(cv_results["param_n_features_to_select"], cv_results["mean_train_score"])
plt.xlabel('number of features')
plt.ylabel('r-squared')
plt.title("Optimal Number of Features")
plt.legend(['test score', 'train score'], loc='upper left')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12575039494470774, 'fraction_numerical': 0.018325434439178515, 'mean_word_length': 6.413348946135831, 'pattern_counts': {'":': 0, '<': 60, '<?xml version=': 0, '>': 60, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '286259', 'license': 'MIT', 'n_tokens_mistral': 2407, 'n_tokens_neox': 2261, 'n_words': 412, 'path': 'ML-LogisticReg.ipynb', 'repo_name': 'sahilsarin390/ML-LogisticRegression-Project', 'size': '6336'}