<jupyter_start><jupyter_text>There are 5 features namely 'Weight', 'Weight1', 'Length', 'Height', 'Width' and "Cost" is the target variable.<jupyter_code># Know the shape of data
data.shape<jupyter_output><empty_output><jupyter_text>There are 160 rows and 6 columns available in the dataset.<jupyter_code>data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 159 entries, 0 to 158
Data columns (total 6 columns):
 #   Column   Non-Null Count  Dtype  
---  ------   --------------  -----  
 0   Cost     159 non-null    float64
 1   Weight   159 non-null    float64
 2   Weight1  159 non-null    float64
 3   Length   159 non-null    float64
 4   Height   159 non-null    float64
 5   Width    159 non-null    float64
dtypes: float64(6)
memory usage: 7.6 KB<jupyter_text>All the columns are of float type. There is no categorical or string data.<jupyter_code>data.isnull().sum()<jupyter_output><empty_output><jupyter_text>There is no missing data. *That's really a very good new!* Let see the first few columns of the data.<jupyter_code>data.head()
data.describe()<jupyter_output><empty_output><jupyter_text>It appears that there is some **outlier** in the *Cost*, *Weight*, *Weight1* and *Height* columns. There is big difference in them 75th percentile of cost (650.00) and max value cost (1650.00). Similarly there is gap for other prescribed features as well.The min value of cost is 0.00 but the 25% tile is 120. Hence there is outlier in that region as well. Distribution of features is one effective way to find any outlier. We'll see the distribution of various features<jupyter_code>sns.distplot(data["Cost"])<jupyter_output><empty_output><jupyter_text>The last bin is kind of disjoint from the other histogram bins. This reinforced the idea that outliear exist. Also, the first bin contains price of 0 as well. There are very less price for a bag. The distant cost should be in the last bin of the plot. Let see deeper with other plots as well.<jupyter_code>sns.boxplot(x="Cost", data=data)<jupyter_output><empty_output><jupyter_text>*Cost* have outlier with values more that 1250<jupyter_code>data.tail()<jupyter_output><empty_output><jupyter_text>Similarly `boxplot` of *Weight*, *Weight1*, *Height* are clearly visible.<jupyter_code>sns.boxplot(x=data["Weight"])
sns.boxplot(x=data["Weight1"])
sns.boxplot(x=data["Length"])<jupyter_output><empty_output><jupyter_text>**From the above boxplots it is obvious that following are outliers*** data["Cost"] > 1250* data["Weight"] > 55* data["Weight1"] > 55* data["Length] > 65  Let see the outliers data.<jupyter_code>outliers = (data.loc[(data["Cost"] < 100) | (data["Cost"] > 1250)]).sort_values(by=["Cost"])
outliers.head()<jupyter_output><empty_output><jupyter_text>There are multiple strategies to deal with outliers. For now I am removing them.<jupyter_code>data = data.loc[(data["Cost"] > 100) & (data["Cost"] < 1250)]<jupyter_output><empty_output><jupyter_text>**Building Linear Regression Models**  *Cost/Loss function is defined as:* $ J(\theta) = \frac{1}{(2m)} \sum\limits_{i=1}^{m} (h(\theta)^{(i)} - y^{(i)})^2 \\\\m = total \ number \ of \ observations \\\theta = weights \\h(\theta) = \ hypothesis \\y = observed \ target \ value \\i = i^{th} \ sample \ data$  *Gradient defined as:* $ \frac{\partial J(\theta)}{\partial \theta_j} = \frac{1}{(m)} \sum\limits_{i=1}^{m} ((h(\theta)^{(i)} - y^{(i)})^2)X_j^{(i)} \\\\m = total \ number \ of \ observations \\\theta = weights \\h(\theta) = \ hypothesis \\y = observed \ target \ value \\i = i^{th} \ sample \ data$  **Linear Regression using Mini-batch gradient descent** Calculating the cost give by: $ J(\theta) = \frac{1}{(2m)} \sum\limits_{i=1}^{m} (h(\theta)^{(i)} - y^{(i)})^2$<jupyter_code>def mini_batch_cal_cost(theta, X, y):
    m = len(y)
    predictions = X.dot(theta)
    cost = (1/2*m) * np.sum(np.square(predictions-y))
    return cost<jupyter_output><empty_output><jupyter_text>Calulating thetas given by: $\frac{\partial J(\theta)}{\partial \theta_j} = \frac{1}{(m)} \sum\limits_{i=1}^{m} ((h(\theta)^{(i)} - y^{(i)})^2)X_j^{(i)}$ and updating thetas on every iteration:$\theta_j := \theta_j - a.(\frac{\partial J(\theta)}{\partial \theta_j} = \frac{1}{(m)} \sum\limits_{i=1}^{m} ((h(\theta)^{(i)} - y^{(i)})^2)X_j^{(i)})$<jupyter_code>def minibatch_gradient_descent(X,y,theta,learning_rate=0.01, iterations=10, batch_size=20):
    m = len(y)
    cost_history = np.zeros(iterations)
    n_batches = int(m/batch_size)
    for it in range(iterations):
        cost =0.0
        indices = np.random.permutation(m)
        X = X[indices]
        y = y[indices]
        for i in range(0,m,batch_size):
            X_i = X[i:i+batch_size, :]
            y_i = y[i:i+batch_size]
            X_i = np.c_[np.ones(len(X_i)),X_i]
            prediction = np.dot(X_i,theta)
            theta = theta - (1/m)*learning_rate*( X_i.T.dot((prediction - y_i)))
            cost += mini_batch_cal_cost(theta,X_i,y_i)
        cost_history[it]  = cost
    return theta, cost_history
# making copy of data to work with
mini_batch_df = data.copy()
mini_batch_df.shape
# scaling data
mini_batch_scaler = StandardScaler()
mini_batch_scaled = mini_batch_scaler.fit_transform(mini_batch_df)
# separating feature and target variables.
mini_batch_X = mini_batch_scaled[:, 1:]
mini_batch_y = mini_batch_scaled[:, 0]
print(mini_batch_X.shape)
print(mini_batch_y.shape)
# split training and testing data
min_batch_X_train, min_batch_X_test, min_batch_y_train, min_batch_y_test = train_test_split(mini_batch_X, mini_batch_y, train_size=0.8)
print(min_batch_X_train.shape)
print(min_batch_X_test.shape)
print(min_batch_y_train.shape)
print(min_batch_y_test.shape)
min_batch_y_train = min_batch_y_train.reshape(-1, 1)
min_batch_y_test = min_batch_y_test.reshape(-1, 1)
print(min_batch_y_train.shape)
print(min_batch_y_test.shape)
# setting hyper parameters
lr = 0.05
n_iter = 150
mini_batch_theta = np.random.randn((min_batch_X_train.shape[1] + 1), 1)
mini_batch_theta, mini_batch_cost_history = minibatch_gradient_descent(min_batch_X_train, min_batch_y_train, mini_batch_theta, lr, n_iter, batch_size=4)
print("theta = ", mini_batch_theta)
print('MSE:  {:0.3f}'.format(mini_batch_cost_history[-1]))
# prepending 1 for vector multiplication.
mini_batch_X_test_1 = np.c_[np.ones((min_batch_X_test.shape[0], 1)), min_batch_X_test]
print(mini_batch_X_test_1.shape)
# making predictions
mini_batch_y_pred = np.dot(mini_batch_X_test_1, mini_batch_theta)<jupyter_output><empty_output><jupyter_text>**MSE, RMSE and R squared of the model are following:**<jupyter_code>print("Coefficient = ", mini_batch_theta[0, 0])
print("Mean Squared Error (MSE) = {:0.3f}".format(mini_batch_cost_history[-1]))
print("Root Mean Squared Error (RMSE) = {:0.3f}: ".format(np.sqrt(mini_batch_cost_history[-1])))
print("r2_score = {:0.3f}".format(metrics.r2_score(min_batch_y_test, mini_batch_y_pred)))
fig,ax = plt.subplots(figsize=(10,8))
_=ax.plot(range(n_iter),mini_batch_cost_history, 'b.')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1502634952285999, 'fraction_numerical': 0.0259222333000997, 'mean_word_length': 5.0639032815198615, 'pattern_counts': {'":': 0, '<': 55, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '249138', 'license': 'MIT', 'n_tokens_mistral': 2685, 'n_tokens_neox': 2510, 'n_words': 590, 'path': 'mini_batch_gradient_descent.ipynb', 'repo_name': 'vivek2089/gradient_descent_in_linear_regression', 'size': '7021'}