<jupyter_start><jupyter_text>Gradient Descent Implementation In this Python notebook we will go through an example of implementing **Gradient Descent** in simple and multiple linear regression, for this we will be using housing dataset.<jupyter_code>#Importing the dataset
import pandas as pd
housing = pd.read_csv('D:\\upgrad\\ML\\Gradient+Descent\\gd\\Housing.csv')
housing.head()
# Converting Yes to 1 and No to 0
housing['mainroad'] = housing['mainroad'].map({'yes': 1, 'no': 0})
housing['guestroom'] = housing['guestroom'].map({'yes': 1, 'no': 0})
housing['basement'] = housing['basement'].map({'yes': 1, 'no': 0})
housing['hotwaterheating'] = housing['hotwaterheating'].map({'yes': 1, 'no': 0})
housing['airconditioning'] = housing['airconditioning'].map({'yes': 1, 'no': 0})
housing['prefarea'] = housing['prefarea'].map({'yes': 1, 'no': 0})
#Converting furnishingstatus column to binary column using get_dummies
status = pd.get_dummies(housing['furnishingstatus'])
housing = pd.concat([housing,status],axis=1)
housing.drop(['furnishingstatus'],axis=1,inplace=True)
housing.head()
#Normalisisng the data
housing = (housing - housing.mean())/housing.std()
housing.head()
# Putting feature variable to X
X = housing['area']
# Putting response variable to y
y = housing['price']
# Conventional way to import seaborn
import seaborn as sns
import matplotlib.pyplot as plt

plt.style.use('ggplot')
# To visualise in the notebook
%matplotlib inline
# Visualise the relationship between the features and the response using scatterplots
sns.pairplot(housing, x_vars='area', y_vars='price',size=7, aspect=0.7, kind='scatter')<jupyter_output><empty_output><jupyter_text>**For linear regressions we use a cost function known as the mean squared error or MSE.**  Now we will apply partial derivative with respect to m and c and will equate it to zero to find the least value of m and c for which our cost function get the lowest value as possible.  Now to apply gradient descent from scratch we need our X and y variables as numpy arrays, Let's convert them.<jupyter_code>import numpy as np
X = np.array(X)
y = np.array(y)
# Applying gradient descent by creating UDF.
def gradient(X, y, m_current=0, c_current=0, iters=1000, learning_rate=0.001):
    N = float(len(y))
    gd_df = pd.DataFrame( columns = ['m_current', 'c_current','cost'])
    for i in range(iters):
        y_current = (m_current * X) + c_current
        cost = sum([data**2 for data in (y-y_current)]) / N
        m_gradient = -(2/N) * sum(X * (y - y_current))
        c_gradient = -(2/N) * sum(y - y_current)
        m_current = m_current - (learning_rate * m_gradient)
        c_current = c_current - (learning_rate * c_gradient)
        gd_df.loc[i] = [m_current,c_current,cost]
    return(gd_df)

gradient(X,y)
gradient(X,y).reset_index().plot.line(x='index', y=['cost'])<jupyter_output>C:\Users\<NAME>\Anaconda3\lib\site-packages\pandas\plotting\_core.py:1716: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access
  series.name = label<jupyter_text>Applying Gradient Descent for multiple variable<jupyter_code># Putting feature variable to X
X = housing[['area','bedrooms']]
# Putting response variable to y
y = housing['price']
## Add a columns of 1s as intercept to X.
X['intercept'] = 1
X = X.reindex_axis(['intercept','area','bedrooms'], axis=1)
X.head()
import numpy as np
X = np.array(X)
y = np.array(y)
# Theta needed to be changed with the number of response varaible used.
theta = np.matrix(np.array([0,0,0])) 
print(theta)
alpha = 0.01
iterations = 1000
print(X.T.shape)
import numpy as np

def compute_cost(X, y, theta):
    return np.sum(np.square(np.matmul(X, theta) - y)) / (2 * len(y))<jupyter_output><empty_output><jupyter_text>More on [Numpy Matmul](https://docs.scipy.org/doc/numpy/reference/generated/numpy.matmul.html)<jupyter_code>def gradient_descent_multi(X, y, theta, alpha, iterations):
    theta = np.zeros(X.shape[1])
    m = len(X)
    gdm_df = pd.DataFrame( columns = ['Bets','cost'])

    for i in range(iterations):
        gradient = (1/m) * np.matmul(X.T, np.matmul(X, theta) - y)
        theta = theta - alpha * gradient
        cost = compute_cost(X, y, theta)
        gdm_df.loc[i] = [theta,cost]

    return gdm_df
gradient_descent_multi(X, y, theta, alpha, iterations)
gradient_descent_multi(X, y, theta, alpha, iterations).reset_index().plot.line(x='index', y=['cost'])<jupyter_output>C:\Users\<NAME>\Anaconda3\lib\site-packages\pandas\plotting\_core.py:1716: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access
  series.name = label
{'fraction_non_alphanumeric': 0.1419869775257299, 'fraction_numerical': 0.012182314639781559, 'mean_word_length': 4.937655860349127, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '478119', 'license': 'MIT', 'n_tokens_mistral': 1690, 'n_tokens_neox': 1605, 'n_words': 447, 'path': 'Gradient Descent.ipynb', 'repo_name': 'aryancodify/LinearRegression', 'size': '4773'}