<jupyter_start><jupyter_text>Mathematical Model For Linear Data<img src="https://regenerativetoday.com/wp-content/uploads/2019/06/GradientDescentWithMutlipleLocalMinimum.jpg"     alt="Gradient Descent"     style="float: left; margin-right: 10px;" />--- ABSTRACTInformation is mounting significantly and hungry for knowledge. As data is expanding, the world is moving in chasing information with the assistance of analytics in this era of Big Data. The flooding of information arising out of different areas is named for automated learning techniques for data analysis, which is implied as Machine Learning (ML) or Artificial Intelligence (AI). Linear Regression is a statistical strategy for plotting the line and is used for predictive analysis. Gradient descent is the cycle that uses a cost function on gradients for limiting the complexity in computing mean square error. The above-mentioned methods are implemented in the project using python programming tools for analyzing the datasets.*Keywords: Machine Learning, Artificial Intelligence, Linear Regression, Predictive Analysis, Gradient descent, Cost Function*--- PROBLEM STATEMENTTo propose a “Mathematical Model for Linear Data” comparing the accuracy of the model with the existing linear fitting function.--- Dataset: ***Communities and Crime Data Set***Communities within the United States. The data combines socio-economic data from the 1990 US Census, law enforcement data from the 1990 US LEMAS survey, and crime data from the 1995 FBI UCR . Overview of dataset:|Dataset Characteristics |No. of Instances | Area  |Attribute Characteristics|No. of Attributes|Date Donated|Missing value||------------------------|-----------------|-------|-------------------------|-----------------|------------|-------------||Multivariative          | 1994            |Social |Real                     |128              |13-07-2009  |Yes          | Sample Data| polulation(x)| ViolentCrimesPerPop(y)| |--------------|-----------------------||0.19          |0.2                    ||0             |0.67                   |   |0             |0.43                   |   |0.04	       |0.12                   ||0.01	       |0.03                   ||0.02	       |0.14                   ||0.01	       |0.03                   ||0.01	       |0.55                   ||0.03	       |0.53                   ||0.01	       |0.15                   |***Source of dataset: http://archive.ics.uci.edu/ml/datasets/communities+and+crime*****Creator:** <NAME> (redmond '@' lasalle.edu); Computer Science; La Salle University; Philadelphia, PA, 19141, USA-- culled from 1990 US Census, 1995 US FBI Uniform Crime Report, 1990 US Law Enforcement Management and Administrative Statistics Survey, available from ICPSR at U of Michigan.-- Donor: <NAME> (redmond '@' lasalle.edu); Computer Science; La Salle University; Philadelphia, PA, 19141, USA-- Date: July 2009 --- Submitted By :  |Name                     |Roll no    |Sap ID    |Branch          ||-------------------------|-----------|----------|----------------||***<NAME>***    |R133218033 |500071369 |Btech-CSE-BFSI  ||***<NAME>***         |R133218004 |500068007 |Btech-CSE-BFSI  ||***<NAME>***|R133218015 |500069480 |Btech-CSE-BFSI  | Under the guidance of : Dr <NAME> *Assistant Professor, SOCS*  *Department of Informatics* ***University of Petroleum and Energy studies***<jupyter_code>import numpy as np                                   #importing numpy library
import pandas as pd                                  #importing pandas library


class Reg:                                           # here we make Class Reg
    
    def __init__(self):
        pass
    def sum(l, power):                               # sum() return the sum of the list with power
        res = 0
        for i in l:
            res += i**power        
        return res
    
    def multiply_of_x_y(l1, l2):                     # multiply_of_x_y() return the summationof  multiplication of two list(x,y) 
        res = 0
        for i in range(len(l1)):
            res += (l1[i]*l2[i])        
        return res
    
    def equation(sum_x, sum_x2, sum_y, sum_xy):      # euation() finds the coefficient of linear equation (slope and y-intercept)
        n = 38
        p = np.array([[sum_x,n], [sum_x2,sum_x]])    # Coefficient matrix
        q = np.array([sum_y, sum_xy])                # Ordinate or “dependent variable” values
        
        # 'np.linalg.solve' used to Solve a linear matrix equation, or system of linear scalar equations
        # Computes the “exact” solution, x, of the well-determined, i.e., full rank, linear matrix equation ax = b.
        # and return 'Solution to the system a x = b. Returned shape is identical to b'
        res = np.linalg.solve(p, q)                  
        return res
    
    def predict(x, res):                            # predict() return the predicted value of y and store it into y_pred[] list
        y_pred = []
        for i in x:
            y_pred.append(res[0] * i + res[1])
        return y_pred
    
def main():
    x = [0.04,0.01,0.02,0.01,0.01,0.03,0.01,0.13,0.02,0.03,0.01,0.04,0.03,0.15,0.01,0.02,0.01,0.11,0.02,0.06,0.01,0.01,0.01,0.04,0.09,0.03,0.03,0.01,0.02,0.01,0.01,0.06,0.09,0.02,0.12,0.03,0.04,0.01]
    y = [0.12,0.03,0.14,0.03,0.55,0.53,0.15,0.24,0.08,0.06,0.09,0.21,0.3,0.49,0.07,0.15,0.21,0.63,0.31,0.84,0.1,0.49,0.16,0.15,0.31,0.08,0.11,0.22,0.13,0.07,0.11,0.06,0.22,0.17,0.16,0.2,0.11,0.12]
    
    r = Reg                     #creates a new instance of the class 'Reg' and assigns this object to the local variable 'r'    
    
    sum_x = r.sum(x, 1)         # call the function sum with parameter x with power 1 and store into sum_x
    sum_y = r.sum(y, 1)         # call the function sum with parameter y with power 1 and store into sum_y
    sum_x2 = r.sum(x, 2)        # call the function sum with parameter x with power 2 and store into sum_x2
    
    sum_xy = r.multiply_of_x_y(x, y) # call the function multiply_of_x_y with parameter x and y
    
    result = []                      # make empty list 'result'
    
    result = r.equation(sum_x, sum_x2, sum_y, sum_xy)  # call the function equation()
    
    y_pred = r.predict(x, result)    # call the predict for Yhat_predicted
    
    print('Actual value \n',y)       # print Actual value of y
    print("\n")
    print('Predicted value \n',y_pred) # print predicted value of y
        
    
    # Finding Mean Squared Error and Root Mean Sqaured Error   
    summ = 0  
    n = len(y) 
    for i in range (0,n):  
      diff = y[i] - y_pred[i]  
      squared_diff = diff**2   
      summ = summ + squared_diff  
    MSE = summ/n
    
    # finding Relative Squared Error and R-Squared
    summ1 = 0
    summ2 = 0
    n = len(y)
    def Ymean(y):
        return sum(y) / len(y)
    ymean = Ymean(y)
    
    for i in range (0,n):  
      diff1 = y[i] - y_pred[i]
      diff2 = y[i] - ymean
      squared_diff1 = diff1**2
      squared_diff2 = diff2**2
      summ1 = summ1 + squared_diff1
      summ2 = summ2 + squared_diff2
    RSE = summ1/summ2
    
    print("\n")
    print("------------------------------------------------------------")
    print("|\tPerformance Measure of our Mathematical Model\t   |")
    print("------------------------------------------------------------")
    print("|  Mean Squared Error(Cost Function)\t\t","|",round(MSE,5),"|")
    print("------------------------------------------------------------")
    print("|  Root Mean Squared Error is   \t\t", "|",round(MSE**0.5,5),"|")
    print("------------------------------------------------------------")
    print("|  Relative Squared Error            \t\t","|",round(RSE,5),"|")
    print("------------------------------------------------------------")
    print("|  R-Squared \t\t\t\t\t","|",round(1-RSE,5),"|")
    print("------------------------------------------------------------")
if __name__ == "__main__":
    main()<jupyter_output>Actual value 
 [0.12, 0.03, 0.14, 0.03, 0.55, 0.53, 0.15, 0.24, 0.08, 0.06, 0.09, 0.21, 0.3, 0.49, 0.07, 0.15, 0.21, 0.63, 0.31, 0.84, 0.1, 0.49, 0.16, 0.15, 0.31, 0.08, 0.11, 0.22, 0.13, 0.07, 0.11, 0.06, 0.22, 0.17, 0.16, 0.2, 0.11, 0.12]


Predicted value 
 [0.2210875331564986, 0.16618037135278507, 0.18448275862068958, 0.16618037135278507, 0.16618037135278507, 0.2027851458885941, 0.16618037135278507, 0.38580901856763916, 0.18448275862068958, 0.2027851458885941, 0.16618037135278507, 0.2210875331564986, 0.2027851458885941, 0.4224137931034482, 0.16618037135278507, 0.18448275862068958, 0.16618037135278507, 0.34920424403183015, 0.18448275862068958, 0.2576923076923076, 0.16618037135278507, 0.16618037135278507, 0.16618037135278507, 0.2210875331564986, 0.31259946949602113, 0.2027851458885941, 0.2027851458885941, 0.16618037135278507, 0.18448275862068958, 0.16618037135278507, 0.16618037135278507, 0.2576923076923076, 0.31259946949602113, 0.18448275862068958, 0.36750663129973465, 0.202785145888[...]<jupyter_text>After Cleaning the Data we got MSE=0.02853 but our Goal is MSE=0.0160so we in our futher step we apply Gradient Descent Minimization Alorithms for Minimize the cost Function  Apply Gradient Descent Algorithms for Minimizing the Cost Function*Apply gradient descent until converengce*<jupyter_code>x = [0.04,0.01,0.02,0.01,0.01,0.03,0.01,0.13,0.02,0.03,0.01,0.04,0.03,0.15,0.01,0.02,0.01,0.11,0.02,0.06,0.01,0.01,0.01,0.04,0.09,0.03,0.03,0.01,0.02,0.01,0.01,0.06,0.09,0.02,0.12,0.03,0.04,0.01]
y = [0.12,0.03,0.14,0.03,0.55,0.53,0.15,0.24,0.08,0.06,0.09,0.21,0.3,0.49,0.07,0.15,0.21,0.63,0.31,0.84,0.1,0.49,0.16,0.15,0.31,0.08,0.11,0.22,0.13,0.07,0.11,0.06,0.22,0.17,0.16,0.2,0.11,0.12]

sample_size = len(x)   # finding the size of x variable

def calculate_cost(q0,q1): # thsi Function return the 
    
    q1x = np.dot(q1,x)                 # Dot product of two arrays
    q1xq0 = np.add(q1x, q0)            # Add arguments element-wise
    q1xq0_y = np.subtract(q1xq0, y)    # Subtract arguments, element-wise (here we calute updated error)
    q1xq0_y_square = np.dot(q1xq0_y, q1xq0_y) # calculating the updated sqaured error
    return np.sum(q1xq0_y_square) * (1.0/(2*sample_size))
    

def minimize(q0,q1, includeX=False):
    q1x = np.dot(q1,x)                # Dot product of two arrays
    q1xq0 = np.add(q1x, q0)           # Add arguments element-wise
    q1xq0_y = np.subtract(q1xq0, y)   # Subtract arguments, element-wise
    q1xq0_yx = q1xq0_y                
    if includeX:
        q1xq0_yx = np.dot(q1xq0_y, x) # Dot product of two arrays
    res_matrix = np.sum(q1xq0_yx) * (1.0/sample_size)
    return res_matrix


def compute_and_show(alpha, iteration):

    costs = []                         # making empty list 'costs=[]' where we collect the cost
    q0 = 0                             # initialy   q0(y-intercept) = 0
    q1 = 0                             # intiatialy q1(slope) = 0
    for idx in range(iteration):       # run the loop no of iteration times
        q0 = q0 - (alpha * minimize(q0,q1))        #finding new q0(y-intercept) using gradient descent (Partial derivative formula for q0)
        q1 = q1 - (alpha * minimize(q0,q1,True))   # findind new q1(slope) using gradient descent (Partial derivative formula for q1)
        cost = calculate_cost(q0, q1)              
        costs.append(cost)

    new_df = pd.DataFrame(costs)
    new_df.plot()
    print ('q0 {0}, q1 {1}'.format(q0, q1))<jupyter_output><empty_output><jupyter_text>****Our Goal for MSE(Cost Function)=0.0160**** ***Attempt 1st***  ***A very small learning rate (α = 0.001)*****After 2000 minimization, the cost not reached to our Goal (around 0.020)**<jupyter_code>compute_and_show(0.001, 2000)               # call the function compute_and_show(learning rate, no. of iteration)<jupyter_output>q0 0.18633227409614428, q1 0.011952553210254594<jupyter_text>***Attempt 2nd***  ***A very High learning rate (α = 5)*****After 2000 minimization, the cost shoots up after 250 attempts**<jupyter_code>compute_and_show(5, 2000)<jupyter_output>c:\users\dell\appdata\local\programs\python\python39\lib\site-packages\numpy\core\fromnumeric.py:87: RuntimeWarning: overflow encountered in reduce
  return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
<ipython-input-78-3973eca79561>:9: RuntimeWarning: invalid value encountered in add
  q1xq0 = np.add(q1x, q0)
<ipython-input-78-3973eca79561>:17: RuntimeWarning: invalid value encountered in add
  q1xq0 = np.add(q1x, q0)           # Add arguments element-wise<jupyter_text>***Attempt 3rd*** ***A good learning rate (α = 0.12)***Gradient descent is converged after 100-150 attempts. but can be improved<jupyter_code>compute_and_show(0.12, 2000)<jupyter_output>q0 0.19649674038817982, q1 0.521756892438034<jupyter_text>***Attempt 4th******A good learning rate (α = 0.5)***Gradient descent is converged after 100-150 attempts. but can be improved<jupyter_code>compute_and_show(0.5, 2000)<jupyter_output>q0 0.16489384515180008, q1 1.3722893537018366<jupyter_text>***Attempt 5th******A good learning rate (α = 1)***Gradient descent is converged after 500 attempts.***here we reached our Goal. Cost Function(MSE) is less than 0.0160***<jupyter_code>compute_and_show(1, 2000)<jupyter_output>q0 0.1521420448143877, q1 1.715479792978122
{'fraction_non_alphanumeric': 0.1642512077294686, 'fraction_numerical': 0.13383152173913043, 'mean_word_length': 2.3541772151898734, 'pattern_counts': {'":': 1, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '445728', 'license': 'Apache-2.0', 'n_tokens_mistral': 5578, 'n_tokens_neox': 4540, 'n_words': 1181, 'path': 'code/Final Code ( Minor II - End Semester ).ipynb', 'repo_name': 'Anupriya212001/MATHEMATICAL-MODEL-OF-LINEAR-DATA', 'size': '13296'}