<jupyter_start><jupyter_text>Creating the database using python<jupyter_code>import random
y=[]
x=[]

for i in range(1500):
  x_domain = random.randrange(100)
  y_domain = (random.randrange(1,10)/10)*30 + (x_domain)
  x.append(x_domain)
  y.append(y_domain)

print(x[:5])
print(y[:5])<jupyter_output>[97, 14, 74, 66, 98]
[115.0, 32.0, 89.0, 78.0, 122.0]<jupyter_text>Plotting the data<jupyter_code>import matplotlib.pyplot as plt

plt.scatter(x,y,s=2)
plt.show()<jupyter_output><empty_output><jupyter_text>Saving the data using Pickle<jupyter_code>import pickle

pickle_out = open("saved_x_values.pickle","wb")
pickle.dump(x, pickle_out)
pickle_out.close()

pickle_out = open("saved_y_values.pickle","wb")
pickle.dump(y, pickle_out)
pickle_out.close()<jupyter_output><empty_output><jupyter_text>Importing back the data back using pickle<jupyter_code>import pickle


pickle_in = open("drive/My Drive/Colab Notebooks/1. Predicting a scatter graph/Study files/saved_x_values.pickle","rb")
a = pickle.load(pickle_in)

pickle_in = open("drive/My Drive/Colab Notebooks/1. Predicting a scatter graph/Study files/saved_y_values.pickle","rb")
b = pickle.load(pickle_in)

print(a[:5])
print(b[:5])<jupyter_output>[97, 14, 74, 66, 98]
[115.0, 32.0, 89.0, 78.0, 122.0]<jupyter_text>Modeling the Linear Regression modelcurrently data is imported into a and bour job is to train on a and use it to predict -> b<jupyter_code>import matplotlib.pyplot as plt
import keras
model = keras.Sequential()
model.add(keras.layers.Dense(1, input_shape=(1,)))
model.compile(keras.optimizers.Adam(lr=1), 'mean_squared_error')

model.fit(a, b, epochs=30, batch_size=10)<jupyter_output>Epoch 1/30
1500/1500 [==============================] - 0s 256us/step - loss: 144.8333
Epoch 2/30
1500/1500 [==============================] - 0s 88us/step - loss: 105.2365
Epoch 3/30
1500/1500 [==============================] - 0s 97us/step - loss: 121.8771
Epoch 4/30
1500/1500 [==============================] - 0s 91us/step - loss: 109.0939
Epoch 5/30
1500/1500 [==============================] - 0s 92us/step - loss: 106.6567
Epoch 6/30
1500/1500 [==============================] - 0s 89us/step - loss: 104.8069
Epoch 7/30
1500/1500 [==============================] - 0s 90us/step - loss: 110.5028
Epoch 8/30
1500/1500 [==============================] - 0s 89us/step - loss: 162.3004
Epoch 9/30
1500/1500 [==============================] - 0s 88us/step - loss: 81.6231
Epoch 10/30
1500/1500 [==============================] - 0s 90us/step - loss: 102.5167
Epoch 11/30
1500/1500 [==============================] - 0s 88us/step - loss: 155.4922
Epoch 12/30
1500/1500 [=============================[...]<jupyter_text>Checking if it fits the data properlyblue = input datared = the line of best fit that the model thinks is the best at predicting outcome<jupyter_code>plt.scatter(a, b, s=1)
b_pread = model.predict(a)
plt.plot(a, b_pread, color='red')<jupyter_output><empty_output><jupyter_text>So as seen above, the model thinks that the red line is the best way way to predict the data.notice that the gradient of the data that I had manually put does not co-relate with the gradient of the best fit line drawn by the model. My guess is that since I added randomness to the training data, the data may get squeed with the data density clumping in certain space so the model will try to fit into that and introduce a slight curve.  Using the model to predict a data that it has never seen before<jupyter_code>print(model.predict([50]))<jupyter_output>[[63.92444]]
{'fraction_non_alphanumeric': 0.21499573984663448, 'fraction_numerical': 0.09571144561204203, 'mean_word_length': 5.607879924953096, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '231327', 'license': 'MIT', 'n_tokens_mistral': 1511, 'n_tokens_neox': 1224, 'n_words': 337, 'path': 'project files/Scatter graph prediction.ipynb', 'repo_name': 'monacotime/1.Predicting-a-scatter-graph', 'size': '3521'}