<jupyter_start><jupyter_text>Data Preprocessing<jupyter_code>import pandas as pd
data_1 = pd.read_csv('data-1.csv')
data_2 = pd.read_csv('data-2.csv')
data_3 = pd.read_csv('data-3.csv')
data_4 = pd.read_csv('data-4.csv')
data_5 = pd.read_csv('data-5.csv')
data_6 = pd.read_csv('data-6.csv')
data_1<jupyter_output><empty_output><jupyter_text>Statistics  Dataset 1<jupyter_code>x1_mean = data_1.x.mean()
y1_mean = data_1.y.mean()
x1_std = data_1.x.std()
y1_std = data_1.y.std()
x1_min = data_1.x.min()
x1_max = data_1.x.max()
y1_min = data_1.y.min()
# y1_min
y1_max = data_1.y.max()
# y1_max
# data_1.corr()
data1_corr = data_1.corr()['x']['y']
# data1_corr
print('Data Summry of The First Dataset:')
print('The mean of x variable is %f;' % x1_mean)
print('The mean of y variable is %f;' % y1_mean)
print('The standard deviation of x variable is %f;' % x1_std)
print('The standard deviation of y variable is %f;' % y1_std)
print('The range of x variable is [%f, %f];' % (x1_min, x1_max))
print('The range of y variable is [%f, %f];' % (y1_min, y1_max))
print('The correlation between the x and y variables are %f.' % data1_corr)
import matplotlib.pyplot as plt
plt.scatter(data_1.x, data_1.y)
plt.title('Sctter Plot of Dataset 1')
print('One Sentence Description of Dataset 1:')
print('According to the coorelation value and the scatter plot of the first dataset, the two variables have strong linear correlation.')<jupyter_output>One Sentence Description of Dataset 1:
According to the coorelation value and the scatter plot of the first dataset, the two variables have strong linear correlation.<jupyter_text>Dataset 2<jupyter_code>x2_mean = data_2.x.mean()
y2_mean = data_2.y.mean()
x2_std = data_2.x.std()
y2_std = data_2.y.std()
x2_min = data_2.x.min()
x2_max = data_2.x.max()
y2_min = data_2.y.min()
y2_max = data_2.y.max()
data2_corr = data_2.corr()['x']['y']
print('Data Summry of The Second Dataset:')
print('The mean of x variable is %f;' % x2_mean)
print('The mean of y variable is %f;' % y2_mean)
print('The standard deviation of x variable is %f;' % x2_std)
print('The standard deviation of y variable is %f;' % y2_std)
print('The range of x variable is [%f, %f];' % (x2_min, x2_max))
print('The range of y variable is [%f, %f];' % (y2_min, y2_max))
print('The correlation between the x and y variables are %f.' % data2_corr)
plt.scatter(data_2.x, data_2.y)
plt.title('Sctter Plot of Dataset 2')
print('One Sentence Description of Dataset 2:')
print('According to the coorelation value and the scatter plot of the second dataset, the two variables have strong linear correlation.')<jupyter_output>One Sentence Description of Dataset 2:
According to the coorelation value and the scatter plot of the second dataset, the two variables have strong linear correlation.<jupyter_text>Dataset 3<jupyter_code>x3_mean = data_3.x.mean()
y3_mean = data_3.y.mean()
x3_std = data_3.x.std()
y3_std = data_3.y.std()
x3_min = data_3.x.min()
x3_max = data_3.x.max()
y3_min = data_3.y.min()
y3_max = data_3.y.max()
data3_corr = data_3.corr()['x']['y']
print('Data Summry of The Third Dataset:')
print('The mean of x variable is %f;' % x3_mean)
print('The mean of y variable is %f;' % y3_mean)
print('The standard deviation of x variable is %f;' % x3_std)
print('The standard deviation of y variable is %f;' % y3_std)
print('The range of x variable is [%f, %f];' % (x3_min, x3_max))
print('The range of y variable is [%f, %f];' % (y3_min, y3_max))
print('The correlation between the x and y variables are %f.' % data3_corr)
plt.scatter(data_3.x, data_3.y)
plt.title('Sctter Plot of Dataset 3')
print('One Sentence Description of Dataset 3:')
print('According to the coorelation value and the scatter plot of the third dataset, although the two variables have strong linear correlation, there are 4 clusters on the graph.')<jupyter_output>One Sentence Description of Dataset 3:
According to the coorelation value and the scatter plot of the third dataset, although the two variables have strong linear correlation, there are 4 clusters on the graph.<jupyter_text>Dataset 4<jupyter_code>x4_mean = data_4.x.mean()
y4_mean = data_4.y.mean()
x4_std = data_4.x.std()
y4_std = data_4.y.std()
x4_min = data_4.x.min()
x4_max = data_4.x.max()
y4_min = data_4.y.min()
y4_max = data_4.y.max()
data4_corr = data_4.corr()['x']['y']
print('Data Summry of The Fourth Dataset:')
print('The mean of x variable is %f;' % x4_mean)
print('The mean of y variable is %f;' % y4_mean)
print('The standard deviation of x variable is %f;' % x4_std)
print('The standard deviation of y variable is %f;' % y4_std)
print('The range of x variable is [%f, %f];' % (x4_min, x4_max))
print('The range of y variable is [%f, %f];' % (y4_min, y4_max))
print('The correlation between the x and y variables are %f.' % data4_corr)
plt.scatter(data_4.x, data_4.y)
plt.title('Sctter Plot of Dataset 4')
print('One Sentence Description of Dataset 4:')
print('According to the coorelation value and the scatter plot of the fourth dataset,there is NO linear correlation, but the graph is the logo of Simon Fraser University, SFU.')<jupyter_output>One Sentence Description of Dataset 4:
According to the coorelation value and the scatter plot of the fourth dataset,there is NO linear correlation, but the graph is the logo of Simon Fraser University, SFU.<jupyter_text>Dataset 5<jupyter_code>x5_mean = data_5.x.mean()
y5_mean = data_5.y.mean()
x5_std = data_5.x.std()
y5_std = data_5.y.std()
x5_min = data_5.x.min()
x5_max = data_5.x.max()
y5_min = data_5.y.min()
y5_max = data_5.y.max()
data5_corr = data_5.corr()['x']['y']
print('Data Summry of The Fifth Dataset:')
print('The mean of x variable is %f;' % x5_mean)
print('The mean of y variable is %f;' % y5_mean)
print('The standard deviation of x variable is %f;' % x5_std)
print('The standard deviation of y variable is %f;' % y5_std)
print('The range of x variable is [%f, %f];' % (x5_min, x5_max))
print('The range of y variable is [%f, %f];' % (y5_min, y5_max))
print('The correlation between the x and y variables are %f.' % data5_corr)
plt.scatter(data_5.x, data_5.y)
plt.title('Sctter Plot of Dataset 5')
print('One Sentence Description of Dataset 5:')
print('According to the coorelation value and the scatter plot of the fourth dataset,there is NO linear correlation, but on vertical view, there are 5 clusters.')<jupyter_output>One Sentence Description of Dataset 5:
According to the coorelation value and the scatter plot of the fourth dataset,there is NO linear correlation, but on vertical view, there are 5 clusters.<jupyter_text>Dataset 6<jupyter_code>x6_mean = data_6.x.mean()
y6_mean = data_6.y.mean()
x6_std = data_6.x.std()
y6_std = data_6.y.std()
x6_min = data_6.x.min()
x6_max = data_6.x.max()
y6_min = data_6.y.min()
y6_max = data_6.y.max()
data6_corr = data_6.corr()['x']['y']
print('Data Summry of The Sixth Dataset:')
print('The mean of x variable is %f;' % x6_mean)
print('The mean of y variable is %f;' % y6_mean)
print('The standard deviation of x variable is %f;' % x6_std)
print('The standard deviation of y variable is %f;' % y6_std)
print('The range of x variable is [%f, %f];' % (x6_min, x6_max))
print('The range of y variable is [%f, %f];' % (y6_min, y6_max))
print('The correlation between the x and y variables are %f.' % data6_corr)
plt.scatter(data_6.x, data_6.y)
plt.title('Sctter Plot of Dataset 6')
print('One Sentence Description of Dataset 6:')
print('According to the coorelation value and the scatter plot of the fourth dataset,the two variables have strong linear correlation, but there are two outliers which are away from the regression line.')<jupyter_output>One Sentence Description of Dataset 6:
According to the coorelation value and the scatter plot of the fourth dataset,the two variables have strong linear correlation, but there are two outliers which are away from the regression line.<jupyter_text>Summary<jupyter_code>print('SUMMARY')
print()
print('Data Summry of The First Dataset:')
print('The mean of x variable is %f;' % x1_mean)
print('The mean of y variable is %f;' % y1_mean)
print('The standard deviation of x variable is %f;' % x1_std)
print('The standard deviation of y variable is %f;' % y1_std)
print('The range of x variable is [%f, %f];' % (x1_min, x1_max))
print('The range of y variable is [%f, %f];' % (y1_min, y1_max))
print('The correlation between the x and y variables are %f.' % data1_corr)
print('One Sentence Description of Dataset 1:')
print('According to the coorelation value and the scatter plot of the first dataset, the two variables have strong linear correlation.')
print()
print('Data Summry of The Second Dataset:')
print('The mean of x variable is %f;' % x2_mean)
print('The mean of y variable is %f;' % y2_mean)
print('The standard deviation of x variable is %f;' % x2_std)
print('The standard deviation of y variable is %f;' % y2_std)
print('The range of x variable is [%f, %f];' % (x2_min, x2_max))
print('The range of y variable is [%f, %f];' % (y2_min, y2_max))
print('The correlation between the x and y variables are %f.' % data2_corr)
print('One Sentence Description of Dataset 2:')
print('According to the coorelation value and the scatter plot of the second dataset, the two variables have strong linear correlation.')
print()
print('Data Summry of The Third Dataset:')
print('The mean of x variable is %f;' % x3_mean)
print('The mean of y variable is %f;' % y3_mean)
print('The standard deviation of x variable is %f;' % x3_std)
print('The standard deviation of y variable is %f;' % y3_std)
print('The range of x variable is [%f, %f];' % (x3_min, x3_max))
print('The range of y variable is [%f, %f];' % (y3_min, y3_max))
print('The correlation between the x and y variables are %f.' % data3_corr)
print('One Sentence Description of Dataset 3:')
print('According to the coorelation value and the scatter plot of the third dataset, although the two variables have strong linear correlation, there are 4 clusters on the graph.')
print()
print('Data Summry of The Fourth Dataset:')
print('The mean of x variable is %f;' % x4_mean)
print('The mean of y variable is %f;' % y4_mean)
print('The standard deviation of x variable is %f;' % x4_std)
print('The standard deviation of y variable is %f;' % y4_std)
print('The range of x variable is [%f, %f];' % (x4_min, x4_max))
print('The range of y variable is [%f, %f];' % (y4_min, y4_max))
print('The correlation between the x and y variables are %f.' % data4_corr)
print('One Sentence Description of Dataset 4:')
print('According to the coorelation value and the scatter plot of the fourth dataset,there is NO linear correlation, but the graph is the logo of Simon Fraser University, SFU.')
print()
print('Data Summry of The Fifth Dataset:')
print('The mean of x variable is %f;' % x5_mean)
print('The mean of y variable is %f;' % y5_mean)
print('The standard deviation of x variable is %f;' % x5_std)
print('The standard deviation of y variable is %f;' % y5_std)
print('The range of x variable is [%f, %f];' % (x5_min, x5_max))
print('The range of y variable is [%f, %f];' % (y5_min, y5_max))
print('The correlation between the x and y variables are %f.' % data5_corr)
print('One Sentence Description of Dataset 5:')
print('According to the coorelation value and the scatter plot of the fourth dataset,there is NO linear correlation, but on vertical view, there are 5 clusters.')
print()
print('Data Summry of The Sixth Dataset:')
print('The mean of x variable is %f;' % x6_mean)
print('The mean of y variable is %f;' % y6_mean)
print('The standard deviation of x variable is %f;' % x6_std)
print('The standard deviation of y variable is %f;' % y6_std)
print('The range of x variable is [%f, %f];' % (x6_min, x6_max))
print('The range of y variable is [%f, %f];' % (y6_min, y6_max))
print('The correlation between the x and y variables are %f.' % data6_corr)
print('One Sentence Description of Dataset 6:')
print('According to the coorelation value and the scatter plot of the fourth dataset,the two variables have strong linear correlation, but there are two outliers which are away from the regression line.')
print()
print('END OF SUMMARY')<jupyter_output>SUMMARY

Data Summry of The First Dataset:
The mean of x variable is 8.717696;
The mean of y variable is 12.522874;
The standard deviation of x variable is 12.155517;
The standard deviation of y variable is 6.639964;
The range of x variable is [-22.635878, 35.237055];
The range of y variable is [-2.565088, 27.384153];
The correlation between the x and y variables are 0.950326.
One Sentence Description of Dataset 1:
According to the coorelation value and the scatter plot of the first dataset, the two variables have strong linear correlation.

Data Summry of The Second Dataset:
The mean of x variable is -0.710787;
The mean of y variable is -0.748987;
The standard deviation of x variable is 11.481584;
The standard deviation of y variable is 6.257918;
The range of x variable is [-33.271113, 27.647000];
The range of y variable is [-18.621804, 14.147315];
The correlation between the x and y variables are 0.955153.
One Sentence Description of Dataset 2:
According to the coorelation value and [...]
{'fraction_non_alphanumeric': 0.1184411226357535, 'fraction_numerical': 0.031955460646735816, 'mean_word_length': 5.194142654700047, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '375397', 'license': 'Apache-2.0', 'n_tokens_mistral': 4774, 'n_tokens_neox': 4523, 'n_words': 1759, 'path': 'Exercise 5/summary.ipynb', 'repo_name': 'zeyongj/Computational-Data-Science', 'size': '13112'}