<jupyter_start><jupyter_text>Report Title Team JUSTFAIR MSU  AbstractShort summary about the data and what it tells you about your project.   Data inputIn this section include code that reads in the csv file<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from scipy.spatial import distance_matrix

dist_df = pd.read_csv("2022_Project_distance_Matrix.csv")
dist_df<jupyter_output><empty_output><jupyter_text>Data CleaningIn this section provide code for converting the raw data into clean and usable data structures (if needed)  Data ModelingThis section that builds a model.  This doesn't have to be all that advanced. You should probably start with some simple statistical models such as an average distance matrix.<jupyter_code>dist_matrix = distance_matrix(dist_df.T, dist_df.T)

print(dist_matrix)

avg_qside = np.mean(dist_matrix[:-1, 9])
print(avg_qside)<jupyter_output>[[ 0.         17.69180601 20.59126028 23.70653918 27.01851217 31.76476035
  29.51270913 20.0748599  27.82085549 29.81610303]
 [17.69180601  0.         18.30300522 23.89560629 24.75883681 29.63106478
  29.05167809 20.92844954 27.11088342 29.79932885]
 [20.59126028 18.30300522  0.         29.15475947 26.83281573 36.5650106
  29.88310559 21.97726098 32.15587038 35.19943181]
 [23.70653918 23.89560629 29.15475947  0.         25.8069758  26.92582404
  27.76688675 24.87971061 26.90724809 27.69476485]
 [27.01851217 24.75883681 26.83281573 25.8069758   0.         33.48133809
  28.61817604 23.89560629 34.2636834  34.6554469 ]
 [31.76476035 29.63106478 36.5650106  26.92582404 33.48133809  0.
  26.83281573 26.64582519 15.39480432 14.07124728]
 [29.51270913 29.05167809 29.88310559 27.76688675 28.61817604 26.83281573
   0.         24.24871131 27.14774392 23.91652149]
 [20.0748599  20.92844954 21.97726098 24.87971061 23.89560629 26.64582519
  24.24871131  0.         23.2594067  25.33771892]
 [27.8208[...]<jupyter_text>Data VisualizationThis section make some graphs visualizing your results.  A distance matrix and/or network graph may be cool.  Think though the best way to show what you learned.<jupyter_code>import networkx as nx
import seaborn as sns

sns.heatmap(dist_matrix, cmap = "bone", xticklabels = dist_df.columns, yticklabels = dist_df.columns)
#plt.bar(range(10), dist_matrix[:, 9])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08162393162393163, 'fraction_numerical': 0.31367521367521367, 'mean_word_length': 4.2725225225225225, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '47317', 'license': 'MIT', 'n_tokens_mistral': 1333, 'n_tokens_neox': 889, 'n_words': 228, 'path': 'QSIDE_Report.ipynb', 'repo_name': 'fjevtic46/CMSE495_2022_Distance_Measures', 'size': '2340'}