<jupyter_start><jupyter_text>Survey Analysis Report Team Ford<NAME>, <NAME>, <NAME> and <NAME>  Sam's Analysis  AbstractShort summary about the data and what it tells you about your project.   Data inputIn this section include code that reads in the csv file<jupyter_code>import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import seaborn as sns
dist = pd.read_csv("2022_Project_distance_Matrix.csv")
dist.head()
# Team List
Teams =["ARFL","Argonne","Boeing","Delta Dental","Ford","Hope Village","Kellogg's","Neogen","Old Nation","Qside"]

# Broken up by team
#team = dist[dist['team'] == 0]
ARFL = dist[dist['ARFL'] == 0]
Argonne = dist[dist['Argonne'] == 0]

Boeing = dist[dist['Boeing'] == 0]
Delta = dist[dist['Delta Dental'] == 0]

Ford = dist[dist['Ford'] == 0] #6 members
Hope = dist[dist['Hope Village'] == 0]

Kellogg = dist[dist["Kellogg's"] == 0]
Neogen = dist[dist['Neogen'] == 0]

Old = dist[dist['Old Nation'] == 0]
Qside = dist[dist['Qside'] == 0] #2 members<jupyter_output><empty_output><jupyter_text>Data CleaningIn this section provide code for converting the raw data into clean and usable data structures (if needed) While there is no NA values there is a potental error in the collection of the data. As the team Qside had 2 entries, and Team ford had 6. And these are not the correct number of members for either.<jupyter_code>#no missing entries
dist.isna().sum()<jupyter_output><empty_output><jupyter_text>Data Modeling<jupyter_code>corr = dist.corr()
corr
# bad, should change the groupby. Many "NA" values
summary = dist.groupby(Teams).size().unstack()

from scipy.spatial.distance import squareform
from scipy.spatial.distance import pdist

pairwise = pd.DataFrame(
    squareform(pdist(summary)),
    columns = summary.index,
    index = summary.index)
#pairwise<jupyter_output><empty_output><jupyter_text>Data VisualizationThis section make some graphs visualizing your results.  A distance matrix and/or network graph may be cool.  Think though the best way to show what you learned. Here using Seaborn, it isn't isn't a distance matrix but I thought the visualization could be intresting. I was courious if it could just give a quick glance at the general likness of the projects to another. I would not recommend this model.The easist way to understand this visulization is the paler the color, the more similar the two groups are. The Darker the blue/red is is the more differnet they are.<jupyter_code>mask = np.triu(np.ones_like(corr, dtype=bool))
cmap = sns.diverging_palette(230, 20, as_cmap=True)
sns.heatmap(corr, mask=mask, cmap=cmap, vmax=.3, center=0,
            square=True, linewidths=.5, cbar_kws={"shrink": .5})
#closer to white is more similar to each other. Stronger color is less similar.<jupyter_output><empty_output><jupyter_text>ConclusionThis should be similar to the abstract but with more details. What can you conclude about your project from this data?  Sources[Pairwise Distance Matrix Tutoral](https://drawingfromdata.com/pandas/clustering/making-a-pairwise-distance-matrix-in-pandas.html)[Seaborn Correlation](https://seaborn.pydata.org/examples/many_pairwise_correlations.html)  Jyotsna's Analysis<jupyter_code>data = pd.read_csv("2022_Project_distance_Matrix.csv")<jupyter_output><empty_output><jupyter_text>Disagrement The heatmap below shows how much the people on a team agreed or disagreed on which projects were similar to their own. The darker colors mean higher disagreemnent and lighter colors mean lower disagreemnet.<jupyter_code>temp = {}

for a in data.columns.values:
    df = data[data[a] == 0]
    #temp[a] = np.array(df.sum(0)/df.shape[0])
    temp[a] = df.std(axis=0)
    
prob_matrix = pd.DataFrame(temp).T.fillna(0)

plt.figure(figsize=(12,10))
sns.heatmap(prob_matrix, cmap="Blues")
plt.title("DISAGREEMENT HEATMAP", fontsize = 20);
plt.xticks(fontsize=12);
plt.yticks(fontsize=12, rotation=360);<jupyter_output><empty_output><jupyter_text>Ishaan's Analysis<jupyter_code>import numpy as np
import scipy as sc
from scipy.spatial import distance_matrix
import pandas as pd
import seaborn as sns
import matplotlib as mpl
import matplotlib.pyplot as plt
from scipy.spatial.distance import pdist
from scipy.spatial.distance import squareform
data = pd.read_csv('2022_Project_distance_Matrix.csv')
ARFL = data[data['ARFL'] == 0]
ARFL
team_wise_data = pd.DataFrame(data=[], columns=data.columns)
for i in data.columns:
    temp_team = data[data[i] == 0]
    team_wise_data = pd.concat([team_wise_data, temp_team])
team_wise_data.head()
pairwise_top = pd.DataFrame(squareform(pdist(team_wise_data, metric='cosine')),
                            columns = team_wise_data.index,
                            index = team_wise_data.index)

plt.figure(figsize=(15,15))
sns.heatmap(pairwise_top, cmap='OrRd', linewidth=1)<jupyter_output><empty_output><jupyter_text>Nick's Analysis<jupyter_code>import pandas as pd 
import numpy as np
from scipy.spatial import distance_matrix
import sympy as sym
import seaborn as sns
import matplotlib.pyplot as plt
data = pd.read_csv('2022_Project_distance_Matrix.csv',header='infer')
data.head()
dist_mat = distance_matrix(data.T,data.T,p=2)
fig, ax = plt.subplots(figsize = (15,15))
ax = sns.heatmap(dist_mat,annot=True)
np.dot(data.T,data)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11048158640226628, 'fraction_numerical': 0.013220018885741265, 'mean_word_length': 5.603491271820449, 'pattern_counts': {'":': 1, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '415263', 'license': 'MIT', 'n_tokens_mistral': 1844, 'n_tokens_neox': 1749, 'n_words': 511, 'path': 'Ford-Analysis/Ford-Combined-Survey-Analysis.ipynb', 'repo_name': 'pathakis/CMSE495_2022_Distance_Measures', 'size': '5325'}