<jupyter_start><jupyter_text>Friend Recommender 👩🏻‍💻🧑🏻‍💻Project to demonstrate stuff a data engineer should know. Find it here: https://github.com/zkan/simple-friend-recommender/<jupyter_code>import math
from typing import List

import pandas as pd
DAGS_FOLDER = '..'
distance_columns = "['data-engineer', 'data-scientist', 'cat', 'dog', 'mountain', 'sea', 'java', 'python', 'blackpink', 'gong-yoo']"
distance_columns = eval(distance_columns)<jupyter_output><empty_output><jupyter_text>Below is a simple euclidean distance function inspired by [Vik's article](https://www.dataquest.io/blog/k-nearest-neighbors-in-python/).<jupyter_code>def get_euclidean_distance(row: pd.Series, 
                           distance_columns: List[str], 
                           selected_person: pd.Series) -> float:
    inner_value = 0
    for k in distance_columns:
        inner_value += (row[k] - selected_person[k]) ** 2

    return math.sqrt(inner_value)
df = pd.read_csv(f'{DAGS_FOLDER}/survey_responses_transformed.csv')
for name in df.name.unique():
    print(f'Hey {name}, see the list of people who like something similar to you:')
    
    selected_person = df[df['name'] == name].iloc[0]    

    df['distance'] = df.apply(get_euclidean_distance, args=(distance_columns, selected_person,), axis='columns')
    sorted_df = df[['name', 'distance']].sort_values(by='distance')[:6]

    print(sorted_df)
    print('-' * 50)<jupyter_output>Hey zkan, see the list of people who like something similar to you:
     name  distance
0    zkan  0.000000
4  Python  1.414214
5    Mike  1.414214
2     ake  2.000000
6    Mark  2.000000
1  Beauti  2.449490
--------------------------------------------------
Hey Beauti, see the list of people who like something similar to you:
     name  distance
1  Beauti  0.000000
3    Milo  2.000000
0    zkan  2.449490
6    Mark  2.449490
4  Python  2.828427
5    Mike  2.828427
--------------------------------------------------
Hey ake, see the list of people who like something similar to you:
     name  distance
2     ake  0.000000
4  Python  1.414214
5    Mike  1.414214
0    zkan  2.000000
6    Mark  2.000000
3    Milo  2.449490
--------------------------------------------------
Hey Milo, see the list of people who like something similar to you:
     name  distance
3    Milo  0.000000
1  Beauti  2.000000
2     ake  2.449490
6    Mark  2.449490
4  Python  2.828427
5    Mike  2.828427
--------------[...]
{'fraction_non_alphanumeric': 0.16776045939294504, 'fraction_numerical': 0.081214109926169, 'mean_word_length': 2.469416785206259, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 9, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '36091', 'license': 'MIT', 'n_tokens_mistral': 934, 'n_tokens_neox': 816, 'n_words': 231, 'path': 'airflow/dags/notebooks/friend_recommender.ipynb', 'repo_name': 'zkan/simple-friend-recommender', 'size': '2438'}