<jupyter_start><jupyter_text>Network Mining Lab 2: Similarity Notebook Author: <NAME>The similarity measure is the measure of how much aliketwo data objects are.§ It is a distance with dimensions representing features ofthe objects.§ The similarity is subjective and is highly dependent on thedomain and application.§ For example, two fruits are similar because of color or sizeor taste.§ Similarity are measured in the range 0 to 1, i.e [0,1]  Generating two random vectors<jupyter_code>import random
import numpy as np
random.seed(1)

vector_A = np.random.rand(5)
vector_A
vector_B = np.random.rand(5)
vector_B<jupyter_output><empty_output><jupyter_text>Manhattan Distance<jupyter_code>from math import*

def manhattan_distance(x,y):
    return sum(abs(a-b) for a,b in zip(x,y))

manhattan_distance(vector_A, vector_B)<jupyter_output><empty_output><jupyter_text>Euclidean Distance<jupyter_code>def euclidean_distance(x,y):
    return sqrt(sum(pow(a-b,2) for a, b in zip(x, y)))

euclidean_distance(vector_A, vector_B)<jupyter_output><empty_output><jupyter_text>Minkowski distance<jupyter_code>from decimal import Decimal

def nth_root(value, n_root):
    root_value = 1/float(n_root)
    return round (Decimal(value) ** Decimal(root_value),3)

def minkowski_distance(x,y,p):
    return nth_root(sum(pow(abs(a-b),p) for a,b in zip(x,y)), p)
                    
minkowski_distance(vector_A, vector_B, 4)<jupyter_output><empty_output><jupyter_text>Cosine Similarity<jupyter_code>def square_rooted(x):
    return round(sqrt(sum([a*a for a in x])),3)

def cosine_similarity(x,y):
    numerator = sum(a*b for a,b in zip(x,y))
    denominator = square_rooted(x)*square_rooted(y)
    return round(numerator/float(denominator),3)

cosine_similarity(vector_A, vector_B)<jupyter_output><empty_output><jupyter_text>Jaccard Similarity<jupyter_code>def jaccard_sim(im1, im2):
    im1 = np.asarray(im1).astype(np.bool)
    im2 = np.asarray(im2).astype(np.bool)
    intersection = np.logical_and(im1, im2)
    union = np.logical_or(im1, im2)
    return intersection.sum() / float(union.sum())

jaccard_sim(vector_A, vector_B)<jupyter_output><empty_output><jupyter_text>Weighted Jaccard Similarity<jupyter_code>def weighted_jaccrard(X,Y):
    numerator = sum(min(a,b) for a,b in zip(X,Y))
    denominator = sum(max(a,b) for a,b in zip(X,Y))
    return numerator/denominator

weighted_jaccrard(vector_A, vector_B)<jupyter_output><empty_output><jupyter_text>Using numpy<jupyter_code># Prepare 2 vectors of 100 dimensions
import scipy.spatial.distance as dist
import numpy as np

A = np.random.uniform(0, 10, 100)
B = np.random.uniform(0, 10, 100)
AA = np.random.randint(0, 2, 1000000)
BB = np.random.randint(0, 2, 1000000)<jupyter_output><empty_output><jupyter_text>Manhattan Distance<jupyter_code>dist.cityblock(A, B)<jupyter_output><empty_output><jupyter_text>Euclidean Distance<jupyter_code>dist.euclidean(A, B)<jupyter_output><empty_output><jupyter_text>Jaccard Distance<jupyter_code>dist.jaccard(A, B)<jupyter_output><empty_output><jupyter_text>Chebyshev Distance<jupyter_code>dist.chebyshev(A, B)<jupyter_output><empty_output><jupyter_text>Cosine Similarity<jupyter_code>dist.cosine(A, B)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11689526184538654, 'fraction_numerical': 0.01807980049875312, 'mean_word_length': 7.207161125319693, 'pattern_counts': {'":': 0, '<': 54, '<?xml version=': 0, '>': 54, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '515114', 'license': 'MIT', 'n_tokens_mistral': 1300, 'n_tokens_neox': 1220, 'n_words': 231, 'path': 'NetworkMining_Lab_02_Similarity.ipynb', 'repo_name': 'MarioTheOne/Network_Mining_Labs', 'size': '3213'}