<jupyter_start><jupyter_text>SummaryFace recognition models are regular convolutional neural networks models. They represent face photos as vectors. We find the distance between these two vectors to compare two faces. Finally, we classify two faces as same person whose distance is less than a threshold value.The question is that how to determine the threshold. In this notebook, we will find the best split point for a threshold.  Data set<jupyter_code># Ref: https://github.com/serengil/deepface/tree/master/tests/dataset
idendities = {
    "Angelina": ["img1.jpg", "img2.jpg", "img4.jpg", "img5.jpg", "img6.jpg", "img7.jpg", "img10.jpg", "img11.jpg"],
    "Scarlett": ["img8.jpg", "img9.jpg"],
    "Jennifer": ["img3.jpg", "img12.jpg"],
    "Mark": ["img13.jpg", "img14.jpg", "img15.jpg"],
    "Jack": ["img16.jpg", "img17.jpg"],
    "Elon": ["img18.jpg", "img19.jpg"],
    "Jeff": ["img20.jpg", "img21.jpg"],
    "Marissa": ["img22.jpg", "img23.jpg"],
    "Sundar": ["img24.jpg", "img25.jpg"]
}<jupyter_output><empty_output><jupyter_text>Positive samplesFind different photos of same people<jupyter_code>positives = []

for key, values in idendities.items():
    
    #print(key)
    for i in range(0, len(values)-1):
        for j in range(i+1, len(values)):
            #print(values[i], " and ", values[j])
            positive = []
            positive.append(values[i])
            positive.append(values[j])
            positives.append(positive)
positives = pd.DataFrame(positives, columns = ["file_x", "file_y"])
positives["decision"] = "Yes"<jupyter_output><empty_output><jupyter_text>Negative samplesCompare photos of different people<jupyter_code>samples_list = list(idendities.values())
negatives = []

for i in range(0, len(idendities) - 1):
    for j in range(i+1, len(idendities)):
        #print(samples_list[i], " vs ",samples_list[j]) 
        cross_product = itertools.product(samples_list[i], samples_list[j])
        cross_product = list(cross_product)
        #print(cross_product)
        
        for cross_sample in cross_product:
            #print(cross_sample[0], " vs ", cross_sample[1])
            negative = []
            negative.append(cross_sample[0])
            negative.append(cross_sample[1])
            negatives.append(negative)
        
negatives = pd.DataFrame(negatives, columns = ["file_x", "file_y"])
negatives["decision"] = "No"<jupyter_output><empty_output><jupyter_text>Merge Positives and Negative Samples<jupyter_code>df = pd.concat([positives, negatives]).reset_index(drop = True)
df.shape
df.decision.value_counts()
df.file_x = "deepface/tests/dataset/"+df.file_x
df.file_y = "deepface/tests/dataset/"+df.file_y<jupyter_output><empty_output><jupyter_text>DeepFace<jupyter_code>from deepface import DeepFace
instances = df[["file_x", "file_y"]].values.tolist()
model_name = "VGG-Face"
distance_metric = "cosine"
resp_obj = DeepFace.verify(instances, model_name = model_name, distance_metric = distance_metric)
distances = []
for i in range(0, len(instances)):
    distance = round(resp_obj["pair_%s" % (i+1)]["distance"], 4)
    distances.append(distance)
df["distance"] = distances<jupyter_output><empty_output><jupyter_text>Analyzing Distances<jupyter_code>tp_mean = round(df[df.decision == "Yes"].mean().values[0], 4)
tp_std = round(df[df.decision == "Yes"].std().values[0], 4)
fp_mean = round(df[df.decision == "No"].mean().values[0], 4)
fp_std = round(df[df.decision == "No"].std().values[0], 4)
print("Mean of true positives: ", tp_mean)
print("Std of true positives: ", tp_std)
print("Mean of false positives: ", fp_mean)
print("Std of false positives: ", fp_std)<jupyter_output>Mean of true positives:  0.2263
Std of true positives:  0.0744
Mean of false positives:  0.6489
Std of false positives:  0.12<jupyter_text>Distribution<jupyter_code>df[df.decision == "Yes"].distance.plot.kde()
df[df.decision == "No"].distance.plot.kde()<jupyter_output><empty_output><jupyter_text>Best Split Point<jupyter_code>from chefboost import Chefboost as chef
config = {'algorithm': 'C4.5'}
tmp_df = df[['distance', 'decision']].rename(columns = {"decision": "Decision"}).copy()
model = chef.fit(tmp_df, config)<jupyter_output>C4.5  tree is going to be built...
Accuracy:  98.66666666666667 % on  300  instances
finished in  3.5094187259674072  seconds<jupyter_text>Sigma<jupyter_code>sigma = 2
#2 sigma corresponds 95.45% confidence, and 3 sigma corresponds 99.73% confidence

#threshold = round(tp_mean + sigma * tp_std, 4)
threshold = 0.3147 #comes from c4.5 algorithm
print("threshold: ", threshold)
df[df.decision == 'Yes'].distance.max()
df[df.decision == 'No'].distance.min()<jupyter_output><empty_output><jupyter_text>Evaluation<jupyter_code>df["prediction"] = "No"
idx = df[df.distance <= threshold].index
df.loc[idx, 'prediction'] = 'Yes'
df.sample(5)
cm = confusion_matrix(df.decision.values, df.prediction.values)
cm
tn, fp, fn, tp = cm.ravel()
tn, fp, fn, tp
recall = tp / (tp + fn)
precision = tp / (tp + fp)
accuracy = (tp + tn)/(tn + fp +  fn + tp)
f1 = 2 * (precision * recall) / (precision + recall)
print("Precision: ", 100*precision,"%")
print("Recall: ", 100*recall,"%")
print("F1 score ",100*f1, "%")
print("Accuracy: ", 100*accuracy,"%")
df.to_csv("threshold_pivot.csv", index = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1608695652173913, 'fraction_numerical': 0.02948960302457467, 'mean_word_length': 4.951631046119235, 'pattern_counts': {'":': 10, '<': 40, '<?xml version=': 0, '>': 39, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '52956', 'license': 'MIT', 'n_tokens_mistral': 1963, 'n_tokens_neox': 1836, 'n_words': 400, 'path': 'tests/Fine-Tuning-Threshold.ipynb', 'repo_name': 'quangtm199/DeepFace', 'size': '5290'}