<jupyter_start><jupyter_text>Read in labels and performance data:<jupyter_code>data = pd.read_csv('labels_and_performance.csv')
data.head()<jupyter_output><empty_output><jupyter_text>First, look at the overall performance of the algorithm for the detection of pneumonia using confusion matrix. What is the specificity and sensitivity?<jupyter_code>## YOUR CODE HERE
tn, fp, fn, tp = sklearn.metrics.confusion_matrix(data.Pneumonia.values, data.algorithm_output.values, labels=[0,1]).ravel()
sens = tp/(tp+fn)
print('Sensitivity: ', sens)
spec = tn/(fp+tn)
print('Specificity: ', spec)<jupyter_output>Specificity:  0.8235294117647058<jupyter_text>Now, look at the algorithm's performance in the **presence of the other diseases**. What is the specificity and sensitivity?<jupyter_code>diseases = ['Atelectasis', 'Effusion', 'Pneumothorax', 'Infiltration', 'Cardiomegaly', 'Mass', 'Nodule']

for disease in diseases:
    tn, fp, fn, tp = sklearn.metrics.confusion_matrix(data[data[disease] == 1].Pneumonia.values,
                                                     data[data[disease] == 1].algorithm_output.values).ravel()
    sens, spec = tp/(tp+fn), tn/(tn+fp)
    
    print(disease)
    print('Sensitivity: ', sens)
    print('Specificity: ', spec,'\n')<jupyter_output>Atelectasis
Sensitivity:  0.782608695652174
Specificity:  0.8333333333333334 

Effusion
Sensitivity:  0.6521739130434783
Specificity:  0.8571428571428571 

Pneumothorax
Sensitivity:  0.8571428571428571
Specificity:  0.6666666666666666 

Infiltration
Sensitivity:  0.3888888888888889
Specificity:  0.0 

Cardiomegaly
Sensitivity:  0.8888888888888888
Specificity:  1.0 

Mass
Sensitivity:  0.9285714285714286
Specificity:  0.8666666666666667 

Nodule
Sensitivity:  1.0
Specificity:  0.5384615384615384
{'fraction_non_alphanumeric': 0.11286681715575621, 'fraction_numerical': 0.12020316027088036, 'mean_word_length': 5.177700348432055, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 5, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '427782', 'license': 'MIT', 'n_tokens_mistral': 779, 'n_tokens_neox': 625, 'n_words': 129, 'path': 'Applying AI to 2D Medical Imaging Data/Translating AI Algorithms For Clinical Settings with The FDA/Exercise - Algorithmic Limitations/Exercise.ipynb', 'repo_name': 'mayank1101/AI-for-Healthcare', 'size': '1772'}