<jupyter_start><jupyter_text>Some coding in this kernel is inspired from [this really great work](https://www.kaggle.com/allunia/protein-atlas-exploration-and-baseline) Also, suggestions are a welcome so far they are not about my grammer and spelling(which I admit is borderline horrible and there's no spell check in this environment) LET's EDALet's see the total available images in the dataset<jupyter_code>df = pd.read_csv("../input/train.csv")
print("Total number of unique ids:",df.Id.count())
print("Total number of images:", df.Id.count()*4)
df.head(2)
labels = {
    0:  "Nucleoplasm",  
    1:  "Nuclear membrane",   
    2:  "Nucleoli",   
    3:  "Nucleoli fibrillar center",   
    4:  "Nuclear speckles",
    5:  "Nuclear bodies",   
    6:  "Endoplasmic reticulum",   
    7:  "Golgi apparatus",   
    8:  "Peroxisomes",   
    9:  "Endosomes",   
    10:  "Lysosomes",   
    11:  "Intermediate filaments",   
    12:  "Actin filaments",   
    13:  "Focal adhesion sites",   
    14:  "Microtubules",   
    15:  "Microtubule ends",   
    16:  "Cytokinetic bridge",   
    17:  "Mitotic spindle",   
    18:  "Microtubule organizing center",   
    19:  "Centrosome",   
    20:  "Lipid droplets",   
    21:  "Plasma membrane",   
    22:  "Cell junctions",   
    23:  "Mitochondria",   
    24:  "Aggresome",   
    25:  "Cytosol",   
    26:  "Cytoplasmic bodies",   
    27:  "Rods & rings"
}
for key in labels.keys():
    df[labels[key]] = 0
    
def filltargets(row):
    tar = row.Target.split(" ")
    for i in tar:
        col = labels[int(i)]
        row[str(col)]=1
    return row

df = df.apply(filltargets, axis=1)
df = df.drop(["Target"],axis = 1)
df.head(2)<jupyter_output><empty_output><jupyter_text>Frequency of labels in the data<jupyter_code>freq_df = df.drop(["Id"],axis=1).sum(axis=0).sort_values(ascending = True)
plt.figure(figsize=(15,15))
sns.barplot(y=freq_df.index.values, x=freq_df.values*100/len(df), order=freq_df.index,palette="Blues")<jupyter_output><empty_output><jupyter_text>Insights: * Nucleoplasm is the most common occurance (>40%) followed by Cytosol (>27%)* Plasma Membrane, Nucleoli, Mitochondria(Power-house of the cell) and other relatively bigger objects are present in a quantity that may not surprise any one who knows the basic cell structure.* The most interesting things in the distribution  of less frequent objects such as Rods and Rings, Microtubules Ends, Lysosomes etc as compared to those of Nucleoplasm. Hence a clear classs imbalance that can lead to harmful bias in model. This also gives us the idea about the kind of metric that we might endup using. From my very limited knowledge I think it should be F1.  The grouping of labelsLet's see the grouping of the least frequent variables first.<jupyter_code>def correlated_distribution(label):
    df_ = df[df[label]==1]
    freq_df = df_.drop(["Id",label],axis=1).sum(axis=0).sort_values(ascending = True)
    plt.figure(figsize=(5,5))
    plt.xlabel("Distribution for "+label)
    sns.barplot(y=freq_df.index.values, x=freq_df.values*100/len(df_), order=freq_df.index,palette="Blues")

correlated_distribution("Endosomes")
correlated_distribution("Rods & rings")
correlated_distribution("Peroxisomes")<jupyter_output><empty_output><jupyter_text>Let's see the grouping of most frequent labels<jupyter_code>correlated_distribution("Cytosol")
correlated_distribution("Mitochondria")<jupyter_output><empty_output><jupyter_text>The key insight here is that the more frequent labels occure in more spread-out manner, i.e, they seems to be partnering up with way more variables while the least frequent variables are loyal to a small number of labels.  Multilabel Analysis<jupyter_code>freq_df = pd.DataFrame()
freq_df["number_of_targets"] = df.drop(["Id"],axis=1).sum(axis=1).sort_values(ascending = True)
count_perc = np.round(100 * freq_df["number_of_targets"].value_counts() / freq_df.shape[0], 2)
plt.figure(figsize=(20,5))
sns.barplot(x=count_perc.index.values, y=count_perc.values, palette="Blues")
plt.xlabel("Number of targets per image")
plt.ylabel("% of data")<jupyter_output><empty_output><jupyter_text>Okay so majority is either single or double labeled.  Image combination and preview<jupyter_code>shape = (299,299)

def load_image(id,path="../input/train/"):
    global shape
    R = np.array(Image.open(path+id+'_red.png'))
    G = np.array(Image.open(path+id+'_green.png'))
    B = np.array(Image.open(path+id+'_blue.png'))
    Y = np.array(Image.open(path+id+'_yellow.png'))

    image = np.stack((
        R/2+Y/2,
        G/2+Y/2, 
        B),-1)

    image = cv2.resize(image, (shape[0], shape[1]))
    image = np.divide(image, 255)
    return image  

plt.figure(figsize=(25,15))
for n in range(4):
    idx = np.random.randint(0,20000,1)
    im_name = df.Id[idx[0]]
    im = load_image(im_name)
    plt.subplot(1,4,n+1)
    plt.imshow(im)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1300020312817388, 'fraction_numerical': 0.02396912451757059, 'mean_word_length': 4.216101694915254, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 27, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '461194', 'license': 'MIT', 'n_tokens_mistral': 1836, 'n_tokens_neox': 1682, 'n_words': 435, 'path': 'ExploratoryDataAnalysis.ipynb', 'repo_name': 'nikhilpandey360/AtlasHumanProtienClassification', 'size': '4923'}