<jupyter_start><jupyter_text>ExplanationThis code identifies bacterial colonies from Petri dish time-lapse images and based on the fluorescent signal it is able to compute a characteristic pattern signal for each fluorescent protein. With this information, the code classify each of the identified colonies. You can see the mathematical development at [Colony classificator maths](https://github.com/SynBioUC/FluoPi/blob/master/Maths/Colony%2Bclassificator%2Bmaths.pdf). Requirements - Image data - Have _fluopi_ module installed (open Terminal in Mac or command line in Windows and type  `pip install fluopi`; more info [here](https://packaging.python.org/tutorials/installing-packages/) about installing python packages) Analysis Loading packages<jupyter_code># import some useful packages to manage the data

%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import os.path

# modify some matplotlib parameters to manage the images in illustrator
matplotlib.rcParams['pdf.fonttype'] = 42
matplotlib.rcParams['ps.fonttype'] = 42
# Import the FluoPi module
import fluopi.analysis as flua
import fluopi.plotting as flup<jupyter_output><empty_output><jupyter_text>Loading DataIn this section you have to include the path of the data to be used. Be careful because it asumes you have the images in a subfolder in the same path as this notebook. If you have the images in another external folder you have to include the full path direction (e.g. _'C/myuser/.../my_images_folder'_)If you downloaded all the github repository you should have the images in the subfolder _'Images/'_ and then the path should be up and ready.Also, the example data can be obtained from the **_Files_** section in the [OSF FluoPi proyect](https://osf.io/dy6p2/). There you can download it from the *Google drive/Example_data/Classifier* folder.Moreover, there you can found other image series of your interest.In each _pathname_ is the complete path of the folder where images are stored.<jupyter_code>pathname1 = os.path.abspath('Images/Classifier/sfGFP_data/')
#refered to google drive folder Example_data\Classifier\sfGFP_data`
                      
pathname2 = os.path.abspath('Images/Classifier/CyOFP_data/')
#refered to google drive folder Example_data\Classifier\CyOFP_data

pathname3 = os.path.abspath('Images/Classifier/BeRFP_data/')
#refered to google drive folder Example_data\Classifier\BeRFP_data

pathname4 =  os.path.abspath('Images/Classifier/RGO_data/')
#refered to google drive folder Example_data\Classifier\RGO_data<jupyter_output><empty_output><jupyter_text>Then, make the image names:<jupyter_code>fname1 = os.path.join(pathname1, "image_%04d"+".jpg")
                      
fname2 = os.path.join(pathname2, "image_%04d"+".jpg")

fname3 = os.path.join(pathname3, "image_%04d"+".jpg")

fname4 =  os.path.join(pathname4, "image_%04d"+".jpg")<jupyter_output><empty_output><jupyter_text>And also save the number name of the last image<jupyter_code>lim1=int(os.listdir(pathname1)[-1].split('_')[-1].split('.')[0])
lim2=int(os.listdir(pathname2)[-1].split('_')[-1].split('.')[0])
lim3=int(os.listdir(pathname3)[-1].split('_')[-1].split('.')[0])
lim4=int(os.listdir(pathname4)[-1].split('_')[-1].split('.')[0])<jupyter_output><empty_output><jupyter_text>Let's check the data loading and channel structure (e.g. with the last frame image of the second data set)<jupyter_code>flup.plot_im_frame(fname2,lim2)
flup.plt_im_frame_channels(fname2,lim2)<jupyter_output><empty_output><jupyter_text>Count the files in each data set:<jupyter_code>imCount1=flua.count_files(pathname1,"jpg")
imCount2=flua.count_files(pathname2,"jpg")
imCount3=flua.count_files(pathname3,"jpg")
imCount4=flua.count_files(pathname4,"jpg")<jupyter_output><empty_output><jupyter_text>Now, we organize the data and split it into the different channels. Also, here you can define with frames of the whole original data you are going to use (i.e. by defining `frame > 1` you can select which data to use)<jupyter_code>DataS1={}
frames1 = 10   # use one image every 10 frames
DataS1['R'],DataS1['G'],DataS1['B']=flua.get_im_data(frames1,imCount1,fname1)
DataS1['Im']=fname1     # to store the related image source
DataS2={}
frames2 = 5     
DataS2['R'],DataS2['G'],DataS2['B']=flua.get_im_data(frames2,imCount2,fname2)
DataS2['Im']=fname2
DataS3={}
frames3 = 5     
DataS3['R'],DataS3['G'],DataS3['B']=flua.get_im_data(frames3,imCount3,fname3) 
DataS3['Im']=fname3
DataS4={}
frames4 = 5    
DataS4['R'],DataS4['G'],DataS4['B']=flua.get_im_data(frames4,imCount4,fname4)
DataS4['Im']=fname4<jupyter_output><empty_output><jupyter_text>Now we have each data serie on a dictionary, organized in arrays for each channel.  Background subtractionFirst step is to identify a good background area and compute his value over time:<jupyter_code>BG1=flua.bg_value(200,370,10,400,DataS1,imCount1)
BG2=flua.bg_value(180,360,420,750,DataS2,imCount2)
BG3=flua.bg_value(100,380,50,430,DataS3,imCount3)
BG4=flua.bg_value(450,670,260,420,DataS4,imCount4)<jupyter_output><empty_output><jupyter_text>We can subtract this background vector value to each frame on the data to eliminate the background effect<jupyter_code>DataSB1=flua.bg_subst(DataS1,BG1)
DataSB2=flua.bg_subst(DataS2,BG2)
DataSB3=flua.bg_subst(DataS3,BG3)
DataSB4=flua.bg_subst(DataS4,BG4)<jupyter_output><empty_output><jupyter_text>Get colony positionA good way to identify where colonies are situated is by summing the value of each pixel for all the channels and all the frames<jupyter_code>SumData1=flua.data_sum_time(DataSB1)
SumData2=flua.data_sum_time(DataSB2)
SumData3=flua.data_sum_time(DataSB3)
SumData4=flua.data_sum_time(DataSB4)<jupyter_output><empty_output><jupyter_text>Apply a filter to smooth the data This process let us reduce noise and improve border detection<jupyter_code>sDatS1,sDatSall1,SDatST1=flua.smooth_data(DataSB1,1.5)
sDatS2,sDatSall2,SDatST2=flua.smooth_data(DataSB2,1.5)
sDatS3,sDatSall3,SDatST3=flua.smooth_data(DataSB3,1.5)
sDatS4,sDatSall4,SDatST4=flua.smooth_data(DataSB4,1.5)<jupyter_output><empty_output><jupyter_text>Blob detection To properly identification, it's important to set the **threshold value** for each dataset manually (_i.e._ Define it based on the previous colormap)<jupyter_code>#blobD1 = flua.colony_blobs_id(sDatS1['G'],0.35,DataS1['Im']%(lim1-1)) 
#for green it was better to use only the green channel

blobD1 = flua.colony_blobs_id(sDatS1['G'],0.35,DataS1['Im']%(lim1),filename='Plots/Classifier/Blobs_sfGFP') 

# Threshold is the second argument e.g. 0.35
#blobD2 = flua.colony_blobs_id(sDatSall2,0.34,DataS2['Im']%(lim2-1))
blobD2 = flua.colony_blobs_id(sDatSall2,0.34,DataS2['Im']%(lim2),filename='Plots/Classifier/Blobs_CyOFP')
#blobD3 = flua.colony_blobs_id(sDatSall3,0.34,DataS3['Im']%(lim3-1))
blobD3 = flua.colony_blobs_id(sDatSall3,0.34,DataS3['Im']%(lim3),filename='Plots/Classifier/Blobs_BeRFP')
#blobD4 = flua.colony_blobs_id(sDatSall4,0.37,DataS4['Im']%(lim4-1))
blobD4 = flua.colony_blobs_id(sDatSall4,0.37,DataS4['Im']%(lim4),filename='Plots/Classifier/Blobs_MixFPs')
Rois1,RoisC1,NC1 = flua.obtain_rois(DataSB1,blobD1)
Rois2,RoisC2,NC2 = flua.obtain_rois(DataSB2,blobD2)
Rois3,RoisC3,NC3 = flua.obtain_rois(DataSB3,blobD3)
Rois4,RoisC4,NC4 = flua.obtain_rois(DataSB4,blobD4)<jupyter_output><empty_output><jupyter_text>ROIS Color Analysis  Obtaining a characteristic relation for the signal of each fluorescent proteinNow we are able to compute a characteristic channel signal relation for each fluorescent protein. To get a detailed mathematical explanation remit to You can see the mathematical development at [Colony classificator maths](https://github.com/SynBioUC/FluoPi/blob/master/Maths/Colony%2Bclassificator%2Bmaths.pdf). To accomplish it, we are going to get the characteristic relation from each of the experiment involving only one fluorescent protein (data sets 1 to 3).<jupyter_code>ChanX='G'
ChanY='R'
Xchan1,Ychan1=flup.rois_last_frame_2chan_plt(RoisC1,ChanX,ChanY,'Serie sfGFP')
Xchan2,Ychan2=flup.rois_last_frame_2chan_plt(RoisC2,ChanX,ChanY,'Serie BeRFP')
Xchan3,Ychan3=flup.rois_last_frame_2chan_plt(RoisC3,ChanX,ChanY,'Serie CyOFP')
Xchan4,Ychan4=flup.rois_last_frame_2chan_plt(RoisC4,ChanX,ChanY,'Serie Mix')
#fit1=flua.linear_fit(Xchan1[:,0],Ychan1[:,0])
fit1=flua.linear_fit(Xchan1[:,0],Ychan1[:,0],filename="Plots/Classifier/RG_fit1.pdf")
#fit2=flua.linear_fit(Xchan2[:,0],Ychan2[:,0])
fit2=flua.linear_fit(Xchan2[:,0],Ychan2[:,0],filename="Plots/Classifier/RG_fit2.pdf")
#fit3=flua.linear_fit(Xchan3[:,0],Ychan3[:,0])
fit3=flua.linear_fit(Xchan3[:,0],Ychan3[:,0],filename="Plots/Classifier/RG_fit3.pdf")<jupyter_output><empty_output><jupyter_text>Colony classificationAs we have found a characteristic linear relation for each fluorescent protein, we are able to test them in the _dataset 4_ with the mixed colonies and classify them based on the nearest linear relation to each of them.<jupyter_code>ClasS4,NClasS4=flua.colony_classifier([fit1,fit2,fit3],['GFP','CyOFP','BeRFP'],Xchan4,Ychan4)
Xmin=0
flup.plt_lin_fit(Xmin,90000,fit1,'g')
flup.plt_lin_fit(Xmin,90000,fit2,'y')
flup.plt_lin_fit(Xmin,30000,fit3,'r')
plt.plot(NClasS4['GFP'][0],NClasS4['GFP'][1],'go')
plt.plot(NClasS4['CyOFP'][0],NClasS4['CyOFP'][1],'yo')
plt.plot(NClasS4['BeRFP'][0],NClasS4['BeRFP'][1],'ro')
plt.savefig("Plots/Classifier/Classification.pdf", transparent=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12600934976625586, 'fraction_numerical': 0.04090522736931577, 'mean_word_length': 7.86346516007533, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 58, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '121591', 'license': 'MIT', 'n_tokens_mistral': 3696, 'n_tokens_neox': 3547, 'n_words': 710, 'path': 'FluOpti/Colony_classifier.ipynb', 'repo_name': 'jcahill/FluoPi', 'size': '9412'}